<<

NAME

ProductOpener::Producers - functions specific to the platform for producers

SYNOPSIS

ProductOpener::Producers contains the functions specific to the producers platform:

- Functions to import CSV / Excel files, match column names, convert to OFF csv format - Minion tasks for import and export

DESCRIPTION

..

FUNCTIONS

load_csv_or_excel_file ( $file )

Load a CSV or Excel file in a Perl structure.

- CSV files should be in UTF-8 and separated with a comma. They are processed with Text::CSV. - Excel files are first converted to CSV with gnumeric's ssconvert.

Arguments

file name with absolute path

CSV or Excel file

Return values

A hash ref with:

headers

A reference to an array of header names.

rows

A reference to an array of rows, containing each an array of column values

init_fields_columns_names_for_lang ( $l )

Populates global $fields_columns_names_for_lang for the specified language.

The function creates a hash of all the possible localized column names that we can automatically recognize, and maps them to the corresponding field in OFF.

Nutrient column names are not included in the hash: they are matched by match_nutrient_column_name() from the nutrient synonyms and the possible suffixes computed by init_nutrients_columns_names_for_lang()

The column names are computed the first time they are needed for a language.

Arguments

$l - required

Language code (string)

Return value

Reference to the column names hash.

match_nutrient_column_name ( $l, $column_id )

Match a column id to a nutrient field, by splitting it into a nutrient synonym and a suffix.

If the column id can be split in different ways, the match with the last synonym (in the order of the nutrients taxonomy) and then the last suffix is selected.

Return value

Reference to a hash with the field and value_unit, or undef if the column id does not match a nutrient.

get_column_id_field ( $l, $column_id )

Match a column id to a field in a specific language.

Return value

Reference to a hash with the field (and optionally value_unit, lc, tag), or undef if there is no match.

Minion tasks

Minion tasks that can be added to the queue by standalone scripts or the web site, that are then executed by the minion-off and minion-off-pro daemons.

The daemons are configured in /etc/systemd/system

e.g. /etc/systemd/system/minion-off.service

[Unit] Description=off minion workers After=postgresql.service

[Service] Type=simple User=off WorkingDirectory=/srv/off/scripts Environment="PERL5LIB=/srv/off/lib/" ExecStart=/srv/off/scripts/minion_producers.pl minion worker -m production -q openfoodfacts.org KillMode=process

[Install] WantedBy=multi-user.target

<<