NAME

ProductOpener::Import - import products data in CSV format and products photos

SYNOPSIS

ProductOpener::Import is used to import product data in the Open Food Facts CSV format and associated product photos.

    use ProductOpener::Import qw/:all/;
        import_csv_file( {
                user_id => "user",
                org_id => "organization",
                csv_file => "/path/to/product_data.csv",
        });

This module is used to import product data provided by manufacturers on the producers platform: the data from manufacturers (in CSV or Excel files) is first converted to the Open Food Facts CSV format, then imported with import_csv_file.

It is also used to export product data from the producers platform to the public database. The data is first exported from the producers platform with the ProductOpener::Export module, and then imported in the public database with the import_csv_file function.

In the producers platform, the import_csv_file function is executed through a Minion worker.

It is also used in the scripts/import_csv_file.pl script.

DESCRIPTION

FUNCTIONS

preprocess_field($imported_product_ref, $product_ref, $field, $yes_regexp, $no_regexp)

Do some pre-processing on input field values:

- Fields suffixed with _if_not_existing are loaded only if the product does not have an existing value - Special handling of tags fields: - Empty values are skipped - For labels and categories, we can have columns like labels:Bio with values like 1, Y, Yes - [tags type]_if_match_in_taxonomy : contains candidate values that we import only if we have a matching taxonomy entry - Values for multiple columns for the same tag field. e.g. brands, brands.2, brands.3 etc. are concatenated with a comma

set_field_value($args_ref, $imported_product_ref, $product_ref, $field, $yes_regexp, $no_regexp, $stats_ref, $modified_ref, $differing_ref)

Update a product field value in $product_ref based on the value in the imported product $imported_product_ref

Return an incremented $modified value if a change was made.

import_csv_file ( ARGUMENTS )

import_csv_file() imports product data in the Open Food Facts CSV format and associated product photos.

Arguments

Arguments are passed through a single hash reference with the following keys:

user_id - required

User id to which the changes (new products, added or changed values, new images) will be attributed.

If the user_id is 'all', the change will be attributed to the org of the product. (e.g. when importing products from the producers database to the public database)

org_id - optional

Organization id to which the changes (new products, added or changed values, new images) will be attributed.

owner_id - optional

For databases with private products, owner (user or org) that the products belong to. Values are of the form user-[user id] or org-[organization id].

If not set, for databases with private products, it will be constructed from the user_id and org_id parameters.

The owner can be overrode if the CSV file contains a org_name field. In that case, the owner is set to the value of the org_name field, and a new org is created if it does not exist yet.

csv_file - required

Path and file name of the CSV file to import.

The CSV file needs to be in the Open Food Facts CSV format, encoded in UTF-8 with tabs as separators.

global_values - optional

Hash ref that specifies fields and values that will be used as default values.

If the CSV contains a non-empty value for a field, the value from the CSV file is used.

images_dir - optional

Path to a directory that contains images for the products.

comment - optional

Comment that will be saved in the product history.

no_source - optional

Indicates that there should not be a data source attribution.

source_id - required (unless no_source is indicated)

Source id for the data and images.

source_name - required (unless no_source is indicated)

Name of the source.

source_url - required (unless no_source is indicated)

URL for the source.

source_licence - optional (unless no_source is indicated)

License that the source data is available in.

source_licence_url - optional (unless no_source is indicated)

URL for the license.

manufacturer - optional

A positive value indicates that the data is imported from the manufacturer of the products.

test - optional

Compute statistics on the number of products the import would add or change, but do not actually import and save the changes.

skip_if_not_code - optional

Only import one product with the corresponding code.

skip_not_existing_products - optional

Only import product data if the product already exists in the database. This can be useful when we have very sparse data to import (e.g. a list of codes of products sold in a given store chain), and we do not want to create products when we have no other existing data.

skip_products_without_info - optional

Do not import products when we do not have info (product name or brands)

skip_products_without_images - optional

Do not import products if there are no corresponding images in the directory specified by the images_dir argument/

skip_existing_values - optional

If a product already has existing values for some fields, do not overwrite it with values from the CSV file.

only_select_not_existing_images - optional

If the product already has an image for front, ingredients or nutrition in a given language, do not overwrite it with an image from the import. The image will still be uploaded and added to the product, but it will not be selected.

update_export_status_for_csv_file( ARGUMENTS )

Once products from a CSV file have been exported from the producers platform and imported on the public platform, update_export_status_for_csv_file() marks them as exported on the producers platform.