<<

NAME

ProductOpener::Products - create and save products

SYNOPSIS

ProductOpener::Products is used to create products and save them in Product Opener's database and file system.

    use ProductOpener::Products qw/:all/;

        my $product_ref = init_product($User_id, $Org_id, $code, $countryid);

        $product_ref->{product_name_en} = "Chocolate cookies";

        store_product("my-user", $product_ref, 'helpful comment');

DESCRIPTION

Revisions

When a product is saved, a new revision of the product is created. All revisions are saved in the file system:

products/[barcode path]/1.sto - first revision products/[barcode path]/2.sto - 2nd revision ... products/[barcode path]/product.sto - link to latest revision

The latest revision is stored in the products collection of the MongoDB database.

Completeness, data quality and edit history

Before a product is saved, this module compute the completeness and quality of the data, and the edit history.

FUNCTIONS

make_sure_numbers_are_stored_as_numbers ( PRODUCT_REF )

make_sure_numbers_are_stored_as_numbers() forces numbers contained in the product data to be stored as numbers (and not strings) in MongoDB.

Perl scalars are not typed, the internal type depends on the last operator used on the variable... e.g. if it is printed with a string concatenation, then it's converted to a string.

See https://metacpan.org/pod/JSON%3a%3aXS#PERL---JSON

assign_new_code ( )

assign_new_code() assigns a new unused code to store a new product that does not have a barcode.

        my ($code, $product_id) = assign_new_code();

Return values

A list with the new code and the corresponding product_id.

Caveats

Invalid codes

This function currently assign new codes in sequence starting from 2000000000001. We increment the number by 1 for each product (which means codes are not valid as the last digit is supposed to be the check digit), and check if there is already a product for that number.

Code conflicts

Codes starting with 2 are reserved for internal uses, there may be conflicts as other companies can use the same codes.

normalize_code()

normalize_code() this function normalizes the product code by: - running the given code through normalization method provided by GS1 to format a GS1 data string, or data URL to a GTIN, - keeping only digits and removing spaces/dashes etc., - normalizing the length by adding leading zeroes or removing the leading zero (in case of 14 digit codes)

Arguments

Product Code in the Raw form: $code

Return Values

Normalized version of the code

normalize_code_zeroes($code)

On disk, we store product files and images in directories named after the product code, and we add leading 0s to the paths. So we need to normalize the number of leading 0s of product codes, so that we don't have 2 products for codes that differ only by leading 0s.

This function normalizes the product code by: - removing leading zeroes, - adding leading zeroes to have at least 13 digits, - removing leading zeroes for EAN8s to keep only 8 digits

Note: this function adds leading 0s even if the GS1 code is not valid.

normalize_code_with_gs1_ai($code)

normalize_code_with_gs1_ai() this function normalizes the product code by: - running the given code through normalization method provided by GS1 to format a GS1 data string, or data URI to a GTIN, - keeping only digits and removing spaces/dashes etc., - normalizing the length by adding leading zeroes or removing the leading zero (in case of 14 digit codes)

Arguments

Product Code in the Raw form: $code

Return Values

Normalized version of the code, and GS1 AI data string of the code, if a valid GS1 string was given as the argument

is_valid_code($code)

is_valid_code() checks if the given code is a valid product code.

Arguments

Product Code: $code

Return Values

Boolean value indicating if the code is valid or not.

split_code()

split_code() this function splits the product code for determining the product path and the _id. product_path_from_id() utilizes this for the said purpose.

Arguments

Product Code: $code

Return Values

Code that has been split into 3 sections of three digits and one fourth section with the remaining digits. Example: 1234567890123 :- 123/456/789/0123

product_id_for_owner ( OWNER_ID, CODE )

product_id_for_owner() returns the product id associated with a product barcode.

If the products on the server are public, the product id is equal to the product code.

If the products on the server are private (e.g. on the platform for producers), the product_id is of the form user-[user id]/[code] or org-[organization id]/code.

The product id can be prefixed by a server id to indicate that is is on another server (e.g. Open Food Facts, Open Beauty Facts, Open Products Facts or Open Pet Food Facts) e.g. off:[code]

Parameters

Owner id

Code

Product barcode

In most cases, pass $Owner_id which is initialized by ProductOpener::Users::init_user()

  undef for public products
  user-[user id] or org-[organization id] for private products

Return values

The product id.

server_for_product_id ( $product_id )

Returns the server for the product, if it is not on the current server.

Parameters

$product_id

Product id of the form [code], [owner-id]/[code], or [server-id]:[code] or [server-id]:[owner-id]/[code]

Return values

undef is the product is on the current server, or server id of the server of the product otherwise.

data_root_for_product_id ( $product_id )

Returns the data root for the product, possibly on another server.

Parameters

$product_id

Product id of the form [code], [owner-id]/[code], or [server-id]:[code]

Return values

The data root for the product.

www_root_for_product_id ( $product_id )

Returns the www root for the product, possibly on another server.

Parameters

$product_id

Product id of the form [code], [owner-id]/[code], or [server-id]:[code]

Return values

The www root for the product.

product_path_from_id ( $product_id )

Returns the relative path for the product.

Parameters

$product_id

Product id of the form [code], [owner-id]/[code], or [server-id]:[code]

Return values

The relative path for the product.

product_path ( $product_ref )

Returns the relative path for the product.

Parameters

$product_ref

Product object reference.

Return values

The relative path for the product.

product_id_from_path ( $product_path )

Reverse of product_path_from_id.

There is no guarantee the result will be correct... but it's way faster than loading the sto !

init_product ( $userid, $orgid, $code, $countryid )

Initializes and return a $product_ref structure for a new product. If $countryid is defined and is not "en:world", then assign this country for the countries field. Otherwise, use the country associated with the ip address of the user.

Return Type

Returns a $product_ref structure

change_product_code ($product_ref, $new_code)

Utility function to change the barcode of a product. Fails and returns an error if the code is invalid, or if there is already a product with the new code.

Parameters

$product_ref

$new_code

Return value

If successful: undef If there was an error: invalid_code or new_code_already_exists

change_product_type ($product_ref, $new_product_type)

Utility function to change the product type of a product. Fails and returns an error if the product type is invalid.

Parameters

$product_ref

$new_product_type

Return value

If successful: undef If there was an error: invalid_product_type

compute_sort_keys ( $product_ref )

Compute sort keys that are stored in the MongoDB database and used to order results of queries.

last_modified_t - date of last modification of the product page

Used on the web site for facets pages, except the index page.

popularity_key - Popular and recent products

Used for the Personal Search project to provide generic search results that apps can personalize later.

store_product ($user_id, $product_ref, $comment)

Save changes of a product: - in a new .sto file on the disk - in MongoDB (in the products collection, or products_obsolete collection if the product is obsolete)

Before saving, some field values are computed, and product history and completeness is computed.

compute_data_sources ( $product_ref, $changes_ref )

Analyze the sources field of the product, as well as the changes to add to the data_sources field.

Sources allows to add some producers imports that were done before the producers platform was created.

The changes structure allows to add apps.

get_change_userid_or_uuid ( $change_ref )

For a specific change, analyze change identifiers (comment, user agent, userid etc.) to determine if the change was done through an app, the OFF userid, or an app specific UUID

Parameters

$change_ref

reference to a change record

Return value

The function returns by order of preference: - a real user userid if we have an userid which is not the userid of an app - an appid + app uuid (e.g. some-app.Z626FZF4RTFSG6) - an app userid if the app did not provide an app uuid - openfoodfacts-contributors

replace_user_id_in_product ( $product_id, $user_id, $new_user_id )

For a specific product, replace a specific user_id associated with changes (edits, new photos etc.) by another user_id.

This can be used when we want to rename a user_id, or when an user asks its data to be deleted: we can rename it to a generic user account like openfoodfacts-contributors.

Parameters

Product id

User id

New user id

find_and_replace_user_id_in_products ( $user_id, $new_user_id )

Find all products changed by a specific user_id, and replace the user_id associated with changes (edits, new photos etc.) by another user_id.

This can be used when we want to rename a user_id, or when an user asks its data to be deleted: we can rename it to a generic user account like openfoodfacts-contributors.

Parameters

User id

New user id

record_user_edit_type($users_ref, $user_type, $user_id)

Record that a user has made a change of a specific type to the product.

Parameters

$users_ref Structure that holds the records by type

For each type, there is a "list" array, and a "seen" hash

$user_type e.g. editors, photographers, weighers

$user_id

product_url ( $code_or_ref )

Returns a relative URL for a product on the website.

Parameters

Product code or reference to product object $code_or_ref

product_action_url ( $code, $action )

Returns a relative URL for an action on a product on the website.

This function is called by the web/panels/panel.tt.html template for knowledge panels that have associated actions.

Parameters

Product code or reference to product object $code_or_ref

process_product_edit_rules ($product_ref)

Process the edit_rules (see @edit_rules in in Config file).

where it applies

It applies in all API/form that edit the product. It applies to apply an image crop.

It does not block image upload.

edit_rules structure

conditions

Each condition is either a match on user_id or it's contrary user_id_not, or in_TAG_NAME_tags for a tag field to match a specific tag id.

Note that conditions are checked before editing the product !

actions

ignore alone, ignore every edits.

You can also have rules of the form ignore_FIELD and warn_FIELD which will ignore (or notify) edits on the specific field.

Note that ignore rules also create a notification.

For nutriments use nutriments_NUTRIMENT_NAME for FIELD.

You can guard the rule on the field with a condition: ignore_if_CONDITION_FIELD or warn_if_CONDITION_FIELD This time it's to check the value the user want's to add.

CONDITION is one of the following:

notifications

Notifications are email addresses to send emails, or "slack_CHANNEL_NAME" (warning currently channel name is ignored, we post to edit-alerts)

Example of an edit rule

compute_changes_diff_text ( $change_ref )

Generates a text that describes the changes made. The text is displayed in the edit history of products.

Arguments

$change_ref: reference to a change record

add_user_teams ( $product_ref )

If the user who add or edits the product belongs to one or more teams, add them to the teams_tags array.

Parameters

$product_ref

product_data_is_protected ( $product_ref )

Checks if the product data should be protected from edits. e.g. official producer data that should not be changed by anonymous users through the API

Product data is protected if it has an owner and if the corresponding organization has the "protect data" checkbox checked.

Parameters

$product_ref

Return values

- 1 if the data is protected - 0 if the data is not protected

delete_fields ($product_ref, $fields_ref)

Utility function to delete fields from a product_ref or a subfield.

Parameters

$product_ref

Reference to a complete product a subfield.

$fields_ref

An array of field names to remove.

add_images_urls_to_product ($product_ref, $target_lc, $specific_imagetype = undef)

Add fields like image_[front|ingredients|nutrition|packaging]_[url|small_url|thumb_url] to a product object.

If it exists, the image for the target language will be returned, otherwise we will return the image in the main language of the product.

Parameters

$product_ref

Reference to a complete product a subfield.

$target_lc

2 language code of the preferred language for the product images.

$specific_imagetype

Optional parameter to specify the type of image to add. Default is to add all types.

analyze_and_enrich_product_data ($product_ref, $response_ref)

This function processes product raw data to analyze it and enrich it. For instance to analyze ingredients and compute scores such as Nutri-Score and Eco-Score.

Parameters

$product_ref (input)

Reference to a product.

$response_ref (output)

Reference to a response object to which we can add errors and warnings.

is_owner_field($product_ref, $field)

Return 1 if the field value was provided by the owner (producer) and the field is not a tag field.

<<