ProductOpener::GS1 - convert data from GS1 Global Data Synchronization Network (GDSN) to the Open Food Facts format.
This module converts GDSN data that has previously been converted to JSON (either from XML with xml2json (e.g. Equadis data), or from JSON provided by a GDSN partner (e.g. CodeOnline) to the Open Food Facts CSV format.
The conversion is configured through the %gs1_to_off structure to indicate which source field maps to which target field.
And the %gs1_maps translate the GS1 specific identifiers (e.g. for allergens or units) to OFF identifiers.
GS1 uses many different codes for allergens, packaging etc. We need to map the different values to the values we use in the taxonomies.
The GS1 standards are evolving, so new values may be introduced over time.
Used to keep track of values we encounter in GS1 data for which we do not have a mapping.
Maps from GS1 to OFF
Defines the structure of the GS1 message data and how to extract the fields useful to create a message confirmation.
Defines the structure of the GS1 product data and how it maps to the OFF data.
%seen_fields and @fields are used to output the fields in the order of the GS1 to OFF mapping configuration
Used to assign a value to a field, and keep track of the order of the fields we are matching, so that we can output the fields in the same order when we export a CSV.
Recursive function to go through all first level keys of the $gs1_to_off_ref mapping. All values that can be assigned at that level are assigned, and if we need to go into a nested level, the function calls itself again.
The same hash reference is passed to recursive calls to the gs1_to_off function.
There are different ways to convert a XML document to a JSON data structure.
Historically, we used nodejs xml2json module to convert the GS1 XML to JSON.
Then we added support for CodeOnline JSON exports which used slightly different conversions.
In order to remove the dependency on nodejs, we are now supporting Perl's XML:XML2JSON module that results in different structures.
This function is a recursive function to make the output of Perl XML::XML2JSON similar to nodejs xml2json, as the GS1 module expects this format.
Difference:
XML2JSON creates a hash for simple text values. Text values of tags are converted to $t properties. e.g. <gtin>03449862093657</gtin>
becomes:
gtin: { $t: "03449865355608" },
This function converts those hashes with one single $t scalar values to a direct value.
gtin: "03449865355608"
Thus function converts the data for one or more products in the GS1 format converted to JSON. GS1 format is in XML, it needs to be transformed to JSON with xml2json first. In some cases, the conversion to JSON has already be done by a third party (e.g. the CodeOnline database from GS1 France).
Note: This function is recursive if there are child products.
One GS1 message can include 1 or more products, typically products that contain other products (e.g. a pallet of cartons of products).
Each product data will be added as one element (a hash ref) of the product data array.
For each product, the key of the hash is the name of the OFF csv field, and it is associated with the corresponding value for the product.
Each message will be added as one element (a hash ref) of the messages data array.
Read a GS1 message file in json format, convert the included products in the OFF format, and store the resulting products in the $products_ref array
The encapsulating GS1 message is added to the $messages_ref array
GS1 data pools (catalogs) send us GSDN Catalogue Item Notification (CIN) which are messages that contain the data for 1 product (and possibly sub-products).
The GS1 standard offers data recipient (such as Open Food Facts) to send back Catalogue Item Confirmation (CIC) messages to acknowledge the notification and give its status.
This function generates the CIC message corresponding to a CIN message.
See https://www.gs1.org/docs/gdsn/tiig/3_1/GDSN_Trade_Item_Implementation_Guide.pdf for more details.
The current time is passed as a parameter to the function. This is so that we can generate test confirmation messages which don't have a different content every time we run them.
Write all product data from the $products_ref array to a CSV file in OFF format.
Prints the entries for GS1 data types for which we do not have a corresponding OFF match, ordered by the number of occurrences in the GS1 data
Convert a GS1 XML file to a JSON file