ProductOpener::Images - adds, processes, manages and displays product photos
ProductOpener::Images is used to: - upload product images - select and crop product images - run OCR on images - display product images
Product images are stored in html/images/products/[product barcode split with slashes]/
For each product, this directory contains:
Original images uploaded by users or imported
Same image saved as JPEG with specific settings, and after some minimal processing (auto orientation, removing EXIF data, flattening PNG images to remove transparency).
Those images are not displayed on the web site (except on the product edit form), but can be selected and cropped.
Same image saved with a maximum width and height of 100 and 400 pixels. Those thumbnails are used in the product edit form to show the available images.
OCR output from Google Cloud Vision.
When a new image is uploaded,
a symbolic link to it is created in /new_images.
This triggers a script to generate and save the OCR: run_cloud_vision_ocr.pl.
Cropped and selected image for the front of the product, the ingredients list, the nutrition facts table, and the packaging information / recycling instructions, in 4 different sizes (full size, 100 / 200 / 400 pixels maximum width or height).
The product revision is a number that is incremented for each change to the product (each image upload and each image selection are also individual changes that create a new revision).
The selected images are shown on the website, in the app etc.
When a new image is selected for a given field (e.g. ingredients) and language (e.g. French), the existing selected images are kept. (e.g. we can have ingredients_fr.21.100.jpg and a new ingredients_fr.28.100.jpg).
Previously selected images are shown only when people access old product revisions.
Cropping coordinates for all revisions are stored in the "images" field of the product, so we could regenerate old selected and cropped images on demand.
gif, jpeg, jpf, png, heic
Depending on the product type, different image types are allowed. e.g. food, pet food and beauty products have an "ingredients" image type.
We used to identify selected images with a field called "imagefield" which was of the form [image type]_[language code]. In some very old products revisions (e.g. from 2012), we had values with only the image type (e.g. "front").
This function splits the field name into its components, and is used to maintain backward compatibility.
$image_type e.g. "front" $image_lc e.g. "fr"
This function is used in the product edit form to display the select cropper with the images that are already uploaded.
This function is used to generate the code to initialize the select cropper in the product edit form with the images that are already uploaded.
This function is used to guess if an image is the front of the product, its list of ingredients, or the nutrition facts table, based on the filename.
It is used in particular for bulk upload of photos sent by manufacturers. The file names have many different formats, but they very often include the barcode of the product, and sometimes an indication of what the image is.
Producers are advised to use the [front|ingredients|nutrition|packaging]_[language code] format, but in practice we receive many other names.
The %file_names_to_imagefield_regexps structure below contains some patterns used to guess what the image is about.
This function generates resized images from the original image.
For uploaded images, we resize to 100 and 400 pixels maximum width or height.
For selected images, we resize to 100, 200, and 400 pixels maximum width or height.
The path to the image directory (e.g. html/images/products/1234567890123/).
The name of the image file (without the extension).
The source image object (Image::Magick).
A reference to a hash that will be filled with the sizes of the generated images.
An array of sizes to generate. The sizes are the maximum width or height of the image.
The function returns the error code from ImageMagick if there was an error writing the image.
Process an image uploaded to a product (from the web site, from the API, or from an import):
- Read the image - Create a JPEG version - Create resized versions - Store the image in the product data
Indicates what the image is and its language, or indicate a path to the image file (for imports and when uploading an image with a barcode)
Format: [front|ingredients|nutrition|packaging|other]_[2 letter language code]
Used to return the number identifying the image to the caller.
Used to return some debug information to the caller.
-2: imgupload field not set -3: we have already received an image with this file size -4: the image is too small -5: the image file cannot be read by ImageMagick
This function processes an image uploaded to a product using a file handle.
It is called by:
- the process_image_upload() function above when the image is uploaded with a CGI multipart form data encoded field (product form + API <= v2) - APIProductImagesUpload.pm for API v3
Used to return the number identifying the image to the caller.
Used to return some debug information to the caller.
-2: imgupload field not set -3: we have already received an image with this file size -4: the image is too small -5: the image file cannot be read by ImageMagick
This function removes images files from a product by a given prefix (matching uploaded images or selected images). The image files are moved to a deleted directory.
A reference to the product data structure.
The prefix of the images to be removed.
For uploaded images, the prefix is the imgid.
For selected images, the prefix is the image type and language code + the product revision. e.g. "ingredients_en.5" or "nutrition_fr.6".
This function deletes an uploaded image and its associated selected images.
Note: the corresponding product is not saved by this function, it should be saved by the caller. We do not save it in this function so that we can delete multiple images and save the updated product only once.
A reference to the product data structure.
The image id to be deleted.
1: success -1: image not found
This function moves images from one product to another, or to the trash.
The user id of the person moving the image.
The code of the product from which the image is moved.
The image ids to be moved, in a comma-separated list.
The product code to which the image is moved, or 'trash' if the image is deleted.
The owner id of the product from which the image is moved.
The function returns an error message if there was an error, or undef if the operation was successful.
This function checks if the image generation parameters are the same for two images.
It is useful to avoid selecting the same image with the same parameters twice.
A reference to the first image generation parameters.
A reference to the second image generation parameters.
1: the image generation parameters are the same
0: the image generation parameters are different
This function normalizes the generation_ref so that we store only useful values.
- If the image is not rotated, we don't store the angle. - If the image is not cropped, we don't store the coordinates. - If the image is not normalized, we don't store the normalize value. - If the image is not processed white magic, we don't store the white magic value.
If generation_ref is empty, we return an undef value
A reference to the image generation parameters.
A reference to the normalized generation_ref.
Or undef if the generation_ref is empty.
Select and possibly crop an uploaded image to represent the front, ingredients, nutrition or packaging image in a specific language.
1: crop done -1: image not found -2: image cannot be read
Return the URL of the image in the requested size.
Note: $image_ref in selected.images.[image type].[image code] does not contain the id field with the image type and language code (which are keys) It must be added to the image_ref before calling this function.
We return the image object in the best language available for the image type, in the order of preference: - $target_lc - main language of the product - English - any other available language (if any), in alphabetical order
- $product_ref: the product reference - $image_type: the image type (front, ingredients, nutrition, packaging) - $target_lc: the target language code - $image_lc_ref: a reference to return the language code of the image (optional)
- the image reference in the best language available, with an added "id" field containing the image type and language code (e.g. "front_en")
The language code of the best language is set in $image_lc_ref (if provided)
Add fields like image_[front|ingredients|nutrition|packaging]_[url|small_url|thumb_url] to a product object.
If it exists, the image for the target language will be returned, otherwise we will return the image in the main language of the product.
Reference to a complete product a subfield.
2 language code of the preferred language for the product images.
Optional parameter to specify the type of image to add. Default is to add all types.
Generates a data structure to display a product image.
The resulting data structure can be passed to a template to generate HTML or the JSON data for a knowledge panel.
- Reference to a data structure with needed data to display. - undef if no image is available for the requested image type
Generate the HTML code to display a product image.
- HTML code to display the image
Perform OCR for a specific image (either a source image, or a selected image) and return the results.
OCR can be performed with a locally installed Tesseract, or through Google Cloud Vision.
In the case of Google Cloud Vision, we also store the results of the OCR as a JSON file (requested through HTTP by Robotoff).
Either a number like 1, 2 etc. to perform the OCR on a source image (1.jpg, 2.jpg) or a field name in the form of [front|ingredients|nutrition|packaging]_[2 letter language code].
If $id is a field name, the last selected image for that field is used.
Field to update in the product object. e.g. ingredients_text_from_image, nutrition_text_from_image, packaging_text_from_image
Either "tesseract" or "google_cloud_vision"
A hash reference to store the results.
Call to Google Cloud vision API
This determine which detection will be performed. Remember each feature is a cost.
@CLOUD_VISION_FEATURES_FULL and @CLOUD_VISION_FEATURES_TEXT are two constant you can use.
A file where we write additional logs, specific to the service.
Return JSON content of the response.