AI & Machine Learning
Documentation on the machine learning models and prediction pipelines used in Open Prices.
| Model | What it does | Runs on | Provider | Stored as | More info |
|---|---|---|---|---|---|
| OCR | extracts the text of the image | every proof | Google Cloud Vision | .json.gz file next to the image |
unused |
| Proof classification | predicts the proof type (price tag, receipt...) | every proof | Triton (price_proof_classification) |
ProofPrediction (CLASSIFICATION) |
Hugging Face |
| Price tag detection | detects price tags on the image, and creates a PriceTag for each one |
PRICE_TAG proofs |
Triton (price_tag_detection) |
ProofPrediction (OBJECT_DETECTION) |
Hugging Face |
| Price tag classification | predicts the price tag quality (invalid, medium, high) | each price tag | Triton (price_tag_classification) |
PriceTagPrediction (PRICE_TAG_CLF) |
Hugging Face |
| Price tag extraction | extracts the product, price, discount... | each price tag (including invalid ones, temporarily) | Gemini | PriceTagPrediction (PRICE_TAG_EXTRACTION) |
|
| Price tag extraction (WIP) | same as above, with a fine-tuned Qwen3 model (to replace Gemini) | WIP, not integrated yet. Hugging Face | |||
| Receipt extraction | extracts the shop, date & items | RECEIPT proofs |
Gemini | ProofPrediction (RECEIPT_EXTRACTION) |
|
| Receipt anonymization | detects personal info (names, fidelity card ids) to hide with black boxes | RECEIPT proofs (draft only) |
PaddleOCR + LLM (OpenAI-compatible API) | ProofPrediction (RECEIPT_ANONYMIZATION), deleted once the proof is no longer a draft |
All models run asynchronously (Django Q tasks) when a proof is created (or when a price tag is added manually). On proof creation, they can be disabled with ENABLE_ML_PREDICTIONS (and OCR with ENABLE_OCR).
The code lives in open_prices/proofs/ml/ (and openfoodfacts-python)
Providers
- Triton: NVIDIA Triton Inference Server, self-hosted by Open Food Facts. It serves our own computer vision models (proof classification, price tag detection & classification). Configured with
TRITON_URI - Gemini: Google's LLM, called through its API to extract structured data (price tags & receipts). We plan to switch to an alternate LLM soon
- Google Cloud Vision: Google's OCR API
- PaddleOCR: OCR model, used with an LLM (through an OpenAI-compatible API) for receipt anonymization
Pages
- Proof Prediction Flow - Map of sync/async operations when a proof is created, covering type classification, price tag detection, extraction, and receipt processing