Skip to content

Introduction to Open Food Facts Infrastructure#

Welcome to the Open Food Facts Infrastructure documentation! This repository is dedicated to managing the infrastructure that powers Open Food Facts and its related projects. Our goal is to provide a reliable, scalable, and secure infrastructure to support the various services and applications that make up the Open Food Facts ecosystem.

Main Practices#

Proxmox#

Proxmox is an open-source server virtualization management solution that we use extensively in our infrastructure. It allows us to manage virtual machines (VMs) and containers (CTs) efficiently. Proxmox provides a web-based interface for easy management and monitoring of our virtualized environment.

Some software is installed / deployed in containers. Docker deployments normally use a VM.

For more details about our Proxmox setup and management, see Proxmox.

Server Configuration Management#

We manage server configurations using Git. Each server has a clone of this repository, and configuration files are symlinked to the appropriate locations. This allows us to track changes, maintain consistency, and easily roll back to previous configurations if needed. For more details, see Explanation on server configuration with git.

Continuous Integration and Continuous Delivery (CICD)#

We use a lot CICD process to automate the integration of code changes. This ensures all tests pass and desired quality standards are met. Our CICD process includes automated testing, building Docker containers, and, for some software, deploying to pre-production and production environments. For more information, see CICD.

Docker#

Docker is a one of the key component of our infrastructure. We use Docker to containerize our applications, ensuring consistency and ease of deployment. Docker Compose is used for orchestration, allowing us to manage multi-container applications with ease. For more details,

see:

Observability#

Observability (monitoring, alerts) allows us to monitor the health and performance of our systems, detect issues early, and gain insights into the behavior of our applications. We use a combination of tools and practices to achieve observability, including logging, metrics, and tracing. For more details, see Observability.

We also have a status page, driven by openfoodfacts-upptime and a specific repository regarding monitoring.

The How to handle alerts helps you diagnose and fix potential issues that created an alert.

ZFS#

We use a lot ZFS capabilities to store data on disk, and synchronize them accross servers thanks to Sanoid.

See:

  • ZFS Overview: An introduction to ZFS.
  • Sanoid: Information about using Sanoid for ZFS snapshots.

Our Servers#

Our infrastructure is hosted on multiple bare metal servers. They are grouped in different data centers, usually forming a proxmox cluster.

See Infrastructure Overview

Some servers are graciously sponsored by Fondation Free (at Scaleway), OVH and Moji

For more details about our servers and their configurations, see the following pages:

Virtual machines#

See Virtual machines

Production Architecture Overview#

Our production architecture consists of different services to run Open Food Facts and sibling projects. Those are deployed on different servers and different containers and virtual machines.

For a detailed overview of our production architecture, see Production Architecture.

Other tools supporting the community are deployed in containers, some times on the same servers.

Repository Structure#

The repository is organized into several directories, each serving a specific purpose:

  • confs/: Contains configuration files for various servers and services.
  • docker/: Contains Docker-related files, including Docker Compose configurations.
  • docs/: Contains documentation files, including this introduction.
  • docs/reports: contains post mortem or log of installations.
  • scripts/: Contains scripts for managing and maintaining the infrastructure.

Services#

Important Services#

Services Supporting the Main Open Food Facts Deployment#

  • Product Opener: Backend that powers the Open Food Facts website and mobile apps.
  • Open Food Facts Query: Service computing aggregations.
  • Postgres: Information about our PostgreSQL setup and management.
  • MongoDB: Information about our MongoDB setup and management.
  • Redis: Details about our Redis setup and management.
  • Producers SFTP: To push product updates on producer platform.
  • Folksonomy: User editable labels and values.

Tools for the Community#

Additional Resources#

Here are some additional resources that may be of interest:

Incident logs#

You are welcome to contribute#

We hope you find this documentation helpful and welcoming. If you have any questions or need further assistance, please feel free to reach out to us.

Happy contributing!