fromager

Fromager is a tool for completely re-building a dependency tree of Python wheels from source.

The goals are to support guaranteeing

1. The binary package someone is installing was built from source in a known build environment compatible with their own environment
2. All of the package’s dependencies were also built from source -- any binary package installed will have been built from source
3. All of the build tools used to build these binary packages will also have been built from source

Modes

Fromager has different modes for bootstrapping and production builds. The bootstrap mode recursively processes all dependencies starting from the requirements specifications given to determine what needs to be built and what order to build it. The production build commands separate these steps and avoid recursive processing so each step can be performed in isolation.

Bootstrapping

The bootstrap command

• Creates an empty package repository for wheels
• Downloads the source distributions for the input packages and places them under sdists-repo/downloads/.
• Recurses through the dependencies
  • Firstly, any build system dependency specified in the pyproject.toml build-system.requires section as per PEP517
  • Secondly, any build backend dependency returned from the get_requires_for_build_wheel() build backend hook (PEP517 again)
  • Lastly, any install-time dependencies of the project as per the wheel’s core metadata Requires-Dist list.
• As each wheel is built, it is placed in a PEP503 "simple" package repository under wheels-repo/simple generated by pypi-mirror.
• The order the dependencies need to be built bottom-up is written to build-order.json.

Wheels are built by running pip wheel configured so it will only download dependencies from the local wheel repository. This ensures that all dependencies are being built in the correct order.

Production Builds

Production builds use separate commands for the steps described as part of bootstrapping, and accept arguments to control the servers that are used for downloading source or built wheels.

The download-source-archive command finds the source distribution for a specific version of a dependency on the specified package index and downloads it. It will be common to run this step with pypi.org, but for truly isolated and reproducible builds a private index server is more robust.

The prepare-source command unpacks the source archive downloaded from the previous step and applies any patches (refer to customization for details about patching).

The prepare-build command creates a virtualenv with the build dependencies for building the wheel. It expects a --wheel-server-url as argument to control where built wheels can be downloaded.

The build command prepares a wheel, compiling any extensions using the appropriate override environment settings (refer to customization for details about overrides).

Additional docs

• Package build customization instructions
• Developer instructions

What's with the name?

Python's name comes from Monty Python, the group of comedians. One of their skits is about a cheese shop that has no cheese in stock. The original Python Package Index (pypi.org) was called The Cheeseshop, in part because it hosted metadata about packages but no actual packages. The wheel file format was selected because cheese is packaged in wheels. And "fromager" is the French word for someone who makes or sells cheese.