pip-tools = pip-compile + pip-sync

A set of command line tools to help you keep your pip-based packages fresh, even when you’ve pinned them. You do pin them, right? (In building your Python application and its dependencies for production, you want to make sure that your builds are predictable and deterministic.)

This file is autogenerated by pip-compile with Python 3.10

by the following command:

pip-compile –output-file=requirements.txt pyproject.toml

# asgiref==3.6.0

# via django

django==4.1.7

# via my-cool-django-app (pyproject.toml)

sqlparse==0.4.3

# via django

$ pip-compile –extra dev -o dev-requirements.txt pyproject.toml #

This file is autogenerated by pip-compile with Python 3.10

by the following command:

pip-compile –extra=dev –output-file=dev-requirements.txt pyproject.toml

# asgiref==3.6.0

# via django

attrs==22.2.0

# via pytest

django==4.1.7

# via my-cool-django-app (pyproject.toml)

exceptiongroup==1.1.1

# via pytest

iniconfig==2.0.0

# via pytest

packaging==23.0

# via pytest

pluggy==1.0.0

# via pytest

pytest==7.2.2

# via my-cool-django-app (pyproject.toml)

sqlparse==0.4.3

# via django

tomli==2.0.1

# via pytest


This is great for both pinning your applications, but also to keep the CI
of your open-source Python package stable.

### Requirements from `requirements.in`

You can also use plain text files for your requirements (e.g. if you don't
want your application to be a package). To use a `requirements.in` file to
declare the Django dependency:

requirements.in

django


Now, run `pip-compile requirements.in`:

```console
$ pip-compile requirements.in
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile requirements.in
#
asgiref==3.6.0
    # via django
django==4.1.7
    # via -r requirements.in
sqlparse==0.4.3
    # via django

And it will produce your requirements.txt, with all the Django dependencies (and all underlying dependencies) pinned.

(updating-requirements)=

only update the django package

$ pip-compile –upgrade-package django

update both the django and requests packages

$ pip-compile –upgrade-package django –upgrade-package requests

update the django package to the latest, and requests to v2.0.0

$ pip-compile –upgrade-package django –upgrade-package requests==2.0.0


You can combine `--upgrade` and `--upgrade-package` in one command, to
provide constraints on the allowed upgrades. For example to upgrade all
packages whilst constraining requests to the latest version less than 3.0:

```console
$ pip-compile --upgrade --upgrade-package 'requests<3.0'

Using hashes

If you would like to use Hash-Checking Mode available in pip since version 8.0, pip-compile offers --generate-hashes flag:

$ pip-compile --generate-hashes requirements.in
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --generate-hashes requirements.in
#
asgiref==3.6.0 \
    --hash=sha256:71e68008da809b957b7ee4b43dbccff33d1b23519fb8344e33f049897077afac \
    --hash=sha256:9567dfe7bd8d3c8c892227827c41cce860b368104c3431da67a0c5a65a949506
    # via django
django==4.1.7 \
    --hash=sha256:44f714b81c5f190d9d2ddad01a532fe502fa01c4cb8faf1d081f4264ed15dcd8 \
    --hash=sha256:f2f431e75adc40039ace496ad3b9f17227022e8b11566f4b363da44c7e44761e
    # via -r requirements.in
sqlparse==0.4.3 \
    --hash=sha256:0323c0ec29cd52bceabc1b4d9d579e311f3e4961b98d174201d5622a23b85e34 \
    --hash=sha256:69ca804846bb114d2ec380e4360a8a340db83f0ccf3afceeb1404df028f57268
    # via django

Output File

To output the pinned requirements in a filename other than requirements.txt, use --output-file. This might be useful for compiling multiple files, for example with different constraints on django to test a library with both versions using tox:

$ pip-compile --upgrade-package 'django<1.0' --output-file requirements-django0x.txt
$ pip-compile --upgrade-package 'django<2.0' --output-file requirements-django1x.txt

Or to output to standard output, use --output-file=-:

$ pip-compile --output-file=- > requirements.txt
$ pip-compile - --output-file=- < requirements.in > requirements.txt

Forwarding options to `pip`

Any valid pip flags or arguments may be passed on with pip-compile’s --pip-args option, e.g.

$ pip-compile requirements.in --pip-args "--retries 10 --timeout 30"

This file is autogenerated by pip-compile with Python 3.10

by the following command:

./pipcompilewrapper

# asgiref==3.6.0

# via django

django==4.1.7

# via -r requirements.in

sqlparse==0.4.3

# via django


### Workflow for layered requirements

If you have different environments that you need to install different but
compatible packages for, then you can create layered requirements files and use
one layer to constrain the other.

For example, if you have a Django project where you want the newest `2.1`
release in production and when developing you want to use the Django debug
toolbar, then you can create two `*.in` files, one for each layer:

requirements.in

django<2.2


At the top of the development requirements `dev-requirements.in` you use `-c
requirements.txt` to constrain the dev requirements to packages already
selected for production in `requirements.txt`.

dev-requirements.in

-c requirements.txt django-debug-toolbar<2.2


First, compile `requirements.txt` as usual:

$ pip-compile #

This file is autogenerated by pip-compile with Python 3.10

by the following command:

pip-compile

# django==2.1.15

# via -r requirements.in

pytz==2023.3

# via django


Now compile the dev requirements and the `requirements.txt` file is used as
a constraint:

```console
$ pip-compile dev-requirements.in
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile dev-requirements.in
#
django==2.1.15
    # via
    #   -c requirements.txt
    #   django-debug-toolbar
django-debug-toolbar==2.1
    # via -r dev-requirements.in
pytz==2023.3
    # via
    #   -c requirements.txt
    #   django
sqlparse==0.4.3
    # via django-debug-toolbar

As you can see above, even though a 2.2 release of Django is available, the dev requirements only include a 2.1 version of Django because they were constrained. Now both compiled requirements files can be installed safely in the dev environment.

To install requirements in production stage use:

$ pip-sync

You can install requirements in development stage by:

$ pip-sync requirements.txt dev-requirements.txt

Version control integration

You might use pip-compile as a hook for the pre-commit. See pre-commit docs for instructions. Sample .pre-commit-config.yaml:

repos:
  - repo: https://github.com/jazzband/pip-tools
    rev: 7.4.1
    hooks:
      - id: pip-compile

You might want to customize pip-compile args by configuring args and/or files, for example:

repos:
  - repo: https://github.com/jazzband/pip-tools
    rev: 7.4.1
    hooks:
      - id: pip-compile
        files: ^requirements/production\.(in|txt)$
        args: [--index-url=https://example.com, requirements/production.in]

If you have multiple requirement files make sure you create a hook for each file.

repos:
  - repo: https://github.com/jazzband/pip-tools
    rev: 7.4.1
    hooks:
      - id: pip-compile
        name: pip-compile setup.py
        files: ^(setup\.py|requirements\.txt)$
      - id: pip-compile
        name: pip-compile requirements-dev.in
        args: [requirements-dev.in]
        files: ^requirements-dev\.(in|txt)$
      - id: pip-compile
        name: pip-compile requirements-lint.in
        args: [requirements-lint.in]
        files: ^requirements-lint\.(in|txt)$
      - id: pip-compile
        name: pip-compile requirements.in
        args: [requirements.in]
        files: ^requirements\.(in|txt)$

Maximizing reproducibility

pip-tools is a great tool to improve the reproducibility of builds. But there are a few things to keep in mind.

pip-compile will produce different results in different environments as described in the previous section.
pip must be used with the PIP_CONSTRAINT environment variable to lock dependencies in build environments as documented in #8439.
Dependencies come from many sources.

Continuing the pyproject.toml example from earlier, creating a single lock file could be done like:

$ pip-compile --all-build-deps --all-extras --output-file=constraints.txt --strip-extras pyproject.toml
#
# This file is autogenerated by pip-compile with Python 3.9
# by the following command:
#
#    pip-compile --all-build-deps --all-extras --output-file=constraints.txt --strip-extras pyproject.toml
#
asgiref==3.5.2
    # via django
attrs==22.1.0
    # via pytest
backports-zoneinfo==0.2.1
    # via django
django==4.1
    # via my-cool-django-app (pyproject.toml)
editables==0.3
    # via hatchling
hatchling==1.11.1
    # via my-cool-django-app (pyproject.toml::build-system.requires)
iniconfig==1.1.1
    # via pytest
packaging==21.3
    # via
    #   hatchling
    #   pytest
pathspec==0.10.2
    # via hatchling
pluggy==1.0.0
    # via
    #   hatchling
    #   pytest
py==1.11.0
    # via pytest
pyparsing==3.0.9
    # via packaging
pytest==7.1.2
    # via my-cool-django-app (pyproject.toml)
sqlparse==0.4.2
    # via django
tomli==2.0.1
    # via
    #   hatchling
    #   pytest

Some build backends may also request build dependencies dynamically using the get_requires_for_build_ hooks described in PEP 517 and PEP 660. This will be indicated in the output with one of the following suffixes:

(pyproject.toml::build-system.backend::editable)
(pyproject.toml::build-system.backend::sdist)
(pyproject.toml::build-system.backend::wheel)

Other useful tools

pip-compile-multi - pip-compile command wrapper for multiple cross-referencing requirements files.
pipdeptree to print the dependency tree of the installed packages.
requirements.in/requirements.txt syntax highlighting:
- requirements.txt.vim for Vim.
- Python extension for VS Code for VS Code.
- pip-requirements.el for Emacs.

Deprecations

This section lists pip-tools features that are currently deprecated.

In the next major release, the --allow-unsafe behavior will be enabled by default (https://github.com/jazzband/pip-tools/issues/989). Use --no-allow-unsafe to keep the old behavior. It is recommended to pass --allow-unsafe now to adapt to the upcoming change.
The legacy resolver is deprecated and will be removed in future versions. The new default is --resolver=backtracking.
In the next major release, the --strip-extras behavior will be enabled by default (https://github.com/jazzband/pip-tools/issues/1613). Use --no-strip-extras to keep the old behavior.

A Note on Resolvers

You can choose from either default backtracking resolver or the deprecated legacy resolver.

The legacy resolver will occasionally fail to resolve dependencies. The backtracking resolver is more robust, but can take longer to run in general.

You can continue using the legacy resolver with --resolver=legacy although note that it is deprecated and will be removed in a future release.

pip-tools

pip-tools = pip-compile + pip-sync

This file is autogenerated by pip-compile with Python 3.10

by the following command:

pip-compile –output-file=requirements.txt pyproject.toml

This file is autogenerated by pip-compile with Python 3.10

by the following command:

pip-compile –extra=dev –output-file=dev-requirements.txt pyproject.toml

requirements.in

only update the django package

update both the django and requests packages

update the django package to the latest, and requests to v2.0.0

Using hashes

Output File

Forwarding options to pip

This file is autogenerated by pip-compile with Python 3.10

by the following command:

./pipcompilewrapper

requirements.in

dev-requirements.in

This file is autogenerated by pip-compile with Python 3.10

by the following command:

pip-compile

Version control integration

Maximizing reproducibility

Other useful tools

Deprecations

A Note on Resolvers

Articles

Forwarding options to `pip`