CI Caching & Containers

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the Python Reference bibliography.

CI machines and container builds start from nothing, so every run downloads and installs the whole dependency tree unless you tell them what they may reuse. This page covers the caches pip and Poetry keep, how GitHub Actions persists them, and the Dockerfile layouts that let Docker skip the dependency step when only your code changed. It was written against pip 26.2.2, Poetry 2.5.1, actions/setup-python v7.1.0 and actions/checkout v7.0.1. The cache commands ran on CPython 3.14.6, and both images below were built with Docker 29.9.1 from python:3.14-slim (CPython 3.14.9, pip 26.2.2) on 2026-10-22. The GitHub Actions workflows were not run here; they were checked line by line against the action’s documentation. Framework-specific images are covered in FastAPI: Docker and Kubernetes, Flask: Docker and Kubernetes and Django: Docker and Kubernetes.

What the tools cache

pip

pip keeps two caches under one directory (~/.cache/pip on Linux, ~/Library/Caches/pip on macOS, %LocalAppData%\pip\Cache on Windows; --cache-dir or PIP_CACHE_DIR moves it):

  • the HTTP cache (http-v2/ since pip 23.3) of index pages and downloaded files, which honours the index’s cache headers. Since 26.2 pip caches index pages according to Cache-Control instead of revalidating every time; a release published a minute ago may therefore not be visible until you pass --refresh-package <name>;

  • the wheel cache (wheels/) of wheels pip built locally from sdists, so the next install of the same sdist skips the build. Since 26.2 it is never used for local directory requirements.

python -m pip cache dir
python -m pip cache info
/root/.cache/pip
Package index page cache location (pip v23.3+): ~/.cache/pip/http-v2
Package index page cache location (older pips): ~/.cache/pip/http
Package index page cache size: 42.5 MB
Number of HTTP files: 624
Locally built wheels location: ~/.cache/pip/wheels
Locally built wheels size: 0 bytes
Number of locally built wheels: 0

pip cache list, pip cache remove <pattern> and pip cache purge manage it, and --no-cache-dir (PIP_NO_CACHE_DIR=1) disables it for one command. All of them are on pip: Wheels & Caching.

Source: pip: Cache management, pip: Avoiding caching.

Poetry

Poetry’s cache-dir (POETRY_CACHE_DIR; ~/.cache/pypoetry on Linux) holds one cache per source, the downloaded artefacts, and, by default, the project virtual environments (virtualenvs.path is {cache-dir}/virtualenvs):

poetry config cache-dir
poetry cache list
/root/.cache/pypoetry
PyPI
_default_cache

poetry cache clear PyPI --all empties one source’s cache. Caching cache-dir in CI therefore also caches the environments unless you set virtualenvs.in-project = true, which puts .venv in the checkout instead. See Poetry: Configuration.

Source: Poetry: cache-dir.

GitHub Actions

actions/setup-python caching

actions/setup-python (v7.1.0) can save and restore caches for you; caching is off unless you set cache. cache: pip caches pip’s global cache directory, and cache: poetry caches the virtual environment directories, one for each Poetry project found. The cache key includes the hash of a dependency file that the action searches for (requirements.txt or pyproject.toml for pip, poetry.lock for Poetry); cache-dependency-path names other files or glob patterns when the defaults do not fit. The action’s own example for Poetry installs Poetry with pipx before the setup-python step that enables cache: poetry; keep that order.

For the pip flavour of birdtally (not run here):

name: test
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10", "3.14"]
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-python@v7
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
          cache-dependency-path: pyproject.toml
      - run: python -m pip install --upgrade pip
      - run: python -m pip install -e . --group test
      - run: python -m pytest

And for the Poetry flavour (not run here):

name: test
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      POETRY_VIRTUALENVS_IN_PROJECT: "true"
    steps:
      - uses: actions/checkout@v7
      - name: Install Poetry
        run: pipx install poetry==2.5.1
      - uses: actions/setup-python@v7
        with:
          python-version: "3.14"
          cache: poetry
      - run: poetry install --no-interaction --with test
      - run: poetry run pytest

The action’s note applies: if the python-version you ask for does not satisfy the project’s requires-python, Poetry falls back to the runner’s own Python. The GitHub Actions Python tutorial and the Packaging User Guide still show older action majors (checkout@v6, setup-python@v6); the examples here use the current v7 majors.

Source: actions/setup-python: Caching packages, GitHub Docs: Building and testing Python.

Installing Poetry in CI

The official Poetry documentation has no GitHub Actions or CI-caching guide. What it does have is a "CI recommendations" tab in the installation page, which says:

  • pin the Poetry version in CI so that you upgrade only after validating a new release (the docs' examples still pin 2.0.0; use the version you tested, here 2.5.1);

  • prefer pipx (pipx install poetry==2.5.1) or a manual install into a venv;

  • if you use the official installer script, keep a copy of it under your control rather than downloading it on every run, and set POETRY_HOME=/opt/poetry so that the location is predictable:

export POETRY_HOME=/opt/poetry
python3 install-poetry.py --version 2.5.1
$POETRY_HOME/bin/poetry --version

The manual variant, which the Docker examples below use, is a venv for Poetry alone:

export POETRY_HOME=/opt/poetry
python3 -m venv $POETRY_HOME
$POETRY_HOME/bin/pip install poetry==2.5.1
$POETRY_HOME/bin/poetry --version

Poetry must never be installed into the project’s own environment; see Poetry: Installation & Self-Management.

Source: Poetry: CI recommendations.

Non-interactive runs

Both tools can stop to ask questions (credentials, confirmations). In CI, use --no-input (PIP_NO_INPUT=1) for pip and --no-interaction / -n (POETRY_NO_INTERACTION=1) for Poetry, so that a missing credential fails the job instead of hanging it.

Containers

The layer-cache rule

Docker reuses a layer when its instruction and its inputs are unchanged, and rebuilds every layer after the first one that changed. Dependencies change rarely and are slow to install; source files change constantly. So copy the files that describe the dependencies, install them, and only then copy the source:

flowchart TD A["FROM python:3.14-slim"] --> B["Tooling layer: venv, pinned Poetry"] B --> C["COPY pyproject.toml poetry.lock / requirements.txt"] C --> D["Install third-party dependencies (slow, cached)"] D --> E["COPY src/ (changes on every commit)"] E --> F["Install the project itself (fast)"] C -. "lock file changed" .-> D E -. "code changed: only this and later layers rebuild" .-> F

Use a .dockerignore that excludes .venv, dist and __pycache__, so that a local environment never ends up in the image.

Poetry: --no-root --no-directory, then the project

poetry install normally needs the project source, because it also installs the project itself. The Poetry FAQ’s answer is a two step install: --no-root skips the project and --no-directory skips path dependencies, so the first step needs only pyproject.toml and poetry.lock.

FROM python:3.14-slim

ENV POETRY_VERSION=2.5.1 \
    POETRY_HOME=/opt/poetry \
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    POETRY_NO_INTERACTION=1 \
    PIP_NO_CACHE_DIR=1
RUN python -m venv "$POETRY_HOME" \
    && "$POETRY_HOME/bin/pip" install "poetry==$POETRY_VERSION"

WORKDIR /app
# 1. Only the files that describe the dependencies: this layer is reused until they change.
COPY pyproject.toml poetry.lock ./
RUN "$POETRY_HOME/bin/poetry" install --only main --extras pretty --no-root --no-directory

# 2. The project itself.
COPY README.md LICENSE ./
COPY src/ ./src
RUN "$POETRY_HOME/bin/poetry" install --only main --extras pretty

ENV PATH="/app/.venv/bin:$PATH"
ENTRYPOINT ["birdtally"]

The first build of the Poetry flavour of birdtally installs the four pretty dependencies in step 5 and the project in step 8 (BuildKit output, trimmed to the step lines):

#9 [5/8] RUN "/opt/poetry/bin/poetry" install --only main --extras pretty --no-root --no-directory
#9 2.014 Installing dependencies from lock file
#9 2.068 Package operations: 4 installs, 0 updates, 0 removals
#9 2.069   - Installing markdown-it-py (4.2.0)
#9 2.070   - Installing mdurl (0.1.2)
#9 2.071   - Installing pygments (2.21.0)
#9 2.072   - Installing rich (15.0.0)
#9 DONE 3.6s
#10 [6/8] COPY README.md LICENSE ./
#10 DONE 0.0s
#11 [7/8] COPY src/ ./src
#11 DONE 0.0s
#12 [8/8] RUN "/opt/poetry/bin/poetry" install --only main --extras pretty
#12 0.936 Installing dependencies from lock file
#12 0.995 Installing the current project: birdtally (0.1.0)
#12 DONE 1.3s

After editing src/birdtally/tally.py and building again, everything up to the source copy comes from the cache:

#9 [5/8] RUN "/opt/poetry/bin/poetry" install --only main --extras pretty --no-root --no-directory
#9 CACHED
#10 [6/8] COPY README.md LICENSE ./
#10 CACHED
#11 [7/8] COPY src/ ./src
#11 DONE 0.1s
#12 [8/8] RUN "/opt/poetry/bin/poetry" install --only main --extras pretty
#12 1.086 Installing dependencies from lock file
#12 1.180 Installing the current project: birdtally (0.1.0)
#12 DONE 1.5s
docker run --rm -v "$PWD/survey.csv:/data/survey.csv:ro" birdtally:poetry /data/survey.csv --pretty
┏━━━━━━━━━━┳━━━━━━━┓
┃ Species  ┃ Count ┃
┡━━━━━━━━━━╇━━━━━━━┩
│ lapwing  │ 7     │
│ redshank │ 2     │
└──────────┴───────┘

Forgetting --no-root in the first step is the classic mistake. Poetry installs the dependencies and then tries to install the project from a directory that does not contain it yet, and the build stops:

Installing the current project: birdtally (0.1.0)

Error: The current project could not be installed: Readme path `/app/README.md` does not exist.
If you do not want to install the current project use --no-root.
If you want to use Poetry only for dependency management but not for packaging, you can disable package mode by setting package-mode = false in your pyproject.toml file.
If you did intend to install the current project, you may need to set `packages` in your pyproject.toml file.

The image is 346 MB, because Poetry and its dependencies stay inside it; a multi-stage build that copies only /app/.venv into a fresh python:3.14-slim avoids that.

Source: Poetry FAQ: Poetry busts my Docker cache.

pip: a hashed requirements file, then --no-deps

The pip flavour copies a hashed requirements.txt (here exported from the Poetry lock, see Lock Files & Reproducible Installs), installs it in hash-checking, binary-only mode, and then installs the project with --no-deps so that pip does not resolve its dependencies a second time:

FROM python:3.14-slim

ENV PIP_NO_CACHE_DIR=1 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PATH="/opt/venv/bin:$PATH"
RUN python -m venv /opt/venv

WORKDIR /app
# 1. Locked, hashed third-party dependencies: rebuilt only when requirements.txt changes.
COPY requirements.txt ./
RUN pip install --require-hashes --only-binary :all: -r requirements.txt

# 2. The project, without letting pip re-resolve its dependencies.
COPY pyproject.toml README.md LICENSE ./
COPY src/ ./src
RUN pip install --no-deps .

ENTRYPOINT ["birdtally"]
#9 [5/8] RUN pip install --require-hashes --only-binary :all: -r requirements.txt
#9 3.847 Successfully installed markdown-it-py-4.2.0 mdurl-0.1.2 pygments-2.21.0 rich-15.0.0
#9 DONE 4.1s
…
#12 [8/8] RUN pip install --no-deps .
#12 3.548 Successfully built birdtally
#12 3.583 Successfully installed birdtally-0.1.0
#12 DONE 3.7s
docker run --rm -v "$PWD/survey.csv:/data/survey.csv:ro" birdtally:pip /data/survey.csv --by site
{
  "north-marsh": 6,
  "east-dunes": 3
}

The image is 224 MB, the size of the base image plus the four wheels: there is no Poetry in it. PIP_NO_CACHE_DIR=1 keeps pip’s cache out of the image (a cache in a layer only makes the image bigger, because the layer is never reused by pip). If you want the cache between builds instead, use BuildKit’s cache mount (RUN --mount=type=cache,target=/root/.cache/pip pip install …) and drop PIP_NO_CACHE_DIR.

The python:3.14-slim image builds CPython itself under /usr/local and has no PEP 668 marker, so installing into its system Python would work; the venv keeps the application’s packages separate from the image’s own pip and makes a multi-stage copy easy. See Containers on the environments page.

Source: pip: Hash-checking mode, pip install: --no-deps, pip: Disabling caching.

Common mistakes

  • Caching the virtual environment across Python versions. Include the Python version in the cache key (setup-python does) or rebuild the venv.

  • COPY . . before installing dependencies. Every commit invalidates the dependency layer.

  • Installing an unpinned Poetry in CI. A new Poetry release can change the lock format or behaviour under you.

  • Running poetry lock in CI. CI should fail on a stale lock (poetry check --lock), not silently re-resolve.

  • Leaving .venv in the build context. It is copied into the image and may shadow the one built inside it.

References