Skip to content

Python Dependency Management

Purpose

Define strict, repeatable rules for Python dependency management to reduce upgrade risk while keeping dependencies current.

Scope

These rules apply to Python library dependencies managed with pyproject.toml, uv.lock, and requirements exports derived from the lock file.

Legacy dependency tooling is deprecated. Repositories migrating from non-uv tooling may temporarily retain legacy lockfiles, but must document the exception in their repository overlay and remove the legacy tooling once uv.lock is in use.

Sources of truth

  • pyproject.toml declares allowed dependency ranges.
  • uv.lock pins exact versions compiled from those ranges.
  • Requirements files are exported from uv.lock and must never drift from it.

Version specification rules

  • Default dependency specs in pyproject.toml use *.
  • Do not anchor to a major or minor series by default (including pre-1.0 dependencies).
  • Use constrained specs only when necessary and document them using the anchored dependency workflow.
  • Avoid patch-level pinning in pyproject.toml unless an explicit exception is approved.

Example default specification:

example-lib = "*"

Upgrade workflow

Patch-level

The first action after incrementing the application PATCH version is to refresh dependencies. vrg-release does this automatically: its last stage, update-deps, runs on the new develop version (see Dependency updates).

What the stage does for a Python repo:

  1. vrg-release has already incremented PATCH on develop.
  2. The python updater runs uv lock --upgrade, refreshing uv.lock within the existing constraints.
  3. If anything changed, the full validation suite (vrg-container-run -- vrg-validate) runs once.
  4. The change goes to develop as a PR labelled dependency-update, which merges once validation and the PR's checks pass. The lockfile versions then remain fixed for the rest of the PATCH cycle unless an exception is approved.

The update never changes explicit version constraints in pyproject.toml.

Requirements files exported from uv.lock are not regenerated by the built-in updater. A repo that keeps them should regenerate them in its dependency-update hook (for example, uv export --format requirements.txt --output-file requirements.txt).

Minor- or major-level

When incrementing the application MINOR or MAJOR version, perform the patch-level workflow and also attempt to move toward the latest available dependency releases when constraints have been tightened.

The candidates come from the held-back report, which every dependency update produces (in the dependency-update PR body and on the release tracking issue). It lists each directly declared requirement whose latest PyPI release falls outside its own specifier, whether it is constrained below the current minor series (for example, 1.1 when 1.2 exists) or anchored to a major version when a newer major exists (for example, <4 when 5.x is available). Relaxing a constraint is always a deliberate change; the update never does it.

Workflow:

  1. Take the minor-series candidates from the held-back report.
  2. For each dependency, relax the constraint individually and run the full validation and test suite.
  3. If multiple dependencies were relaxed successfully, run the full validation and test suite with all relaxations combined.
  4. Take the major-version candidates from the held-back report.
  5. For each major upgrade candidate, expand the constraint individually and run the full validation and test suite.
  6. If multiple major upgrades are viable, run the full validation and test suite with all major upgrades combined.

The expected steady state is unconstrained (*) unless a documented exception requires anchoring. If no newer major release exists, there is nothing to test.

If a dependency upgrade fails, determine root cause before deciding to pin or defer. A regression in the dependency is a valid reason to stay on the prior major version (for example, pinning pylint to the latest 3.x when 4.0.0 introduces a blocking bug).

In-cycle exception rules

Dependencies may change during a PATCH cycle only when necessary:

  • New functionality requires additional dependencies.
  • A dependency bug impacts the application and requires an upgrade or pin.

Each exception must:

  • include a written rationale in the pull request
  • minimize the scope of the dependency change
  • update uv.lock and any exported requirements
  • complete the full validation and test suite

Handling regressions and non-latest pins

When the dependency update's validation or merge fails, the dependency-update PR stays open, a fix task is filed under the repo's ad-hoc epic, and the next release refuses to start while that PR is open (see When the dependency update fails). Work the fix task on a branch cut from the failed update.

When a uv lock --upgrade introduces failures:

  • determine root cause before deciding to pin
  • do not assume the dependency is at fault
  • verify whether the application is compliant with the dependency's documented API and behavior

Pinning to a non-latest version is acceptable only when:

  • a regression or compatibility break in the dependency is verified, and no fix is available within the current cycle, or
  • the application depends on behavior removed or corrected upstream and a migration is required

If the application is at fault, fix the application and re-run the update instead of pinning.

Every pin must be accompanied by:

  • a written rationale and evidence
  • a clear exit condition and planned removal
  • a review at the next PATCH cycle

Anchored dependency documentation

When a dependency is anchored below the latest acceptable range, document it in two places:

  1. pyproject.toml comment immediately above the dependency specification, noting the latest version that failed, pointing to the dependency record, and linking the related GitHub issue.
  2. A dependency-specific record in docs/dependencies/ that captures failure evidence and preserves history across attempts.

Update both records every time an upgrade is tested and fails. Do not replace or delete prior failure evidence.

Example pyproject.toml comment:

# Anchor: 1.2.5 fails full test suite; issue https://github.com/<org>/<repo>/issues/123;
# see docs/dependencies/example-lib.md
example-lib = ">=1.1,<2.0"

Dependency record requirements:

  • One file per dependency: docs/dependencies/<dependency-name>.md.
  • Record the failing version and a concise description of the failure.
  • Capture failure evidence (test command, error excerpt, and context).
  • Append a new entry for each failed re-test.
  • Keep the latest attempted version in the pyproject.toml comment.

See Dependency anchor records for the required format.

Pre-release dependencies

Pre-release dependencies are allowed only as narrow, explicitly approved exceptions.

Rules:

  • Pre-releases are allowed only in non-production environments.
  • Production use requires an explicit, documented override (mechanism TBD).
  • Pre-releases must be specified as exact versions only; no ranges or pre-release wildcard allowances are permitted.
  • A pre-release dependency can only be added or removed with explicit human confirmation.
  • The dependency must be tracked in a GitHub issue. If no issue exists, create one before adding the dependency.
  • pyproject.toml must include a comment immediately above the dependency specification that includes the issue URL.

The start-of-cycle dependency update is automated and does not pause. Because a pre-release is pinned to an exact version, it appears in the held-back report once a newer release exists. When it does, confirm with a human owner that the pre-release dependency is still required.

When removing a pre-release dependency:

  • If the issue was created solely to track the dependency, close it.
  • If the issue tracks broader work, add a comment noting the removal.

Security-driven updates

If a vulnerability scan fails during a development cycle, update dependencies immediately:

  • Update pyproject.toml as required.
  • Regenerate uv.lock.
  • Re-export any requirements files derived from the lockfile.
  • Run a dependency audit against the exported requirements before committing.

Vulnerability audit scope

The vrg-validate audit stage, which is also the CI audit gate, checks every package pinned in uv.lock. That covers all dependency groups and extras, not only the dev environment that validation installs. A vulnerable pin in a docs group, or in any other group, fails the audit.

The stage exports the lock with uv export --frozen --all-groups --all-extras --no-emit-project --no-emit-local to .vergil/pip-audit-requirements.txt. It then runs pip-audit --requirement=<export> --disable-pip --require-hashes and writes the JSON report to pip-audit.json, which names each vulnerable package. The export leaves out the project itself and local path packages, because neither is published to an index. Any other requirement without a hash, such as a VCS or URL dependency, fails the audit instead of being skipped.

Only the vulnerability audit covers the whole lock. The install step, the uv sync --check drift check and the pip-licenses license check still run against the installed dev environment.

Locked dependency review

At the start of each upgrade cycle (PATCH, MINOR, or MAJOR), review any pinned or tightly constrained dependencies and confirm each pin is still required. The held-back report from the cycle-opening dependency update lists every directly declared requirement whose constraint keeps it below its latest release. Remove unnecessary pins as a follow-up change.

Enforcement

Violations are fatal exceptions that block merges, releases, and deployments.

CI must fail when:

  • uv.lock is out of sync with pyproject.toml
  • a dependency spec is constrained without documented justification
  • a dependency anchor lacks the required pyproject.toml comment
  • a dependency anchor lacks a record in docs/dependencies/
  • a pre-release dependency is used without an exact version pin
  • a pre-release dependency lacks an issue link in pyproject.toml
  • a pre-release dependency is included in production without a documented override
  • dependency updates occur after a vulnerability scan failure without a documented dependency audit
  • dependency updates occur without the required validation run

Examples (TODO)

  • Default * specifications and constrained exceptions.
  • Patch-cycle update checklist in practice.
  • Justified pinning due to upstream regression.
  • Dependency addition for new functionality.