Skip to content

fix(deps): clear the remaining uv.lock highs — cryptography override, seed resolution (#2120) - #2121

Merged
Gregory91G merged 2 commits into
security-hardeningfrom
fix/uv-resolution-remaining-highs
Aug 3, 2026
Merged

fix(deps): clear the remaining uv.lock highs — cryptography override, seed resolution (#2120)#2121
Gregory91G merged 2 commits into
security-hardeningfrom
fix/uv-resolution-remaining-highs

Conversation

@Gregory91G

@Gregory91G Gregory91G commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Closes #2120. Clears the last ten uv.lock findings the scan reports at GitHub-high severity —
the remainder after #2083 covered nltk and langchain-core.

How this is measured

uv.lock is not tracked (.gitignore:24). .github/workflows/trivy.yml regenerates every lock
with uv lock before scanning, so the findings describe uv's resolution of the committed
pyproject.toml files. Every number here comes from running that same step locally — same pinned
uv and trivy images, same flags.

One caveat worth stating: uv lock is incremental and keeps whatever an existing lock already
pins. CI locks a clean checkout; locally the lock must be deleted first or the result is fiction.

1. cryptography in the connector harness — 2 high

airbyte-cdk 6.60.x resolves cryptography 44.0.3: a Bleichenbacher timing oracle in PKCS#1 v1.5
decryption (CVE-2026-26007, fixed in 46.0.5) and GHSA-537c-gmf6-5ccf (fixed in 48.0.1). One
more entry in the existing override list; resolves 49.0.0.

2. deploy/seed — mypy and dbt-core could not share a pathspec, 8 high

The eight findings name jinja2 2.11.3, werkzeug 2.1.2 and sqlparse 0.4.3 — a 2022 dependency
set nothing installs. uv resolves the base dependencies and the dev extra together, and
mypy==2.1.0 needs pathspec>=1.0.0 while dbt-core on the 1.11 line caps pathspec below 0.13.
The pin could not move, so dbt-core gave: the resolver walked it back to 1.2.6, before that
constraint existed, and locked its dependency tree.

dbt-core 1.12 raised its own cap to pathspec<1.1 (dbt-labs/dbt-core#12385), so a current
mypy resolves against a current dbt-core. The extra now asks for mypy>=2,<3:

before after
mypy 2.1.0 (pinned, unresolvable) 2.3.0
pathspec 1.1.1 requested, 0.9.x locked 1.0.4
dbt-core 1.2.6 1.12.0
dbt-adapters 1.24.5
dbt-clickhouse 1.2.2 1.9.3
jinja2 2.11.3 3.1.6
sqlparse 0.4.3 0.5.5
werkzeug 2.1.2 not installed

dbt-clickhouse settles on 1.9.3 rather than 1.10.1: the 1.10 line caps dbt-adapters below
1.23.0, under dbt-core 1.12.0's 1.24.5 floor, and 1.9.3 carries no adapter bound at all.
Requesting dbt-clickhouse>=1.10 together with mypy>=2 gives No solution found, so the
adapter version is not a free choice here.

ruff moves 0.15.16 → 0.15.21 to match the ruff-pre-commit hook, which was already on
v0.15.21. Two different pins meant a local run and the commit hook could reach different
verdicts.

Unchanged: [project.dependencies], the [tool.ruff] and [tool.mypy] config, the Dockerfile
(pip install . does not install the extra), the compose seed-sample service, the developer
recipe in the README, and CI — no workflow references deploy/seed.

Test plan

  • Repo-wide rescan from clean locks, all 11 Python projects — 0 CRITICAL, 0 HIGH
    (trivy fs --scanners vuln --include-dev-deps, aquasec/trivy:0.72.0). By GitHub's
    CVSS-derived severity: 10 high → 0
  • deploy/seed locks to dbt-clickhouse 1.9.3, dbt-core 1.12.0, dbt-adapters 1.24.5,
    pathspec 1.0.4, mypy 2.3.0, ruff 0.15.21, jinja2 3.1.6, sqlparse 0.5.5, no werkzeug
  • The documented recipe, run end to end: pip install -e '.[dev]' succeeds,
    ruff check . → All checks passed (so 0.15.21 introduces no new failures),
    mypy . → Success, no issues found in 14 source files
  • pip install ., the Dockerfile path — succeeds; seed, silver, identity, profiles,
    generators all import
  • Harness pytest --suites-only with the cryptography override — 42 passed, 1 skipped,
    identical to baseline
  • git diff --check clean
  • Post-merge: confirm the ten alerts close. trivy.yml triggers on main plus the nightly
    schedule, so nothing runs against security-hardening — they will not move until this
    reaches main, or the workflow is fired via workflow_dispatch

@Gregory91G
Gregory91G requested a review from a team as a August 1, 2026 12:46
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 76f313c5-e80c-4348-b4ed-acd681a17315

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Gregory91G Gregory91G self-assigned this Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

⚠️ Regenerate the connectors-ddl snapshot

This PR changes src/ingestion/**. If your change affects any
bronze / silver / gold schema, regenerate the committed DDL snapshot
and include it in this PR.

Prerequisites (details: src/ingestion/scripts/bootstrap-db/README.md):

  • docker + a fresh throwaway ClickHouse 25.7.5 (README "Local ClickHouse for testing")
  • .env from .env.bootstrap.example pointing at it; use the host LAN IP,
    reachable from both the host and connector containers
    (host.docker.internal does not resolve on the macOS host itself)
  • python3.12 or python3.11 on PATH (pinned dbt venv)
  • HubSpot + Salesforce credentials in .env — their discover calls the
    live APIs; without them, apply ../connectors-ddl/{hubspot,salesforce}.sql
    (relative to bootstrap-db/) to seed their bronze, then run the dbt step
cd src/ingestion/scripts/bootstrap-db
set -a; source pins.env; source .env; set +a
./bootstrap-db.sh connectors-config.yaml   # fresh ClickHouse 25.7.5
./dump-ddl.sh                              # writes scripts/connectors-ddl/*.sql

Commit the resulting scripts/connectors-ddl/*.sql diff. If nothing
changed, no snapshot update is needed. (Regeneration is manual for now.)

…2120)

airbyte-cdk 6.60.x resolves cryptography 44.0.3, which carries a Bleichenbacher
timing oracle in PKCS#1 v1.5 decryption (CVE-2026-26007, fixed in 46.0.5) and
GHSA-537c-gmf6-5ccf (fixed in 48.0.1). The override resolves 49.0.0.

Regenerating the lock from a clean checkout, as trivy.yml does, and rescanning:
both findings clear. Harness suites with the override applied: 42 passed,
1 skipped -- identical to baseline.

Signed-off-by: Gregory Gogin <grigoriy.gogin@constructor.tech>
@Gregory91G
Gregory91G force-pushed the fix/uv-resolution-remaining-highs branch from 2cfe977 to 50ac046 Compare August 2, 2026 00:08
Comment thread deploy/seed/pyproject.toml Outdated
# pathspec>=0.9,<0.13. Pin mypy above that line and the resolver satisfies
# it by walking dbt-core back to 1.2.6, locking a 2022 dependency set that
# nothing installs -- the image runs `pip install .` without this extra.
"mypy>=1.10",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not understand why newer dbt-core requires so old version of pathspec.
Also pathspec>=0.9 and pathspec>=1.0.0 can work with latelst pathspec, isn't it? I.e. pathspec==1.1.1 satisfies both.

mypy 1.10 is very old.

@Gregory91G Gregory91G Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude proposed a more optimal solution, which should fix it, because in the current state there were some complex dependencies like:
the maximal dbt-core we can resolve, 1.11.12, requires pathspec < 0.13
mypy >= 1.20 requires pathspec >= 1.0.0.
(dbt-core 1.12.0 raised its cap to < 1.1, but it needs dbt-adapters >= 1.24.5 and dbt-clickhouse 1.10.1 needs dbt-adapters < 1.23.0.)

So proposed solution was to remove mypy и ruff from dependencies (seems they are not needed), which led to removing one branch of constraints ("mypy → pathspec" from → dbt-core → dbt-adapters → dbt-clickhouse), so mypy goes to 2.3.0 and pathspec depends only from dbt-core

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, mypy and ruff required for pre commit checks. So they must be installed somehow. It may not be related to things baked in connector images.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dbt-labs/dbt-core#12385

It seems you need to bump dbt-core.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Base automatically changed from fix/nltk-langchain-critical-cves to security-hardening August 3, 2026 07:59
@Gregory91G
Gregory91G force-pushed the fix/uv-resolution-remaining-highs branch from 50ac046 to d829c73 Compare August 3, 2026 10:07
@cyberantonz
cyberantonz self-requested a review August 3, 2026 11:09
…rrent (#2120)

The scan reports eight highs on deploy/seed for jinja2 2.11.3, werkzeug 2.1.2
and sqlparse 0.4.3 -- a 2022 dependency set nothing installs. uv resolves the
base dependencies and the dev extra together, and mypy==2.1.0 needs
pathspec>=1.0.0 while dbt-core on the 1.11 line caps pathspec below 0.13. The
pin could not move, so dbt-core gave: the resolver walked it back to 1.2.6,
before that constraint existed.

dbt-core 1.12 raised its own cap to pathspec<1.1 (dbt-labs/dbt-core#12385), so a
current mypy resolves against a current dbt-core. The extra now asks for
mypy>=2,<3 and the graph comes out at dbt-core 1.12.0, dbt-adapters 1.24.5,
pathspec 1.0.4, mypy 2.3.0. dbt-clickhouse settles on 1.9.3 rather than 1.10.1:
the 1.10 line caps dbt-adapters below 1.23.0, under dbt-core 1.12.0's 1.24.5
floor, and 1.9.3 carries no adapter bound at all.

ruff moves 0.15.16 -> 0.15.21 to match the ruff-pre-commit hook, which was
already on v0.15.21. Two different pins meant a local run and the commit hook
could reach different verdicts.

Locking from clean: dbt-clickhouse 1.9.3, dbt-core 1.12.0, jinja2 3.1.6,
sqlparse 0.5.5, no werkzeug -- 0 CRITICAL and 0 HIGH repo-wide. The documented
recipe still works end to end: `pip install -e '.[dev]'` then `ruff check .`
(All checks passed) and `mypy .` (14 files, no issues).

Signed-off-by: Gregory Gogin <grigoriy.gogin@constructor.tech>
@Gregory91G
Gregory91G force-pushed the fix/uv-resolution-remaining-highs branch from d829c73 to 48f4d64 Compare August 3, 2026 13:33
@Gregory91G
Gregory91G merged commit 6db8c52 into security-hardening Aug 3, 2026
2 checks passed
@Gregory91G
Gregory91G deleted the fix/uv-resolution-remaining-highs branch August 3, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants