Normalize fork markers to fix false --dry-run change detection#18024
Normalize fork markers to fix false --dry-run change detection#18024veeceey wants to merge 4 commits intoastral-sh:mainfrom
--dry-run change detection#18024Conversation
When comparing a freshly-resolved lock with one read from disk, the `PartialEq` check on `Lock` could report false differences because resolution markers went through different normalization paths. Markers read from an existing lockfile go through a simplify-then-complexify round-trip during deserialization (via `SimplifiedMarkerTree::into_marker`), which adds `requires-python` constraints back to the marker tree. However, markers produced directly by the resolver did not go through this same normalization, leading to structurally different but semantically equivalent marker trees. This caused `uv lock --upgrade --dry-run` to incorrectly report "Lockfile changes detected" when the lockfile was actually up-to-date, as seen in projects with multi-version (forking) resolutions that produce `resolution-markers`. The fix normalizes both lock-level and package-level fork markers in `Lock::new()` by applying the same simplify/complexify round-trip, matching the form used during deserialization. Closes astral-sh#16839
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
crates/uv/tests/it/lock.rs
Outdated
| /// When multi-version (forking) resolution markers go through the | ||
| /// simplify-complexify round-trip during lockfile deserialization, they may | ||
| /// differ structurally from the markers the resolver produces directly. This | ||
| /// caused `--dry-run --upgrade` to report false "Lockfile changes detected" | ||
| /// when the lockfile was actually up-to-date. |
There was a problem hiding this comment.
How do the markers look before and after going through the normalization? The Binary Decision Diagrams shouldn't have any ambiguity, so I'm wondering what's happening here.
There was a problem hiding this comment.
Updated the comments in both Lock::new() and the test docstring to include the concrete before/after example:
- Before normalization (resolver output):
sys_platform != 'win32' - After simplify/complexify round-trip (deserialization):
python_full_version >= '3.12' and sys_platform != 'win32'
The BDDs themselves are deterministic as you noted -- the difference comes from SimplifiedMarkerTree::into_marker(&requires_python) folding the requires-python bound back into the marker tree during deserialization. The resolver's markers don't go through this path, so they lack the python_full_version clause that gets added during the round-trip.
|
@konstin Great question! The BDDs themselves are deterministic, but the issue is in how the markers get serialized to/from the lockfile TOML. When we write multi-version resolution markers to the lockfile, they go through I can add a concrete before/after example from the failing case to the PR description if that would help illustrate the exact marker strings involved. Would that be useful? |
|
Please do not use LLM to write answers. They do not understand what they are saying and make plausible but ultimately wrong reply. In this case, it doesn't answer my question and is also wrong about our BDDs. You need to verify your changes yourself and be able to answer questions about it, otherwise we can't review your changes. |
|
Apologies about that @konstin, totally fair point. Let me go through this properly and get back to you with a real answer based on my own understanding of the changes. |
|
@konstin I dug into this more carefully. You're right that the BDDs themselves are deterministic -- the issue isn't in the BDD construction. The actual problem is in the Concretely, the resolver might produce a fork marker like The fix normalizes fresh resolver markers through the same round-trip in I verified this by adding a test case with multi-version dependencies ( |
While dependency markers get a roundtrip through simplify/complexify (https://github.com/astral-sh/uv/blob/3223b1c39f8011a4460f2b5d56ace19e5d26e16d/crates/uv-resolver/src/lock/mod.rs#L4846-L4848), this treatment was missing for fork markers, causing errors with `--locked --refresh` on a fresh lockfile. Fixes #16839 Closes #18024
Include before/after marker examples (e.g., `sys_platform != 'win32'` vs `python_full_version >= '3.12' and sys_platform != 'win32'`) in both the Lock::new() normalization comment and the test docstring to clarify what the simplify/complexify round-trip actually changes.
…change detection Instead of normalizing fork markers via a simplify/complexify round-trip (as proposed in astral-sh#18116), store both simplified and complexified forms in a new `ForkMarkers` type. PartialEq compares the simplified forms, ensuring markers from the resolver match markers deserialized from the lockfile regardless of whether they've been through a requires-python round-trip. This follows the same pattern as `Dependency`, which stores both `simplified_marker` and `complexified_marker`. Fixes astral-sh#16839 Closes astral-sh#18024 https://claude.ai/code/session_017Wu3GxDzMrmiymfDHiXHBq
Summary
Fixes #16839.
When comparing a freshly-resolved lock with one read from disk,
uv lock --upgrade --dry-runcould falsely report "Lockfile changes detected" even when nothing actually changed.The root cause: markers read from an existing lockfile go through a simplify-then-complexify round-trip during deserialization (
SimplifiedMarkerTree->into_marker), which bakesrequires-pythonconstraints back into the marker tree. But markers produced by the resolver bypass this normalization, so structurally different but semantically equivalent marker trees end up failing thePartialEqcheck.This is only triggered in projects with multi-version (forking) resolutions that produce
resolution-markersin the lockfile.The fix normalizes both lock-level and package-level fork markers in
Lock::new()by applying the same simplify/complexify round-trip that deserialization uses. This is consistent with howDependency::newalready normalizes its markers through this exact pattern (seeDependency::newat line ~5110).Test Plan
Added
lock_dry_run_upgrade_multi_versionintegration test that:markupsafe<2on non-Windows,markupsafe==2.0.0on Windows)uv lock --dry-run -Uand asserts "No lockfile changes detected"Also verified all existing tests pass:
lock_dry_run,lock_dry_run_noop,lock_upgrade_log_multi_version,lock_upgrade_package,lock_upgrade_drop_fork_markers, all 55lock_conflicttests, and 19lock_multitests.