Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove same-graph merging in resolver #6077

Merged
merged 1 commit into from
Aug 14, 2024
Merged

Conversation

charliermarsh
Copy link
Member

Summary

This was added in #5405 but is now the cause of an instability in github_wikidata_bot. Specifically, on the initial run, we fork in pydantic==2.8.2, via:

Requires-Dist: typing-extensions>=4.12.2; python_version >= '3.13'
Requires-Dist: typing-extensions>=4.6.1; python_version < '3.13'

In the end, we resolve a single version of typing-extensions (4.12.2)... But we don't recognize the two resolutions as the "same graph", because we propagate the fork markers, and so the "edges" have different markers on them...

In the second run through, when we have the forks in advance, we don't split on Pydantic... We just try to solve from the root with the current forks. This is fundamentally different and I fear it will be the cause of many instabilities. But removing this graph check fixes the proximate issue.

I don't really understand why this was added since there was no test coverage in the PR.

@charliermarsh charliermarsh added bug Something isn't working resolver Related to the package resolver preview Experimental behavior labels Aug 14, 2024
@charliermarsh charliermarsh changed the base branch from main to charlie/ex August 14, 2024 01:56
@charliermarsh charliermarsh force-pushed the charlie/same-graph branch 2 times, most recently from ca1b896 to 591f126 Compare August 14, 2024 02:56
# via
# -r requirements.in
# cryptography
cffi==1.17.0rc1 ; os_name != 'linux' or platform_python_implementation != 'PyPy'
cffi==1.17.0rc1 ; os_name != 'linux'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this was wrong before...? The requirements are:

cffi ; os_name == 'linux'
cffi >= 1.17.0rc1 ; os_name != 'linux'
cryptography

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or is it just that these are redundant...? Since os_name == 'linux' or os_name != 'linux' already covers the full space?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is definitely redundant in terms of the marker expressions. But it's not immediately obvious to me why this change dropped that part of the marker.

Copy link
Member

@BurntSushi BurntSushi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say we should merge this. I believe the original motivation was as an optimization. Because if we can prune forks then we probably should. But removing this optimization results in a bug fix from what I can see, and I think is potentially confounding our instability investigation.

It does seem like this class of optimization ought to be sound though. But I think we should chop it out for now and revisit it later. (i.e., Prioritize correctness.)

# via
# -r requirements.in
# example
torch==2.0.0+cu118 ; os_name == 'Linux'
torch==2.0.0+cu118 ; os_name == 'Linux' and platform_machine == 'x86_64'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a bug fix.

# via
# -r requirements.in
# cryptography
cffi==1.17.0rc1 ; os_name != 'linux' or platform_python_implementation != 'PyPy'
cffi==1.17.0rc1 ; os_name != 'linux'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is definitely redundant in terms of the marker expressions. But it's not immediately obvious to me why this change dropped that part of the marker.

@charliermarsh charliermarsh force-pushed the charlie/ex branch 2 times, most recently from b540a6a to acb3f31 Compare August 14, 2024 13:46
Base automatically changed from charlie/ex to main August 14, 2024 13:55
@charliermarsh
Copy link
Member Author

Ok this fixes two of the three instabilities that emerged after #6065.

@charliermarsh charliermarsh enabled auto-merge (squash) August 14, 2024 13:59
@charliermarsh charliermarsh merged commit 9de3c94 into main Aug 14, 2024
56 checks passed
@charliermarsh charliermarsh deleted the charlie/same-graph branch August 14, 2024 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working preview Experimental behavior resolver Related to the package resolver
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants