builders/wheel: Ensure dist-info is written deterministically #545

rpurdie · 2022-12-23T11:09:48Z

glob() returns values in "on disk" order. To make the RECORD file deterministic and consistent between builds we need to sort the data before adding to the records list.

Signed-off-by: Richard Purdie [email protected]

Resolves: python-poetry#

Added tests for changed code.
Updated documentation for changed code.

radoering · 2022-12-31T14:21:17Z

Interesting. I suppose, we should sort here as well since _add_file() is called in the loop?

poetry-core/src/poetry/core/masonry/builders/wheel.py

Line 193 in 8718681

for pkg in lib.glob("**/*"):

rpurdie · 2022-12-31T14:43:22Z

Interesting. I suppose, we should sort here as well since _add_file() is called in the loop?

poetry-core/src/poetry/core/masonry/builders/wheel.py

Line 193 in 8718681

for pkg in lib.glob("**/*"):

Yes, that looks like it would have the same issue.

glob() returns values in "on disk" order. To make the RECORD file deterministic and consistent between builds we need to sort the data before adding to the records list. Signed-off-by: Richard Purdie <[email protected]>

sonarcloud · 2023-01-07T12:34:02Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

* Drop deterministic.patch; merged python-poetry/poetry-core#545 * Drop vendored pyparsing LICENSE; no longer vendored https://github.com/python-poetry/poetry-core/blob/main/CHANGELOG.md#150---2023-01-27 (From OE-Core rev: 579500646a8d9c452fb6807011ab82f8f09b32a4) Signed-off-by: Tim Orling <[email protected]> Signed-off-by: Alexandre Belloni <[email protected]>

* Drop deterministic.patch; merged python-poetry/poetry-core#545 * Drop vendored pyparsing LICENSE; no longer vendored https://github.com/python-poetry/poetry-core/blob/main/CHANGELOG.md#150---2023-01-27 (From OE-Core rev: b0c02ebde06e4cb79ce895bfdf4bed60c1c7daee) Signed-off-by: Tim Orling <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

* Drop deterministic.patch; merged python-poetry/poetry-core#545 * Drop vendored pyparsing LICENSE; no longer vendored https://github.com/python-poetry/poetry-core/blob/main/CHANGELOG.md#150---2023-01-27 Signed-off-by: Tim Orling <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

* Drop deterministic.patch; merged python-poetry/poetry-core#545 * Drop vendored pyparsing LICENSE; no longer vendored https://github.com/python-poetry/poetry-core/blob/main/CHANGELOG.md#150---2023-01-27 (From OE-Core rev: 32921155acf776c91ee2fb002660b419512f0932) Signed-off-by: Tim Orling <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

* Drop deterministic.patch; merged python-poetry/poetry-core#545 * Drop vendored pyparsing LICENSE; no longer vendored https://github.com/python-poetry/poetry-core/blob/main/CHANGELOG.md#150---2023-01-27 Signed-off-by: Tim Orling <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [poetry](https://python-poetry.org/) ([source](https://github.com/python-poetry/poetry), [changelog](https://python-poetry.org/history/)) | minor | `1.3.2` -> `1.4.0` | --- ### Release Notes <details> <summary>python-poetry/poetry</summary> ### [`v1.4.0`](https://github.com/python-poetry/poetry/blob/HEAD/CHANGELOG.md#140---2023-02-27) [Compare Source](python-poetry/poetry@1.3.2...1.4.0) ##### Added - **Add a modern installer (`installer.modern-installation`) for faster installation of packages and independence from pip** ([#6205](python-poetry/poetry#6205)). - Add support for `Private ::` trove classifiers ([#7271](python-poetry/poetry#7271)). - Add the version of poetry in the `@generated` comment at the beginning of the lock file ([#7339](python-poetry/poetry#7339)). - Add support for `virtualenvs.prefer-active-python` when running `poetry new` and `poetry init` ([#7100](python-poetry/poetry#7100)). ##### Changed - **Deprecate the old installer, i.e. setting `experimental.new-installer` to `false`** ([#7358](python-poetry/poetry#7358)). - Remove unused `platform` field from cached package info and bump the cache version ([#7304](python-poetry/poetry#7304)). - Extra dependencies of the root project are now sorted in the lock file ([#7375](python-poetry/poetry#7375)). - Remove upper boundary for `importlib-metadata` dependency ([#7434](python-poetry/poetry#7434)). - Validate path dependencies during use instead of during construction ([#6844](python-poetry/poetry#6844)). - Remove the deprecated `repository` modules ([#7468](python-poetry/poetry#7468)). ##### Fixed - Fix an issue where an unconditional dependency of an extra was not installed in specific environments ([#7175](python-poetry/poetry#7175)). - Fix an issue where a pre-release of a dependency was chosen even if a stable release fulfilled the constraint ([#7225](python-poetry/poetry#7225), [#7236](python-poetry/poetry#7236)). - Fix an issue where HTTP redirects were not handled correctly during publishing ([#7160](python-poetry/poetry#7160)). - Fix an issue where `poetry check` did not handle the `-C, --directory` option correctly ([#7241](python-poetry/poetry#7241)). - Fix an issue where the subdirectory information of a git dependency was not written to the lock file ([#7367](python-poetry/poetry#7367)). - Fix an issue where the wrong Python version was selected when creating an virtual environment ([#7221](python-poetry/poetry#7221)). - Fix an issue where packages that should be kept were uninstalled when calling `poetry install --sync` ([#7389](python-poetry/poetry#7389)). - Fix an issue where an incorrect value was set for `sys.argv[0]` when running installed scripts ([#6737](python-poetry/poetry#6737)). - Fix an issue where hashes in `direct_url.json` files were not written according to the specification ([#7475](python-poetry/poetry#7475)). - Fix an issue where poetry commands failed due to special characters in the path of the project or virtual environment ([#7471](python-poetry/poetry#7471)). - Fix an issue where poetry crashed with a `JSONDecodeError` when running a Python script that produced certain warnings ([#6665](python-poetry/poetry#6665)). ##### Docs - Add advice on how to maintain a poetry plugin ([#6977](python-poetry/poetry#6977)). - Update tox examples to comply with the latest tox release ([#7341](python-poetry/poetry#7341)). - Mention that the `poetry export` can export `constraints.txt` files ([#7383](python-poetry/poetry#7383)). - Add clarifications for moving configuration files ([#6864](python-poetry/poetry#6864)). - Mention the different types of exact version specifications ([#7503](python-poetry/poetry#7503)). ##### poetry-core ([`1.5.1`](https://github.com/python-poetry/poetry-core/releases/tag/1.5.1)) - Improve marker handling ([#528](python-poetry/poetry-core#528), [#534](python-poetry/poetry-core#534), [#530](python-poetry/poetry-core#530), [#546](python-poetry/poetry-core#546), [#547](python-poetry/poetry-core#547)). - Validate whether dependencies referenced in `extras` are defined in the main dependency group ([#542](python-poetry/poetry-core#542)). - Poetry no longer generates a `setup.py` file in sdists by default ([#318](python-poetry/poetry-core#318)). - Fix an issue where trailing newlines were allowed in `tool.poetry.description` ([#505](python-poetry/poetry-core#505)). - Fix an issue where the name of the data folder in wheels was not normalized ([#532](python-poetry/poetry-core#532)). - Fix an issue where the order of entries in the RECORD file was not deterministic ([#545](python-poetry/poetry-core#545)). - Fix an issue where zero padding was not correctly handled in version comparisons ([#540](python-poetry/poetry-core#540)). - Fix an issue where sdist builds did not support multiple READMEs ([#486](python-poetry/poetry-core#486)). ##### poetry-plugin-export ([`^1.3.0`](https://github.com/python-poetry/poetry-plugin-export/releases/tag/1.3.0)) - Fix an issue where the export failed if there was a circular dependency on the root package ([#118](python-poetry/poetry-plugin-export#118)). </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.walbeck.it/walbeck-it/docker-python-poetry/pulls/655 Co-authored-by: renovate-bot <[email protected]> Co-committed-by: renovate-bot <[email protected]>

…ic due to python-poetry#545

…ic due to #545

* Drop deterministic.patch; merged python-poetry/poetry-core#545 * Drop vendored pyparsing LICENSE; no longer vendored https://github.com/python-poetry/poetry-core/blob/main/CHANGELOG.md#150---2023-01-27 (From OE-Core rev: 32921155acf776c91ee2fb002660b419512f0932) Signed-off-by: Tim Orling <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

radoering changed the title ~~builders/wheel: Ensure dist-info is written determinisically~~ builders/wheel: Ensure dist-info is written deterministically Jan 7, 2023

rpurdie and others added 2 commits January 7, 2023 13:33

builders/wheel: Ensure dist-info is written determinisically

e368e7a

glob() returns values in "on disk" order. To make the RECORD file deterministic and consistent between builds we need to sort the data before adding to the records list. Signed-off-by: Richard Purdie <[email protected]>

builders/wheel: Ensure dist-info is written deterministically

69d85c2

radoering force-pushed the main branch from bb9258c to 69d85c2 Compare January 7, 2023 12:33

radoering approved these changes Jan 7, 2023

View reviewed changes

radoering merged commit 49d1888 into python-poetry:main Jan 7, 2023

radoering mentioned this pull request Jan 24, 2023

Release 1.5.0 #551

Merged

radoering mentioned this pull request Feb 18, 2023

release: bump version to 1.4.0 python-poetry/poetry#7531

Merged

radoering added a commit to dimbleby/poetry-core that referenced this pull request Mar 11, 2023

test(build): order in RECORD files and archives should be determinist…

b1d3604

…ic due to python-poetry#545

radoering added a commit to dimbleby/poetry-core that referenced this pull request Mar 11, 2023

test(build): order in RECORD files and archives should be determinist…

accd3fc

…ic due to python-poetry#545

radoering added a commit that referenced this pull request Mar 11, 2023

test(build): order in RECORD files and archives should be determinist…

35e656c

…ic due to #545

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

builders/wheel: Ensure dist-info is written deterministically #545

builders/wheel: Ensure dist-info is written deterministically #545

rpurdie commented Dec 23, 2022

radoering commented Dec 31, 2022

rpurdie commented Dec 31, 2022

sonarcloud bot commented Jan 7, 2023

builders/wheel: Ensure dist-info is written deterministically #545

builders/wheel: Ensure dist-info is written deterministically #545

Conversation

rpurdie commented Dec 23, 2022

radoering commented Dec 31, 2022

rpurdie commented Dec 31, 2022

sonarcloud bot commented Jan 7, 2023