Issue with coverage[toml] when installing with require-hashes.

[matejsp]

Summary

There is an issue when installing pytest-cov with require hashes mode.
We install from pip with all hahes provided on target machine. requirements.txt is generated via pipenv.

Expected vs actual result

It should install coverage and pytest-cov without an error. Now this happens everytime there is new coverage release and we are behind latest. It happens always after every new release of coverage, before we can update all the projects, breaking the deployment.

Instead of working we get:

ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    coverage[toml]>=5.2.1 from 

Reproducer

python3.11 -m venv .venv
pip install pipenv --index-url https://pypi.python.org/simple 

Create standard pipenv file:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[requires]
python_version = "3.11"

[packages]
coverage = "==7.3.1"
pytest-cov = "==4.1.0"

PIPENV_IGNORE_VIRTUALENVS=1 pipenv lock
PIPENV_IGNORE_VIRTUALENVS=1 pipenv requirements --hash > requirements.txt

Now that I have requiremernts:

-i https://pypi.org/simple
coverage==7.3.1; python_version >= '3.8' --hash=sha256:025ded371f1ca280c035d91b43252adbb04d2aea4c7105252d3cbc227f03b375 --hash=sha256:04312b036580ec505f2b77cbbdfb15137d5efdfade09156961f5277149f5e344 --hash=sha256:0575c37e207bb9b98b6cf72fdaaa18ac909fb3d153083400c2d48e2e6d28bd8e --hash=sha256:07d156269718670d00a3b06db2288b48527fc5f36859425ff7cec07c6b367745 --hash=sha256:1f111a7d85658ea52ffad7084088277135ec5f368457275fc57f11cebb15607f --hash=sha256:220eb51f5fb38dfdb7e5d54284ca4d0cd70ddac047d750111a68ab1798945194 --hash=sha256:229c0dd2ccf956bf5aeede7e3131ca48b65beacde2029f0361b54bf93d36f45a --hash=sha256:245c5a99254e83875c7fed8b8b2536f040997a9b76ac4c1da5bff398c06e860f --hash=sha256:2829c65c8faaf55b868ed7af3c7477b76b1c6ebeee99a28f59a2cb5907a45760 --hash=sha256:4aba512a15a3e1e4fdbfed2f5392ec221434a614cc68100ca99dcad7af29f3f8 --hash=sha256:4c96dd7798d83b960afc6c1feb9e5af537fc4908852ef025600374ff1a017392 --hash=sha256:50dd1e2dd13dbbd856ffef69196781edff26c800a74f070d3b3e3389cab2600d --hash=sha256:5289490dd1c3bb86de4730a92261ae66ea8d44b79ed3cc26464f4c2cde581fbc --hash=sha256:53669b79f3d599da95a0afbef039ac0fadbb236532feb042c534fbb81b1a4e40 --hash=sha256:553d7094cb27db58ea91332e8b5681bac107e7242c23f7629ab1316ee73c4981 --hash=sha256:586649ada7cf139445da386ab6f8ef00e6172f11a939fc3b2b7e7c9082052fa0 --hash=sha256:5ae4c6da8b3d123500f9525b50bf0168023313963e0e2e814badf9000dd6ef92 --hash=sha256:5b4ee7080878077af0afa7238df1b967f00dc10763f6e1b66f5cced4abebb0a3 --hash=sha256:5d991e13ad2ed3aced177f524e4d670f304c8233edad3210e02c465351f785a0 --hash=sha256:614f1f98b84eb256e4f35e726bfe5ca82349f8dfa576faabf8a49ca09e630086 --hash=sha256:636a8ac0b044cfeccae76a36f3b18264edcc810a76a49884b96dd744613ec0b7 --hash=sha256:6407424621f40205bbe6325686417e5e552f6b2dba3535dd1f90afc88a61d465 --hash=sha256:6bc6f3f4692d806831c136c5acad5ccedd0262aa44c087c46b7101c77e139140 --hash=sha256:6cb7fe1581deb67b782c153136541e20901aa312ceedaf1467dcb35255787952 --hash=sha256:74bb470399dc1989b535cb41f5ca7ab2af561e40def22d7e188e0a445e7639e3 --hash=sha256:75c8f0df9dfd8ff745bccff75867d63ef336e57cc22b2908ee725cc552689ec8 --hash=sha256:770f143980cc16eb601ccfd571846e89a5fe4c03b4193f2e485268f224ab602f --hash=sha256:7eb0b188f30e41ddd659a529e385470aa6782f3b412f860ce22b2491c89b8593 --hash=sha256:7eb3cd48d54b9bd0e73026dedce44773214064be93611deab0b6a43158c3d5a0 --hash=sha256:87d38444efffd5b056fcc026c1e8d862191881143c3aa80bb11fcf9dca9ae204 --hash=sha256:8a07b692129b8a14ad7a37941a3029c291254feb7a4237f245cfae2de78de037 --hash=sha256:966f10df9b2b2115da87f50f6a248e313c72a668248be1b9060ce935c871f276 --hash=sha256:a6191b3a6ad3e09b6cfd75b45c6aeeffe7e3b0ad46b268345d159b8df8d835f9 --hash=sha256:aab8e9464c00da5cb9c536150b7fbcd8850d376d1151741dd0d16dfe1ba4fd26 --hash=sha256:ac3c5b7e75acac31e490b7851595212ed951889918d398b7afa12736c85e13ce --hash=sha256:ac9ad38204887349853d7c313f53a7b1c210ce138c73859e925bc4e5d8fc18e7 --hash=sha256:b9c0c19f70d30219113b18fe07e372b244fb2a773d4afde29d5a2f7930765136 --hash=sha256:c397c70cd20f6df7d2a52283857af622d5f23300c4ca8e5bd8c7a543825baa5a --hash=sha256:c6601a60318f9c3945be6ea0f2a80571f4299b6801716f8a6e4846892737ebe4 --hash=sha256:c6f55d38818ca9596dc9019eae19a47410d5322408140d9a0076001a3dcb938c --hash=sha256:ca70466ca3a17460e8fc9cea7123c8cbef5ada4be3140a1ef8f7b63f2f37108f --hash=sha256:ca833941ec701fda15414be400c3259479bfde7ae6d806b69e63b3dc423b1832 --hash=sha256:cd0f7429ecfd1ff597389907045ff209c8fdb5b013d38cfa7c60728cb484b6e3 --hash=sha256:cd694e19c031733e446c8024dedd12a00cda87e1c10bd7b8539a87963685e969 --hash=sha256:cdd088c00c39a27cfa5329349cc763a48761fdc785879220d54eb785c8a38520 --hash=sha256:de30c1aa80f30af0f6b2058a91505ea6e36d6535d437520067f525f7df123887 --hash=sha256:defbbb51121189722420a208957e26e49809feafca6afeef325df66c39c4fdb3 --hash=sha256:f09195dda68d94a53123883de75bb97b0e35f5f6f9f3aa5bf6e496da718f0cb6 --hash=sha256:f12d8b11a54f32688b165fd1a788c408f927b0960984b899be7e4c190ae758f1 --hash=sha256:f1a317fdf5c122ad642db8a97964733ab7c3cf6009e1a8ae8821089993f175ff --hash=sha256:f2781fd3cabc28278dc982a352f50c81c09a1a500cc2086dc4249853ea96b981 --hash=sha256:f4f456590eefb6e1b3c9ea6328c1e9fa0f1006e7481179d749b3376fc793478e
iniconfig==2.0.0; python_version >= '3.7' --hash=sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3 --hash=sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374
packaging==23.2; python_version >= '3.7' --hash=sha256:048fb0e9405036518eaaf48a55953c750c11e1a1b68e0dd1a9d62ed0c092cfc5 --hash=sha256:8c491190033a9af7e1d931d0b5dacc2ef47509b34dd0de67ed209b5203fc88c7
pluggy==1.3.0; python_version >= '3.8' --hash=sha256:cf61ae8f126ac6f7c451172cf30e3e43d3ca77615509771b3a984a0730651e12 --hash=sha256:d89c696a773f8bd377d18e5ecda92b7a3793cbe66c87060a6fb58c7b6e1061f7
pytest==7.4.2; python_version >= '3.7' --hash=sha256:1d881c6124e08ff0a1bb75ba3ec0bfd8b5354a01c194ddd5a0a870a48d99b002 --hash=sha256:a766259cfab564a2ad52cb1aae1b881a75c3eb7e34ca3779697c23ed47c47069
pytest-cov==4.1.0; python_version >= '3.7' --hash=sha256:3904b13dfbfec47f003b8e77fd5b589cd11904a21ddf1ab38a64f204d6a10ef6 --hash=sha256:6ba70b9e97e69fcc3fb45bfeab2d0a138fb65c4d0d6a41ef33983ad114be8c3a

pip install -r requirements.txt --require-hashes              

(.venv) ➜  testbug pip install -r requirements.txt --require-hashes
Collecting coverage==7.3.1 (from -r requirements.txt (line 2))
  Using cached coverage-7.3.1-cp311-cp311-macosx_10_9_x86_64.whl (201 kB)
Collecting iniconfig==2.0.0 (from -r requirements.txt (line 3))
  Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting packaging==23.2 (from -r requirements.txt (line 4))
  Using cached packaging-23.2-py3-none-any.whl (53 kB)
Collecting pluggy==1.3.0 (from -r requirements.txt (line 5))
  Using cached pluggy-1.3.0-py3-none-any.whl (18 kB)
Collecting pytest==7.4.2 (from -r requirements.txt (line 6))
  Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Collecting pytest-cov==4.1.0 (from -r requirements.txt (line 7))
  Using cached pytest_cov-4.1.0-py3-none-any.whl (21 kB)
Collecting coverage[toml]>=5.2.1 (from pytest-cov==4.1.0->-r requirements.txt (line 7))
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    coverage[toml]>=5.2.1 from https://files.pythonhosted.org/packages/a9/6b/4d3b9ce8b79378f960e3b74bea4569daf6bd3e1d562a15c9ce4d40be182c/coverage-7.3.2-cp311-cp311-macosx_10_9_x86_64.whl (from pytest-cov==4.1.0->-r requirements.txt (line 7))
(.venv) ➜  testbug 

Versions

(.venv) ➜ testbug pip --version
pip 23.2.1 from /Users/myuser/temp/testbug/.venv/lib/python3.11/site-packages/pip (python 3.11)
(.venv) ➜ testbug python --version
Python 3.11.5

[matejsp]

this issue in PIP is still open for 2.5 years with no solution in sight:
pypa/pip#9644

[matejsp]

I saw that this issue was fixed, reverted ... then upstreamchanges toml to tomli etc etc ...

Why does pytest-cov even need toml from coverage? Why even use that extra?

[ionelmc]

Yeah I guess we could stop depending on coverage with an extra :/

[alinocco]

I upgraded pip with pip install -U pip from version 23.0.1 to 24.1.2. It helped.

[webknjaz]

@ionelmc I don't think so. Pip's behavior is correct — if hash checking is requested by the end-user, it requires that the entire dependency tree has hashes recorded. It's the users' responsibility to provide said hashes. This has nothing to do with extras, just the transitive deps.

[matejsp]

@alinocco Yey ... latest pip 24.x works !!!

@webknjaz Actually pip behavior with extras was broken forever (even ack bug in pip is still open), but it seems they actually patched it in 24.x.

Basically it does not matter if you define extras or not ... dependency with extras will always have the same hashes:

coverage==7.3.1; ... SAME HASHES
coverage[toml]==7.3.1; ... SAME HASHES

User responsibility is to provide extra dependencies (tomli in this case) together with hashes. And expect pip to correctly resolve coverage in both cases (with extras flags) ... For coverage in this case it is the same set of whls and hashes in the end.

Not to mention that coverage[toml] resolves to nothing for python 3.11 (https://github.com/nedbat/coveragepy/blob/master/setup.py#L102)

You can check it, if you my requirements.txt from the issue and check that there was indeed a bug in pip:
It fails with 23.2.1:

rm -rf .venv && python3.11 -m venv .venv && pip install pip==23.2.1 --index-url https://pypi.python.org/simple && pip install -r requirements.txt --require-hashes --index-url https://pypi.python.org/simple

It works with 24.1.2:

rm -rf .venv && python3.11 -m venv .venv && pip install pip==24.1.2 --index-url https://pypi.python.org/simple && pip install -r requirements.txt --require-hashes --index-url https://pypi.python.org/simple

Now regarding toml extras ... if pytest-cov needs tomli itself to work it should include it itself ... if it does not ... it should be removed ... What if tomli is removed form coverage or switched to other implementation. Would pytest-cov be broken then?