Helps you write your requirements.txt
with hashes so you can
install with pip install --require-hashes -r ...
If you want to add a package or edit the version of one you're currently using you have to do the following steps:
- Go to pypi for that package
- Download the
.tgz
file - Possibly download the
.whl
file - Run
pip hash downloadedpackage-1.2.3.tgz
- Run
pip hash downloadedpackage-1.2.3.whl
- Edit
requirements.txt
This script does all those things. Hackishly wonderfully so.
The whole point of hashing is that you vet the packages that you use on your laptop and that they haven't been tampered with. Then you can confidently install them on a server.
This tool downloads from PyPI (over HTTPS) and runs pip hash
on the downloaded files.
You should check that the packages that are downloaded
are sane and not tampered with. The way you do that is to run
hashin
as normal but with the --verbose
flag. When you do that
it will print where it downloaded the relevant files and those
files are not deleted. For example:
$ hashin --verbose bgg /tmp/reqs.txt https://pypi.python.org/pypi/bgg/json * Latest version for 0.22.1 * Found URL https://pypi.python.org/packages/2.7/b/bgg/bgg-0.22.1-py2-none-any.whl * Re-using /var/folders/1x/2hf5hbs902q54g3bgby5bzt40000gn/T/bgg-0.22.1-py2-none-any.whl * Hash e5172c3fda0e8a42d1797fd1ff75245c3953d7c8574089a41a219204dbaad83d * Found URL https://pypi.python.org/packages/source/b/bgg/bgg-0.22.1.tar.gz * Re-using /var/folders/1x/2hf5hbs902q54g3bgby5bzt40000gn/T/bgg-0.22.1.tar.gz * Hash aaa53aea1cecb8a6e1288d6bfe52a51408a264a97d5c865c38b34ae16c9bff88 * Editing /tmp/reqs.txt
You might not have time to go through the lines one by one but you should be aware that the vetting process is your responsibility.
This is something you only do or ever need in a development environment. Ie. your laptop:
pip install hashin
Suppose you want to install futures
. You can either do this:
hashin futures
Which will download the latest version tarball (and wheel) and
calculate their pip hash and edit your requirements.txt
file.
Or you can be specific about exactly which version you want:
hashin "futures==2.1.3"
You can also specify more than one package at a time:
hashin "futures==2.1.3" requests
Suppose you don't have a requirements.txt
right there in the same
directory you can specify --requirements-file
:
hashin futures --requirements-file=stuff/requirements/prod.txt
By default sha256
hashes are used, but this can be overridden using the
--algorithm
argument:
hashin futures --algorithm=sha512
If there's no output, it worked. Check how it edited your requirements file.
Some requirements have many releases built for different versions of Python and
different architectures. These hashes aren't useful in some cases, if those
wheels don't work with your project. hashin
can filter on the Python
version to skip these extraneous hashes.
For example, the cffi
package offers wheels built for many versions of
CPython from 2.6 to 3.5. To select only one of them, you can use the
--python-version
option:
hashin "cffi==1.5.2" --python-version 3.5
If you need to support multiple versions, you can pass this option multiple times:
hashin "cffi==1.5.2" --python-version 2.7 --python-version 3.5
hashin
will expand these Python versions to a full list of identifers that
could be found on PyPI. For example, 3.5
will expand to match any of
3.5
, py3
, py3.5
, py2.py3
, or cp3.5
. You can also specify
these exact identifiers directly, if you need something specific.
The source
release is always automatically included. pip
will use
this as a fallback in the case a suitable wheel cannot be found.
There are some use cases, when you maybe don't want to edit your requirements.txt
right away. You can use the --dry-run
argument to show the diff, so you
can preview the changes to your requirements.txt
file.
Example:
hashin --dry-run requests==2.19.1
Would result in a printout on the command line:
--- Old +++ New @@ -0,0 +1,3 @@ +requests==2.19.1 \ + --hash=sha256:63b52e3c866428a224f97cab011de738c36aec0185aa91cfacd418b5d58911d1 \ + --hash=sha256:ec22d826a36ed72a7358ff3fe56cbd4ba69dd7a6718ffd450ff0e9df7a47ce6a
Requirements can use PEP-0496 style specifiers (e.g. like
cffi==1.5.2; python_version >= '3.4'
) and these will be passed
through when re-writing the requirements.txt
file. hashin
doesn't
parse the specifiers themselves and will take anything after the
semicolon. If you are using python_version
you will still need to
pass appropriate options if you don't want every available hash.
An example of this might be:
hashin "pywin32-ctypes ; sys_platform == 'win32'"
which will result it something like this in the requirements.txt
file:
pywin32-ctypes==0.1.2; sys_platform == 'win32' \ --hash=sha256:4820b830f42e6889d34142bcd07b3896018c3620d8c31f5e13b72caf1f4d1d0f
And if you want to limit it to certain Python versions, here's an example:
hashin "cffi==1.5.2; python_version >= '3.4'" -p 3.4 -p 3.5
Everything you can do with hashin
on the command line you can do
in running Python too. For example:
import hashin from pprint import pprint pprint(hashin.get_package_hashes('Django'))
This will print out:
{'hashes': [{'hash': 'fbc7ffaa45a4a67cb45f77dbd94e8eceecebe1d0959fe9c665dfbf28b41899e6', 'url': 'https://pypi.python.org/packages/41/c1/68dd27946b03a3d756b0ff665baad25aee1f59918891d86ab76764209208/Django-1.11b1-py2.py3-none-any.whl'}], 'package': 'Django', 'version': '1.11b1'}
Or with specific version, algorithm and certain Python versions:
import hashin from pprint import pprint pprint(hashin.get_package_hashes( 'Django', version='1.10', algorithm='sha512', python_versions=('3.5',) ))
After you have cloned the project, created a virtual environment and run:
pip install -e ".[dev]"
Now, to run it you can use the installed executable hashin
and do things
like:
touch /tmp/reqs.txt hashin -r /tmp/reqs.txt Django
Simply run:
python setup.py test
When you use pip install ".[dev]"
it will install tox
which you can use
to run the full test suites (plus linting) in different Python environments:
tox
To run the tests with test coverage, with pytest
run something like
this:
$ pip install pytest-cover $ pytest --cov=hashin --cov-report=html $ open htmlcov/index.html
To avoid having to install hashin
just to test it or debug a feature
you can simply just run it like this:
touch /tmp/whatever.txt python hashin.py --verbose Django /tmp/whatever.txt
All Python code should be run through Black.
This is checked in CI and you can test it locally with tox
.
Also, this project uses pre-commit
which helps with checking code style as a git pre-commit hook. pre-commit
is used in tox
. To run all code style checks, use tox -e lint
but
make sure your version of tox
is built on a Python 3.
This program is a "fork" of https://pypi.python.org/pypi/peepin
peepin
was a companion to the program peep
https://pypi.python.org/pypi/peep/ but the functionality of peep
has been put directly into pip
as of version 8.
If this script proves itself to work and be useful, I hope we can
put it directly into pip
.
- 1.0.1
- Update change log about the 1.0.0 release.
- 1.0.0
- Update
setup.py
,tox.ini
and GitHub Actions to use Python>=3.8
and up to 3.12.
- Update
- 0.17.0
- Add python 3.9 and 3.10 to the test matrix.
- Preserve lexigraphical order of hashes for the output of the
get_releases_hashes
function. See #126
- 0.16.0
- 0.15.0
- Use of underscore or hyphens in package names is corrected See #116 Thanks @caphrim007
- 0.14.6
- 0.14.5
- When writing down hashes, they are now done in a lexigraphically ordered way. This makes the writes to the requirements file more predictable. See #105
- 0.14.4
- Bugfix for new
--index-url
option feature in version 0.14.3. See #108
- Bugfix for new
- 0.14.3
- New parameter
--index-url
which allows to override the default which ishttps://pypi.org
. Thanks @nmacinnis See #107
- New parameter
- 0.14.2
- When using
--update-all
and parsing requirements file it could be fooled by comments that look like package specs (e.g# check out foo==1.0
) See #103
- When using
- 0.14.1
- All HTTP GET work to fetch information about packages from PyPI is done in concurrent threads. Requires backport for Python 2.7. See #101
- 0.14.0
--interactive
(when you use--update-all
) will iterate over all outdated versions in your requirements file and ask, for each one, if you want to updated it. See #90- Order of hashes should not affect if a package in the requirements file should be replaced or not. See #93
- (Internal) All tests have been rewritten as plain pytest functions.
- In Python 3, if the package can't be found you get a more explicit exception pointing out which package (URL) that failed. See #87
- New flag
--update-all
(alias-u
) will parse the requirements file, ignore the version, and update all packages that have new versions. See #88 - Support for "extras syntax". E.g.
hashin "requests[security]"
. Doesn't actually get hashes forsecurity
(in this case, that's not even a package) but allows that syntax into yourrequirements.txt
file. See #70 - All code is now formatted with Black.
- 0.13.4
- 0.13.3
- Makes it possible to install
nltk
on Windows. Thanks @chrispbailey!
- Makes it possible to install
- 0.13.2
- Match Python versions as
py{major}{minor}
additionally. Solves problem with installing packages with files likePaste-2.0.3-py34-none-any.whl
. Thanks @danfoster!
- Match Python versions as
- 0.13.1
- Ability to pass
--include-prereleases
if you're trying to add a package that only has pre-releases.
- Ability to pass
- 0.13.0
- Two new dependencies for
hashin
:pip-api
andpackaging
. This means we no longer need to importpip
and rely on private APIs. Thanks @di! This also means you can no longer installhashin
on Python 2.6 and Python<=3.3
.
- Two new dependencies for
- 0.12.0
- Switch from
pypi.python.org/pypi/<package>/json
topypi.org/pypi/<package>/json
which also means the sha256 hash is part of the JSON payload immediately instead of having to download and runpip
to get the hash. - Testing no runs Python 2.6 and Python 3.3.
- All hashes, per package, are sorted (by the hash) to make it more predictable.
- Switch from
- 0.11.5
- You can now pass PEP-0496 Environment Markers together with the package
name, and they get passed into the
requirements.txt
file. Thanks @meejah
- You can now pass PEP-0496 Environment Markers together with the package
name, and they get passed into the
- 0.11.4
- PackageErrors happening in CLI suppressed just the error message out on stderr. No full traceback any more.
- 0.11.3
- Better error if you typo the package name since it'll 404 on PyPI.
- 0.11.2
- Run continuous integration tests with Python 3.6 too.
- 0.11.1
- Ability to run
hashin --version
to see what version of hashin is installed. See #41
- Ability to run
- 0.11.0
- Cope with leading zeros in version numbers when figuring out what the latest version is. See #39
- 0.10.0
- Latest version is now figured out by looking at all version numbers in the list of releases from the JSON payload. The pre releases are skipped.
- 0.9.0
- Fixed a bug where it would fail to install a package whose name is
partially part of an existing (installed) package.
E.g. installing
redis==x.y.z
whendjango-redis==a.b.c
was already in the requirements file.
- Fixed a bug where it would fail to install a package whose name is
partially part of an existing (installed) package.
E.g. installing
- 0.8.0
- Ability to make
hashin
work as a library. Thanks @jayfk ! - pep8 cleanups.
- Ability to make
- 0.7.2
- Fixes bug related to installing platform specific archives See #33 Thanks @mythmon
- 0.7.1
- Package matching is now case insensitive. E.g.
hashin dJaNgO
- Package matching is now case insensitive. E.g.
- 0.7.0
- The requirements file and algorithm arguments are now keyword arguments. Now, the second, third, nth positional argument are additional arguments. Thanks @https://github.com/ahal
- 0.6.1
- Support windows binaries packaged as a
.msi
file.
- Support windows binaries packaged as a
- 0.6.0
- Fix compatibility issue with pip 8.1.2 and 8.1.1-2ubuntu0.1 and drop support for Python 2.6
- 0.5.0
- Important bug fix. As an example, if you had
pytest-selenium==...
already in yourrequirements.txt
file and addselenium==x.y.z
it would touch the line withpytest-selenium
too.
- Important bug fix. As an example, if you had
- 0.4.1
- Support for PyPI links that have a hash in the file URL.
- 0.4.1
- Fix PackageError if no Python version is defined.
- 0.4
- Add filtering of package releases by Python version.
- 0.3
- Issue a warning for users of Python before version 2.7.9.
- 0.2
- Last character a single newline. Not two.
- 0.1
- First, hopefully, working version.