-
Notifications
You must be signed in to change notification settings - Fork 322
Build Python wheels for windows on travis #160
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 4 commits
bd583e2
b4249c4
c6965bf
e0b6881
b0cefb6
eddc9c5
e38c203
9003836
1cb337f
6e25817
6d1bf25
6b2d18f
71e889a
db38b33
5746456
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,29 +4,50 @@ matrix: | |
| include: | ||
| # Linux Python 2 | ||
| - sudo: required | ||
| if: branch=master | ||
| language: python | ||
| python: 2.7 | ||
| services: docker | ||
| env: PYTHON=python | ||
|
|
||
| # Linux Python 3 | ||
| - sudo: required | ||
| if: branch=master | ||
| language: python | ||
| python: 3.4 | ||
| services: docker | ||
| env: PYTHON=python | ||
|
|
||
| # macOS Python 2 | ||
| - os: osx | ||
| if: branch=master | ||
| env: PYTHON=python2 | ||
|
|
||
| # macOS Python 3 | ||
| - os: osx | ||
| if: branch=master | ||
| env: PYTHON=python3 | ||
| before_install: | ||
| - brew update | ||
| - brew outdated python || brew upgrade python | ||
|
|
||
| - os: windows | ||
| language: shell | ||
| before_install: | ||
| - choco install python3 --version 3.5.4 --no-progress --force -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python35-x64'" | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just for future reference, is there a reason we need all of these different arguments, here? I understand we need more flags/control for the different versions, but for the one that's just running
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| install: | ||
| - C:\\Python35-x64\\python -m pip install -r requirements-dev.txt | ||
| script: | ||
| - C:\\Python35-x64\\python ./bin/run_tests.py | ||
|
|
||
| install: $PYTHON -m pip install -r requirements-dev.txt | ||
|
|
||
| script: $PYTHON ./bin/run_tests.py | ||
|
|
||
|
|
||
| # - choco install python3-x86_32 --version 3.4.3.20150501 --no-progress -y --allowmultiple --override --installargs "'/qn /norestart ALLUSERS=1 TARGETDIR=C:\Python34'" | ||
| # - choco install python3 --version 3.4.4.20180111 --no-progress --force -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python34-x64'" | ||
| # - choco install python2 --version 2.7.16 --no-progress --x86 -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python27'" | ||
| # - choco install python2 --version 2.7.16 --no-progress -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python27-x64'" | ||
| # - wget -q http://download.microsoft.com/download/7/9/6/796EF2E4-801B-4FC4-AB28-B59FBF6D907B/VCForPython27.msi | ||
| # - msiexec /i VCForPython27.msi ALLUSERS=1 ACCEPT=YES /passive | ||
|
Czaki marked this conversation as resolved.
Outdated
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,13 +14,15 @@ What does it do? | |
|
|
||
| | | macOS 10.6+ | manylinux i686 | manylinux x86_64 | Windows 32bit | Windows 64bit | | ||
| |---|---|---|---|---|---| | ||
| | Python 2.7 | ✅ | ✅ | ✅ | ✅ | ✅ | | ||
| | Python 3.4 | ✅ | ✅ | ✅ | ✅* | ✅* | | ||
| | Python 2.7 | ✅ | ✅ | ✅ | ✅² | ✅² | | ||
| | Python 3.4 | ✅ | ✅ | ✅ | ✅¹²| ✅¹²| | ||
| | Python 3.5 | ✅ | ✅ | ✅ | ✅ | ✅ | | ||
| | Python 3.6 | ✅ | ✅ | ✅ | ✅ | ✅ | | ||
| | Python 3.7 | ✅ | ✅ | ✅ | ✅ | ✅ | | ||
|
|
||
| > \* Not supported on Azure Pipelines | ||
| > ¹ Not supported on Azure Pipelines | ||
| > | ||
| > ² Current configuration for travis do not support this python version but user can find method for install proper python on travis | ||
|
|
||
| - Builds manylinux, macOS and Windows (32 and 64bit) wheels using Azure Pipelines, Travis CI, AppVeyor, and CircleCI | ||
| - Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate) | ||
|
|
@@ -34,9 +36,10 @@ Usage | |
| | | Linux | macOS | Windows | | ||
| |-----------------|-------|-------|---------| | ||
| | Azure Pipelines | ✅ | ✅ | ✅ | | ||
| | Travis CI | ✅ | ✅ | | | ||
| | Travis CI | ✅ | ✅ | ✅* | | ||
| | AppVeyor | | | ✅ | | ||
| | CircleCI | ✅ | ✅ | | | ||
| > \* Travis provides windows machines but user need install python manually | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fix typos to "Travis provides a Windows environment but Python needs to be installed manually" ? |
||
|
|
||
| `cibuildwheel` is not intended to run on your development machine. It will try to install packages globally; this is no good. Travis CI, CircleCI, and AppVeyor run their builds in isolated environments, so are ideal for this kind of script. | ||
|
|
||
|
|
@@ -101,11 +104,12 @@ jobs: | |
| <summary><b>Travis CI</b> | ||
| <img width="16" src="https://unpkg.com/simple-icons@latest/icons/apple.svg" /> | ||
| <img width="16" src="https://unpkg.com/simple-icons@latest/icons/linux.svg" /> | ||
| <img width="16" src="https://unpkg.com/simple-icons@latest/icons/windows.svg" /> | ||
| </summary> | ||
|
|
||
| - To build Linux and Mac wheels on Travis CI, create a `.travis.yml` file in your repo. | ||
|
|
||
| ``` | ||
| ```yaml | ||
| language: python | ||
|
|
||
| matrix: | ||
|
|
@@ -123,6 +127,16 @@ jobs: | |
| - cibuildwheel --output-dir wheelhouse | ||
| ``` | ||
|
|
||
| and matrix entry for windows | ||
|
Czaki marked this conversation as resolved.
Outdated
|
||
| ```yaml | ||
| - os: windows | ||
| language: shell | ||
| before_install: | ||
| - choco install python3 --version 3.6.8 --no-progress --force -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python36-x64'" | ||
| env: | ||
| - PATH=/c/Python36-x64:/c/Python36-x64/Scripts:$PATH | ||
| ``` | ||
|
|
||
|
Czaki marked this conversation as resolved.
|
||
| Then setup a deployment method by following the [Travis CI deployment docs](https://docs.travis-ci.com/user/deployment/), or see [Delivering to PyPI](#delivering-to-pypi) below. | ||
|
|
||
| </details> | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| __version__ = '0.11.1' | ||
| __version__ = '0.11.2' | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Leave this out, please. @joerick will update the version when he makes a new release :-) |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,18 @@ | ||
| from __future__ import print_function | ||
| import os, tempfile, subprocess, shutil | ||
| import os, tempfile, subprocess, shutil, sys | ||
| from collections import namedtuple | ||
| from glob import glob | ||
|
|
||
| try: | ||
| from shlex import quote as shlex_quote | ||
| except ImportError: | ||
| from pipes import quote as shlex_quote | ||
|
|
||
| from .util import prepare_command, get_build_verbosity_extra_flags | ||
|
|
||
|
|
||
| IS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache') | ||
| IS_RUNNING_ON_TRAVIS = os.environ.get('TRAVIS_OS_NAME') == 'windows' | ||
|
YannickJadoul marked this conversation as resolved.
|
||
|
|
||
| def get_python_path(config): | ||
| if IS_RUNNING_ON_AZURE: | ||
|
|
@@ -31,26 +37,51 @@ def get_python_path(config): | |
| arch = '-x64' if config.arch == '64' else '' | ||
| ) | ||
|
|
||
|
|
||
|
|
||
| def get_python_configurations(build_selector): | ||
| PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'arch', 'identifier', 'path']) | ||
|
|
||
| bace_choco_args = "--no-progress --force -y --allowmultiple --override".split() | ||
| instal_args = "'/quiet InstallAllUsers=1 TargetDir={}'" | ||
|
|
||
| class PythonConfiguration(object): | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just a though: could we keep this as a simple We could even just make a simple function |
||
| def __init__(self, version, arch, identifier, path, choco_args=None): | ||
| self.version = version | ||
| self.arch = arch | ||
| self.identifier = identifier | ||
| self.path = path | ||
| if isinstance(choco_args, str): | ||
| choco_args = choco_args.split() | ||
| if isinstance(choco_args, list): | ||
| self.choco_args = choco_args + bace_choco_args + ["--installargs", instal_args.format(path)] | ||
| else: | ||
| self.choco_args = choco_args | ||
|
|
||
| def __str__(self): | ||
| return "PythonConfiguration({}, {}, {}, {}, {})".format(self.version, self.arch, self.identifier, self.path, str(self.choco_args)) | ||
|
|
||
| python_configurations = [ | ||
| PythonConfiguration(version='2.7.x', arch="32", identifier='cp27-win32', path='C:\Python27'), | ||
| PythonConfiguration(version='2.7.x', arch="64", identifier='cp27-win_amd64', path='C:\Python27-x64'), | ||
| PythonConfiguration(version='3.4.x', arch="32", identifier='cp34-win32', path='C:\Python34'), | ||
| PythonConfiguration(version='3.4.x', arch="64", identifier='cp34-win_amd64', path='C:\Python34-x64'), | ||
| PythonConfiguration(version='3.5.x', arch="32", identifier='cp35-win32', path='C:\Python35'), | ||
| PythonConfiguration(version='3.5.x', arch="64", identifier='cp35-win_amd64', path='C:\Python35-x64'), | ||
| PythonConfiguration(version='3.6.x', arch="32", identifier='cp36-win32', path='C:\Python36'), | ||
| PythonConfiguration(version='3.6.x', arch="64", identifier='cp36-win_amd64', path='C:\Python36-x64'), | ||
| PythonConfiguration(version='3.7.x', arch="32", identifier='cp37-win32', path='C:\Python37'), | ||
| PythonConfiguration(version='3.7.x', arch="64", identifier='cp37-win_amd64', path='C:\Python37-x64'), | ||
| PythonConfiguration(version='3.5.x', arch="32", identifier='cp35-win32', path='C:\Python35', choco_args="python3-x86_32 --version 3.5.2"), | ||
| PythonConfiguration(version='3.5.x', arch="64", identifier='cp35-win_amd64', path='C:\Python35-x64', choco_args="python3 --version 3.5.4"), | ||
| PythonConfiguration(version='3.6.x', arch="32", identifier='cp36-win32', path='C:\Python36', choco_args="python3 --version 3.6.8 --x86"), | ||
| PythonConfiguration(version='3.6.x', arch="64", identifier='cp36-win_amd64', path='C:\Python36-x64', choco_args="python3 --version 3.6.8"), | ||
| PythonConfiguration(version='3.7.x', arch="32", identifier='cp37-win32', path='C:\Python37', choco_args="python3 --version 3.7.4 --x86"), | ||
| PythonConfiguration(version='3.7.x', arch="64", identifier='cp37-win_amd64', path='C:\Python37-x64', choco_args="python3 --version 3.7.4") | ||
| ] | ||
|
|
||
| if IS_RUNNING_ON_AZURE: | ||
| if IS_RUNNING_ON_AZURE or IS_RUNNING_ON_TRAVIS: | ||
| # Python 3.4 isn't supported on Azure. | ||
| # I meet problem with install python on travis. | ||
|
joerick marked this conversation as resolved.
Outdated
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What problems are you getting, exactly? Do you have a link we can use for future reference? Also, please move this line down, to not split up "Python 3.4 isn't supported on Azure." and "See https://..." since these lines belong together.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Exact problem is that python installer do not allow to install two instances of same python version. One of pre-installed program has own instance of python 3.4 x64. But it is installed under strange path. I try to write part of code for discovery installation https://github.com/Czaki/cibuildwheel/commit/b0cefb63c75e17952a19dabe4ffe410b75af3254#diff-bf885b72fecfdf7485cab147f7957d5c but all info which i found in internet is connected with pep 514, but python 3.4 do not satisfy it. should I describe it in comment inside code?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Aha, I see. It's probably just fine to write a single line that this version clashes with an already installed version of Python 3.4 on Travis CI's Windows image. Thanks! |
||
| # See https://github.com/Microsoft/azure-pipelines-tasks/issues/9674 | ||
| python_configurations = [c for c in python_configurations if c.version != '3.4.x'] | ||
|
|
||
| if IS_RUNNING_ON_TRAVIS: | ||
| # cannot install VCForPython27.msi | ||
| python_configurations = [c for c in python_configurations if c.version != '2.7.x'] | ||
|
|
||
| # skip builds as required | ||
| return [c for c in python_configurations if build_selector(c.identifier)] | ||
|
|
@@ -77,10 +108,21 @@ def shell(args, env=None, cwd=None): | |
| temp_dir = tempfile.mkdtemp(prefix='cibuildwheel') | ||
| built_wheel_dir = os.path.join(temp_dir, 'built_wheel') | ||
|
|
||
| python_configurations = get_python_configurations(build_selector) | ||
| def call(args, env=None, cwd=None, shell=False): | ||
| # print the command executing for the logs | ||
| if shell: | ||
| print('+ %s' % args) | ||
| else: | ||
| print('+ ' + ' '.join(shlex_quote(a) for a in args)) | ||
|
|
||
| return subprocess.check_call(args, env=env, cwd=cwd, shell=shell) | ||
|
Czaki marked this conversation as resolved.
Outdated
|
||
|
|
||
| python_configurations = get_python_configurations(build_selector) | ||
| for config in python_configurations: | ||
| print(config, file=sys.stderr) | ||
|
Czaki marked this conversation as resolved.
Outdated
|
||
| config_python_path = get_python_path(config) | ||
| if IS_RUNNING_ON_TRAVIS and config.choco_args is not None and not os.path.exists(config.path): | ||
| call(["choco", "install"] + config.choco_args) | ||
|
|
||
| # check python & pip exist for this configuration | ||
| assert os.path.exists(os.path.join(config_python_path, 'python.exe')) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,7 +6,8 @@ | |
|
|
||
| import subprocess, sys, os | ||
|
|
||
| IS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache') | ||
| IS_WINDOWS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache') | ||
| IS_WINDOWS_RUNNING_ON_TRAVIS = os.environ.get('TRAVIS_OS_NAME') == 'windows' | ||
|
|
||
|
|
||
| def cibuildwheel_get_build_identifiers(project_path, env=None): | ||
|
|
@@ -36,12 +37,12 @@ def cibuildwheel_run(project_path, env=None, add_env=None): | |
| if add_env is not None: | ||
| env.update(add_env) | ||
|
|
||
| subprocess.check_call( | ||
| p = subprocess.check_call( | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What are these changes?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will remove this. Artefact after debug.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, don't worry. Sorry to be too quick about it! |
||
| [sys.executable, '-m', 'cibuildwheel', project_path], | ||
| env=env, | ||
| env=env | ||
| ) | ||
|
|
||
|
|
||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A small spurious change left? |
||
| def expected_wheels(package_name, package_version): | ||
| ''' | ||
| Returns a list of expected wheels from a run of cibuildwheel. | ||
|
|
@@ -85,9 +86,12 @@ def expected_wheels(package_name, package_version): | |
| else: | ||
| raise Exception('unsupported platform') | ||
|
|
||
| if IS_RUNNING_ON_AZURE: | ||
| if IS_WINDOWS_RUNNING_ON_AZURE: | ||
| # Python 3.4 isn't supported on Azure. | ||
| templates = [t for t in templates if '-cp34-' not in t] | ||
| if IS_WINDOWS_RUNNING_ON_TRAVIS: | ||
| # Python 2.7 and 3.4 isn't supported on Travis. | ||
| templates = [t for t in templates if '-cp34-' not in t and '-cp27-' not in t] | ||
|
|
||
| return [filename.format(package_name=package_name, package_version=package_version) | ||
| for filename in templates] | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's not forget about this when merging :-)