Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,24 @@ matrix:
- brew update
- brew outdated python || brew upgrade python

- os: windows
language: shell
env: VS100COMNTOOLS=$VS140COMNTOOLS
Comment thread
Czaki marked this conversation as resolved.
Outdated
before_install:
- choco install python3 --version 3.6.8 --no-progress -y
install:
- C:\\Python36\\python -m pip install -r requirements-dev.txt
script:
- C:\\Python36\\python ./bin/run_tests.py

install: $PYTHON -m pip install -r requirements-dev.txt

script: $PYTHON ./bin/run_tests.py


# - choco install python3-x86_32 --version 3.4.3.20150501 --no-progress -y --allowmultiple --override --installargs "'/qn /norestart ALLUSERS=1 TARGETDIR=C:\Python34'"
# - choco install python3 --version 3.4.4.20180111 --no-progress --force -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python34-x64'"
# - choco install python2 --version 2.7.16 --no-progress --x86 -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python27'"
# - choco install python2 --version 2.7.16 --no-progress -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python27-x64'"
# - wget -q http://download.microsoft.com/download/7/9/6/796EF2E4-801B-4FC4-AB28-B59FBF6D907B/VCForPython27.msi
# - msiexec /i VCForPython27.msi ALLUSERS=1 ACCEPT=YES /passive
Comment thread
Czaki marked this conversation as resolved.
Outdated
24 changes: 19 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@ What does it do?

| | macOS 10.6+ | manylinux i686 | manylinux x86_64 | Windows 32bit | Windows 64bit |
|---|---|---|---|---|---|
| Python 2.7 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.4 | ✅ | ✅ | ✅ | ✅* | ✅* |
| Python 2.7 | ✅ | ✅ | ✅ | ✅² | ✅² |
| Python 3.4 | ✅ | ✅ | ✅ | ✅¹²| ✅¹²|
| Python 3.5 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.6 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.7 | ✅ | ✅ | ✅ | ✅ | ✅ |

> \* Not supported on Azure Pipelines
> ¹ Not supported on Azure Pipelines
>
> ² Current configuration for travis do not support this python version but user can find method for install proper python on travis. For python 2.7 there is a problem with installation "Microsoft Visual C++ Compiler for Python 2.7"
Comment thread
Czaki marked this conversation as resolved.
Outdated

- Builds manylinux, macOS and Windows (32 and 64bit) wheels using Azure Pipelines, Travis CI, AppVeyor, and CircleCI
- Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
Expand All @@ -34,9 +36,10 @@ Usage
| | Linux | macOS | Windows |
|-----------------|-------|-------|---------|
| Azure Pipelines | ✅ | ✅ | ✅ |
| Travis CI | ✅ | ✅ | |
| Travis CI | ✅ | ✅ | ✅* |
| AppVeyor | | | ✅ |
| CircleCI | ✅ | ✅ | |
> \* Travis provides a Windows environment but Python needs to be installed manually
Comment thread
Czaki marked this conversation as resolved.
Outdated

`cibuildwheel` is not intended to run on your development machine. It will try to install packages globally; this is no good. Travis CI, CircleCI, and AppVeyor run their builds in isolated environments, so are ideal for this kind of script.

Expand Down Expand Up @@ -101,11 +104,12 @@ jobs:
<summary><b>Travis CI</b>
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/apple.svg" />
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/linux.svg" />
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/windows.svg" />
</summary>

- To build Linux and Mac wheels on Travis CI, create a `.travis.yml` file in your repo.

```
```yaml
language: python

matrix:
Expand All @@ -123,6 +127,16 @@ jobs:
- cibuildwheel --output-dir wheelhouse
```

and matrix entry for windows
Comment thread
Czaki marked this conversation as resolved.
Outdated
```yaml
- os: windows
language: shell
before_install:
- choco install python3 --version 3.6.8 --no-progress -y
env:
- PATH=/c/Python36:/c/Python36/Scripts:$PATH
```

Comment thread
Czaki marked this conversation as resolved.
Then setup a deployment method by following the [Travis CI deployment docs](https://docs.travis-ci.com/user/deployment/), or see [Delivering to PyPI](#delivering-to-pypi) below.

</details>
Expand Down
2 changes: 1 addition & 1 deletion cibuildwheel/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = '0.11.1'
__version__ = '0.11.2'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave this out, please. @joerick will update the version when he makes a new release :-)

2 changes: 2 additions & 0 deletions cibuildwheel/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ def main():
platform = 'linux'
elif os.environ.get('TRAVIS_OS_NAME') == 'osx':
platform = 'macos'
elif os.environ.get('TRAVIS_OS_NAME') == 'windows':
platform = 'windows'
elif 'APPVEYOR' in os.environ:
platform = 'windows'
elif 'BITRISE_BUILD_NUMBER' in os.environ:
Expand Down
89 changes: 72 additions & 17 deletions cibuildwheel/windows.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
from __future__ import print_function
import os, tempfile, subprocess, shutil
import os, tempfile, subprocess, shutil, sys
from collections import namedtuple
from glob import glob

try:
from shlex import quote as shlex_quote
except ImportError:
from pipes import quote as shlex_quote

from .util import prepare_command, get_build_verbosity_extra_flags


IS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
IS_RUNNING_ON_TRAVIS = os.environ.get('TRAVIS_OS_NAME') == 'windows'
Comment thread
YannickJadoul marked this conversation as resolved.

def get_python_path(config):
if IS_RUNNING_ON_AZURE:
Expand All @@ -22,6 +28,12 @@ def get_python_path(config):
return glob(path_pattern)[0]
except IndexError:
raise Exception('Could not find a Python install at ' + path_pattern)
elif IS_RUNNING_ON_TRAVIS:
if config.version == "3.4.x":
return config.path
Comment thread
joerick marked this conversation as resolved.
else:
nuget_args = get_nuget_args(config)
return os.path.join(nuget_args[-1], nuget_args[0] + "." + config.exact_version, "tools")
else:
# Assume we're running on AppVeyor
major, minor = config.version.split('.')[:2]
Expand All @@ -32,32 +44,49 @@ def get_python_path(config):
)


def get_nuget_args(configuration):
if configuration.exact_version is None:
return None
python_name = "python" if configuration.version[0] == '3' else "python2"
if configuration.arch == "32":
python_name = python_name + "x86"
return [python_name, "-Version", configuration.exact_version, "-OutputDirectory", "C:/python"]

def get_python_configurations(build_selector):
PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'arch', 'identifier', 'path'])
PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'arch', 'identifier', 'path', "exact_version"])
Comment thread
Czaki marked this conversation as resolved.
Outdated
python_configurations = [
PythonConfiguration(version='2.7.x', arch="32", identifier='cp27-win32', path='C:\Python27'),
PythonConfiguration(version='2.7.x', arch="64", identifier='cp27-win_amd64', path='C:\Python27-x64'),
PythonConfiguration(version='3.4.x', arch="32", identifier='cp34-win32', path='C:\Python34'),
PythonConfiguration(version='3.4.x', arch="64", identifier='cp34-win_amd64', path='C:\Python34-x64'),
PythonConfiguration(version='3.5.x', arch="32", identifier='cp35-win32', path='C:\Python35'),
PythonConfiguration(version='3.5.x', arch="64", identifier='cp35-win_amd64', path='C:\Python35-x64'),
PythonConfiguration(version='3.6.x', arch="32", identifier='cp36-win32', path='C:\Python36'),
PythonConfiguration(version='3.6.x', arch="64", identifier='cp36-win_amd64', path='C:\Python36-x64'),
PythonConfiguration(version='3.7.x', arch="32", identifier='cp37-win32', path='C:\Python37'),
PythonConfiguration(version='3.7.x', arch="64", identifier='cp37-win_amd64', path='C:\Python37-x64'),
PythonConfiguration(version='2.7.x', arch="32", identifier='cp27-win32', path='C:\Python27', exact_version="2.7.16"),
PythonConfiguration(version='2.7.x', arch="64", identifier='cp27-win_amd64', path='C:\Python27-x64', exact_version="2.7.16"),
PythonConfiguration(version='3.4.x', arch="32", identifier='cp34-win32', path='C:\Python34', exact_version=None),
PythonConfiguration(version='3.4.x', arch="64", identifier='cp34-win_amd64', path='C:\Python34-x64', exact_version=None),
PythonConfiguration(version='3.5.x', arch="32", identifier='cp35-win32', path='C:\Python35', exact_version="3.5.4"),
PythonConfiguration(version='3.5.x', arch="64", identifier='cp35-win_amd64', path='C:\Python35-x64', exact_version="3.5.4"),
PythonConfiguration(version='3.6.x', arch="32", identifier='cp36-win32', path='C:\Python36', exact_version="3.6.8"),
PythonConfiguration(version='3.6.x', arch="64", identifier='cp36-win_amd64', path='C:\Python36-x64', exact_version="3.6.8"),
PythonConfiguration(version='3.7.x', arch="32", identifier='cp37-win32', path='C:\Python37', exact_version="3.7.4"),
PythonConfiguration(version='3.7.x', arch="64", identifier='cp37-win_amd64', path='C:\Python37-x64', exact_version="3.7.4")
]

if IS_RUNNING_ON_AZURE:
# Python 3.4 isn't supported on Azure.
# See https://github.com/Microsoft/azure-pipelines-tasks/issues/9674
python_configurations = [c for c in python_configurations if c.version != '3.4.x']

if IS_RUNNING_ON_TRAVIS:
# cannot install VCForPython27.msi which is needed for compiling C software
# try with (and similar): msiexec /i VCForPython27.msi ALLUSERS=1 ACCEPT=YES /passive
# no easy and stable way fo installing python 3.4
python_configurations = [c for c in python_configurations if c.version != '2.7.x' and c.version != '3.4.x']

# skip builds as required
python_configurations = [c for c in python_configurations if build_selector(c.identifier)]

return python_configurations

# skip builds as required
return [c for c in python_configurations if build_selector(c.identifier)]


def build(project_dir, output_dir, test_command, test_requires, test_extras, before_build, build_verbosity, build_selector, environment):
if IS_RUNNING_ON_AZURE:
if IS_RUNNING_ON_AZURE or IS_RUNNING_ON_TRAVIS:
def shell(args, env=None, cwd=None):
print('+ ' + ' '.join(args))
args = ['cmd', '/E:ON', '/V:ON', '/C'] + args
Expand All @@ -77,10 +106,32 @@ def shell(args, env=None, cwd=None):
temp_dir = tempfile.mkdtemp(prefix='cibuildwheel')
built_wheel_dir = os.path.join(temp_dir, 'built_wheel')

python_configurations = get_python_configurations(build_selector)
def call(args, env=None, cwd=None, shell=False):
# print the command executing for the logs
if shell:
print('+ %s' % args)
else:
print('+ ' + ' '.join(shlex_quote(a) for a in args))

return subprocess.check_call(args, env=env, cwd=cwd, shell=shell)
Comment thread
Czaki marked this conversation as resolved.
Outdated

if IS_RUNNING_ON_TRAVIS:
# instal nuget as best way for provide python
call(["choco", "install", "nuget.commandline"])
# get pip fo this installation which not have.
get_pip_url = 'https://bootstrap.pypa.io/get-pip.py'
get_pip_script = 'C:\get-pip.py'
call(['curl', '-L', '-o', get_pip_script, get_pip_url])

python_configurations = get_python_configurations(build_selector)
for config in python_configurations:
print(config, file=sys.stderr)
Comment thread
Czaki marked this conversation as resolved.
Outdated
config_python_path = get_python_path(config)
if IS_RUNNING_ON_TRAVIS and config.exact_version is not None and not os.path.exists(config_python_path):
call(["nuget", "install"] + get_nuget_args(config))
if not os.path.exists(os.path.join(config_python_path, 'Scripts', 'pip.exe')):
call([os.path.join(config_python_path, 'python.exe'), get_pip_script ])
print(config_python_path, file=sys.stderr)
Comment thread
Czaki marked this conversation as resolved.
Outdated

# check python & pip exist for this configuration
assert os.path.exists(os.path.join(config_python_path, 'python.exe'))
Expand All @@ -107,8 +158,12 @@ def shell(args, env=None, cwd=None):
shell(['python', '-c', '"import struct; print(struct.calcsize(\'P\') * 8)\"'], env=env)

# prepare the Python environment
shell(['python', '-m', 'pip', 'install', '--upgrade', 'pip'],
if config.version == "3.4.x":
shell(['python', '-m', 'pip', 'install', 'pip==19.1.1'],
env=env)
else:
shell(['python', '-m', 'pip', 'install', '--upgrade', 'pip'],
env=env)
Comment thread
joerick marked this conversation as resolved.
shell(['pip', 'install', '--upgrade', 'setuptools'], env=env)
shell(['pip', 'install', 'wheel'], env=env)

Expand Down
1 change: 1 addition & 0 deletions test/01_basic/cibuildwheel_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,5 @@ def test_build_identifiers():
# identifiers
expected_wheels = utils.expected_wheels('spam', '0.1.0')
build_identifiers = utils.cibuildwheel_get_build_identifiers(project_dir)
print("bb", build_identifiers)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this, still?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem. Very easily overlooked. I was just lucky to notice it ;-)

assert len(expected_wheels) == len(build_identifiers)
14 changes: 9 additions & 5 deletions test/shared/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@

import subprocess, sys, os

IS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
IS_WINDOWS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
IS_WINDOWS_RUNNING_ON_TRAVIS = os.environ.get('TRAVIS_OS_NAME') == 'windows'


def cibuildwheel_get_build_identifiers(project_path, env=None):
Expand Down Expand Up @@ -38,10 +39,10 @@ def cibuildwheel_run(project_path, env=None, add_env=None):

subprocess.check_call(
[sys.executable, '-m', 'cibuildwheel', project_path],
env=env,
env=env
)


Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small spurious change left?

def expected_wheels(package_name, package_version):
'''
Returns a list of expected wheels from a run of cibuildwheel.
Expand Down Expand Up @@ -85,9 +86,12 @@ def expected_wheels(package_name, package_version):
else:
raise Exception('unsupported platform')

if IS_RUNNING_ON_AZURE:
if IS_WINDOWS_RUNNING_ON_AZURE or IS_WINDOWS_RUNNING_ON_TRAVIS:
# Python 3.4 isn't supported on Azure.
templates = [t for t in templates if '-cp34-' not in t]
if IS_WINDOWS_RUNNING_ON_TRAVIS:
# Python 2.7 and 3.4 isn't supported on Travis.
templates = [t for t in templates if '-cp27-' not in t and '-cp34-' not in t]
Comment thread
joerick marked this conversation as resolved.

return [filename.format(package_name=package_name, package_version=package_version)
for filename in templates]
Expand Down