Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,50 @@ matrix:
include:
# Linux Python 2
- sudo: required
if: branch=master

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not forget about this when merging :-)

language: python
python: 2.7
services: docker
env: PYTHON=python

# Linux Python 3
- sudo: required
if: branch=master
language: python
python: 3.4
services: docker
env: PYTHON=python

# macOS Python 2
- os: osx
if: branch=master
env: PYTHON=python2

# macOS Python 3
- os: osx
if: branch=master
env: PYTHON=python3
before_install:
- brew update
- brew outdated python || brew upgrade python

- os: windows
language: shell
before_install:
- choco install python3 --version 3.5.4 --no-progress --force -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python35-x64'"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for future reference, is there a reason we need all of these different arguments, here?
https://docs.travis-ci.com/user/languages/python/ seems to be suggesting to just choco install python (or maybe python3, I guess) and use the python executable afterwards.

I understand we need more flags/control for the different versions, but for the one that's just running cibuildwheel in the examples, it would be great to keep things simple.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--no-progress means not show progress of download. In other cases the 4MB limit of log on travis may run out.
-y accept all questions during installation.
-allowmultiple allow multiple versions of package. Not sure if need here. Needed with next installations
--override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python35-x64'" - set install directory

install:
- C:\\Python35-x64\\python -m pip install -r requirements-dev.txt
script:
- C:\\Python35-x64\\python ./bin/run_tests.py

install: $PYTHON -m pip install -r requirements-dev.txt

script: $PYTHON ./bin/run_tests.py


# - choco install python3-x86_32 --version 3.4.3.20150501 --no-progress -y --allowmultiple --override --installargs "'/qn /norestart ALLUSERS=1 TARGETDIR=C:\Python34'"
# - choco install python3 --version 3.4.4.20180111 --no-progress --force -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python34-x64'"
# - choco install python2 --version 2.7.16 --no-progress --x86 -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python27'"
# - choco install python2 --version 2.7.16 --no-progress -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python27-x64'"
# - wget -q http://download.microsoft.com/download/7/9/6/796EF2E4-801B-4FC4-AB28-B59FBF6D907B/VCForPython27.msi
# - msiexec /i VCForPython27.msi ALLUSERS=1 ACCEPT=YES /passive
Comment thread
Czaki marked this conversation as resolved.
Outdated
24 changes: 19 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@ What does it do?

| | macOS 10.6+ | manylinux i686 | manylinux x86_64 | Windows 32bit | Windows 64bit |
|---|---|---|---|---|---|
| Python 2.7 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.4 | ✅ | ✅ | ✅ | ✅* | ✅* |
| Python 2.7 | ✅ | ✅ | ✅ | ✅² | ✅² |
| Python 3.4 | ✅ | ✅ | ✅ | ✅¹²| ✅¹²|
| Python 3.5 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.6 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.7 | ✅ | ✅ | ✅ | ✅ | ✅ |

> \* Not supported on Azure Pipelines
> ¹ Not supported on Azure Pipelines
>
> ² Current configuration for travis do not support this python version but user can find method for install proper python on travis

- Builds manylinux, macOS and Windows (32 and 64bit) wheels using Azure Pipelines, Travis CI, AppVeyor, and CircleCI
- Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
Expand All @@ -34,9 +36,10 @@ Usage
| | Linux | macOS | Windows |
|-----------------|-------|-------|---------|
| Azure Pipelines | ✅ | ✅ | ✅ |
| Travis CI | ✅ | ✅ | |
| Travis CI | ✅ | ✅ | ✅* |
| AppVeyor | | | ✅ |
| CircleCI | ✅ | ✅ | |
> \* Travis provides windows machines but user need install python manually

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix typos to "Travis provides a Windows environment but Python needs to be installed manually" ?


`cibuildwheel` is not intended to run on your development machine. It will try to install packages globally; this is no good. Travis CI, CircleCI, and AppVeyor run their builds in isolated environments, so are ideal for this kind of script.

Expand Down Expand Up @@ -101,11 +104,12 @@ jobs:
<summary><b>Travis CI</b>
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/apple.svg" />
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/linux.svg" />
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/windows.svg" />
</summary>

- To build Linux and Mac wheels on Travis CI, create a `.travis.yml` file in your repo.

```
```yaml
language: python

matrix:
Expand All @@ -123,6 +127,16 @@ jobs:
- cibuildwheel --output-dir wheelhouse
```

and matrix entry for windows
Comment thread
Czaki marked this conversation as resolved.
Outdated
```yaml
- os: windows
language: shell
before_install:
- choco install python3 --version 3.6.8 --no-progress --force -y --allowmultiple --override --installargs "'/quiet InstallAllUsers=1 TargetDir=C:\Python36-x64'"
env:
- PATH=/c/Python36-x64:/c/Python36-x64/Scripts:$PATH
```

Comment thread
Czaki marked this conversation as resolved.
Then setup a deployment method by following the [Travis CI deployment docs](https://docs.travis-ci.com/user/deployment/), or see [Delivering to PyPI](#delivering-to-pypi) below.

</details>
Expand Down
2 changes: 1 addition & 1 deletion cibuildwheel/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = '0.11.1'
__version__ = '0.11.2'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave this out, please. @joerick will update the version when he makes a new release :-)

2 changes: 2 additions & 0 deletions cibuildwheel/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ def main():
platform = 'linux'
elif os.environ.get('TRAVIS_OS_NAME') == 'osx':
platform = 'macos'
elif os.environ.get('TRAVIS_OS_NAME') == 'windows':
platform = 'windows'
elif 'APPVEYOR' in os.environ:
platform = 'windows'
elif 'BITRISE_BUILD_NUMBER' in os.environ:
Expand Down
62 changes: 52 additions & 10 deletions cibuildwheel/windows.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
from __future__ import print_function
import os, tempfile, subprocess, shutil
import os, tempfile, subprocess, shutil, sys
from collections import namedtuple
from glob import glob

try:
from shlex import quote as shlex_quote
except ImportError:
from pipes import quote as shlex_quote

from .util import prepare_command, get_build_verbosity_extra_flags


IS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
IS_RUNNING_ON_TRAVIS = os.environ.get('TRAVIS_OS_NAME') == 'windows'
Comment thread
YannickJadoul marked this conversation as resolved.

def get_python_path(config):
if IS_RUNNING_ON_AZURE:
Expand All @@ -31,26 +37,51 @@ def get_python_path(config):
arch = '-x64' if config.arch == '64' else ''
)



def get_python_configurations(build_selector):
PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'arch', 'identifier', 'path'])

bace_choco_args = "--no-progress --force -y --allowmultiple --override".split()
instal_args = "'/quiet InstallAllUsers=1 TargetDir={}'"

class PythonConfiguration(object):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a though: could we keep this as a simple namedtuple, and format/create the full args when we need them.

We could even just make a simple function get_choco_args that takes the version and arch and returns all the arguments? Then we don't need to change the current PythonConfiguration.

def __init__(self, version, arch, identifier, path, choco_args=None):
self.version = version
self.arch = arch
self.identifier = identifier
self.path = path
if isinstance(choco_args, str):
choco_args = choco_args.split()
if isinstance(choco_args, list):
self.choco_args = choco_args + bace_choco_args + ["--installargs", instal_args.format(path)]
else:
self.choco_args = choco_args

def __str__(self):
return "PythonConfiguration({}, {}, {}, {}, {})".format(self.version, self.arch, self.identifier, self.path, str(self.choco_args))

python_configurations = [
PythonConfiguration(version='2.7.x', arch="32", identifier='cp27-win32', path='C:\Python27'),
PythonConfiguration(version='2.7.x', arch="64", identifier='cp27-win_amd64', path='C:\Python27-x64'),
PythonConfiguration(version='3.4.x', arch="32", identifier='cp34-win32', path='C:\Python34'),
PythonConfiguration(version='3.4.x', arch="64", identifier='cp34-win_amd64', path='C:\Python34-x64'),
PythonConfiguration(version='3.5.x', arch="32", identifier='cp35-win32', path='C:\Python35'),
PythonConfiguration(version='3.5.x', arch="64", identifier='cp35-win_amd64', path='C:\Python35-x64'),
PythonConfiguration(version='3.6.x', arch="32", identifier='cp36-win32', path='C:\Python36'),
PythonConfiguration(version='3.6.x', arch="64", identifier='cp36-win_amd64', path='C:\Python36-x64'),
PythonConfiguration(version='3.7.x', arch="32", identifier='cp37-win32', path='C:\Python37'),
PythonConfiguration(version='3.7.x', arch="64", identifier='cp37-win_amd64', path='C:\Python37-x64'),
PythonConfiguration(version='3.5.x', arch="32", identifier='cp35-win32', path='C:\Python35', choco_args="python3-x86_32 --version 3.5.2"),
PythonConfiguration(version='3.5.x', arch="64", identifier='cp35-win_amd64', path='C:\Python35-x64', choco_args="python3 --version 3.5.4"),
PythonConfiguration(version='3.6.x', arch="32", identifier='cp36-win32', path='C:\Python36', choco_args="python3 --version 3.6.8 --x86"),
PythonConfiguration(version='3.6.x', arch="64", identifier='cp36-win_amd64', path='C:\Python36-x64', choco_args="python3 --version 3.6.8"),
PythonConfiguration(version='3.7.x', arch="32", identifier='cp37-win32', path='C:\Python37', choco_args="python3 --version 3.7.4 --x86"),
PythonConfiguration(version='3.7.x', arch="64", identifier='cp37-win_amd64', path='C:\Python37-x64', choco_args="python3 --version 3.7.4")
]

if IS_RUNNING_ON_AZURE:
if IS_RUNNING_ON_AZURE or IS_RUNNING_ON_TRAVIS:
# Python 3.4 isn't supported on Azure.
# I meet problem with install python on travis.
Comment thread
joerick marked this conversation as resolved.
Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What problems are you getting, exactly? Do you have a link we can use for future reference? Also, please move this line down, to not split up "Python 3.4 isn't supported on Azure." and "See https://..." since these lines belong together.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact problem is that python installer do not allow to install two instances of same python version. One of pre-installed program has own instance of python 3.4 x64. But it is installed under strange path. I try to write part of code for discovery installation https://github.com/Czaki/cibuildwheel/commit/b0cefb63c75e17952a19dabe4ffe410b75af3254#diff-bf885b72fecfdf7485cab147f7957d5c but all info which i found in internet is connected with pep 514, but python 3.4 do not satisfy it.

should I describe it in comment inside code?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha, I see. It's probably just fine to write a single line that this version clashes with an already installed version of Python 3.4 on Travis CI's Windows image. Thanks!

# See https://github.com/Microsoft/azure-pipelines-tasks/issues/9674
python_configurations = [c for c in python_configurations if c.version != '3.4.x']

if IS_RUNNING_ON_TRAVIS:
# cannot install VCForPython27.msi
python_configurations = [c for c in python_configurations if c.version != '2.7.x']

# skip builds as required
return [c for c in python_configurations if build_selector(c.identifier)]
Expand All @@ -77,10 +108,21 @@ def shell(args, env=None, cwd=None):
temp_dir = tempfile.mkdtemp(prefix='cibuildwheel')
built_wheel_dir = os.path.join(temp_dir, 'built_wheel')

python_configurations = get_python_configurations(build_selector)
def call(args, env=None, cwd=None, shell=False):
# print the command executing for the logs
if shell:
print('+ %s' % args)
else:
print('+ ' + ' '.join(shlex_quote(a) for a in args))

return subprocess.check_call(args, env=env, cwd=cwd, shell=shell)
Comment thread
Czaki marked this conversation as resolved.
Outdated

python_configurations = get_python_configurations(build_selector)
for config in python_configurations:
print(config, file=sys.stderr)
Comment thread
Czaki marked this conversation as resolved.
Outdated
config_python_path = get_python_path(config)
if IS_RUNNING_ON_TRAVIS and config.choco_args is not None and not os.path.exists(config.path):
call(["choco", "install"] + config.choco_args)

# check python & pip exist for this configuration
assert os.path.exists(os.path.join(config_python_path, 'python.exe'))
Expand Down
16 changes: 10 additions & 6 deletions test/shared/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@

import subprocess, sys, os

IS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
IS_WINDOWS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
IS_WINDOWS_RUNNING_ON_TRAVIS = os.environ.get('TRAVIS_OS_NAME') == 'windows'


def cibuildwheel_get_build_identifiers(project_path, env=None):
Expand Down Expand Up @@ -36,12 +37,12 @@ def cibuildwheel_run(project_path, env=None, add_env=None):
if add_env is not None:
env.update(add_env)

subprocess.check_call(
p = subprocess.check_call(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these changes?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will remove this. Artefact after debug.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, don't worry. Sorry to be too quick about it!

[sys.executable, '-m', 'cibuildwheel', project_path],
env=env,
env=env
)


Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small spurious change left?

def expected_wheels(package_name, package_version):
'''
Returns a list of expected wheels from a run of cibuildwheel.
Expand Down Expand Up @@ -85,9 +86,12 @@ def expected_wheels(package_name, package_version):
else:
raise Exception('unsupported platform')

if IS_RUNNING_ON_AZURE:
if IS_WINDOWS_RUNNING_ON_AZURE:
# Python 3.4 isn't supported on Azure.
templates = [t for t in templates if '-cp34-' not in t]
if IS_WINDOWS_RUNNING_ON_TRAVIS:
# Python 2.7 and 3.4 isn't supported on Travis.
templates = [t for t in templates if '-cp34-' not in t and '-cp27-' not in t]

return [filename.format(package_name=package_name, package_version=package_version)
for filename in templates]
Expand Down