-
-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add docker image from the YAML config integration #3339
Changes from 3 commits
442c68a
961c18e
aa15e78
a705f9d
eb7c4b2
8d1a58d
c27686e
67d5595
5708f10
edfca36
d0ee0f7
da7ec2b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -69,6 +69,39 @@ The file option specified the Conda `environment file`_ to use. | |
|
||
.. note:: Conda is only supported via the YAML file. | ||
|
||
|
||
build | ||
~~~~~ | ||
|
||
The ``build`` block configures specific aspects of the documentation build. | ||
|
||
.. _yaml_build_image: | ||
|
||
build.image | ||
``````````` | ||
|
||
* Default: ``2.0`` | ||
* Options: ``1.0``, ``2.0``, ``latest`` | ||
|
||
The docker image to use for specific builds. | ||
This lets users specify a more experimental builder, | ||
if they want to be on the cutting edge. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For consistency and clarity: "docker image" -> "build image" |
||
|
||
Certain Python versions require a certain builder, | ||
as defined here:: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same on "builder" here, builder is an internal term, this refers to the build image instead. |
||
|
||
* '1.0': 2, 2.7, 3, 3.4 | ||
* '2.0': 2, 2.7, 3, 3.5 | ||
* 'latest': 2, 2.7, 3, 3.3, 3.4, 3.5, 3.6 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You might be able to pull this from settings directly, similar to Literal formatting would make this more clear |
||
|
||
.. code-block:: yaml | ||
|
||
build: | ||
image: latest | ||
|
||
python: | ||
version: 3.6 | ||
|
||
python | ||
~~~~~~ | ||
|
||
|
@@ -85,15 +118,12 @@ This is the version of Python to use when building your documentation. If you | |
specify only the major version of Python, the highest supported minor version | ||
will be selected. | ||
|
||
The supported Python versions depends on the version of the build image your | ||
project is using. The default build image that is used to build documentation | ||
contains support for Python ``2.7`` and ``3.5``. | ||
|
||
There is also an image in testing that supports Python versions ``2.7``, | ||
``3.3``, ``3.4``, ``3.5``, and ``3.6``. If you would like access to this build | ||
image, you can sign up for beta access here: | ||
.. warning:: | ||
|
||
https://goo.gl/forms/AKEoeWHixlzVfqKT2 | ||
The supported Python versions depends on the version of the build image your | ||
project is using. The default build image that is used to build documentation | ||
contains support for Python ``2.7`` and ``3.5``. | ||
See the :ref:`yaml_build_image` for more information on supported Python versions. | ||
|
||
.. code-block:: yaml | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,8 +8,6 @@ | |
from readthedocs_build.config import load as load_config | ||
from readthedocs_build.config import BuildConfig, ConfigError, InvalidConfig | ||
|
||
from .constants import DOCKER_BUILD_IMAGES, DOCKER_IMAGE | ||
|
||
|
||
class ConfigWrapper(object): | ||
|
||
|
@@ -59,7 +57,7 @@ def python_interpreter(self): | |
list( | ||
filter( | ||
lambda x: x < ver + 1, | ||
self._yaml_config.get_valid_python_versions(), | ||
self._yaml_config.PYTHON_SUPPORTED_VERSIONS, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Raised issue on similar change in |
||
))) | ||
return 'python{0}'.format(ver) | ||
|
||
|
@@ -108,6 +106,15 @@ def formats(self): | |
formats += ['pdf'] | ||
return formats | ||
|
||
@property | ||
def build_image(self): | ||
if self._project.container_image: | ||
# Allow us to override per-project still | ||
assert 'readthedocs/build' in self._project.container_image, ( | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How does this Shouldn't we raise a proper exception like ConfigError or InvalidConfig here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. seems we shouldn't need to do validation here at all actually -- There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yea, I know I've set it wrong before in the admin, so like having clarity there. Can definitely raise other errors though :) |
||
'container image must be fully qualified') | ||
return self._project.container_image | ||
return 'readthedocs/build:{}'.format(self._yaml_config['build']['image']) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we want the user to just put the last part? Would it be interesting if I can put something like Just an idea (probably stupid, but...) / comment... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we want users installing their own images on the server. For private instances, I think the addition of Also, this shouldn't hard code the build image prefix here. Instead, just do lookups on |
||
|
||
# Not implemented until we figure out how to keep in sync with the webs. | ||
# Probably needs to be version-specific as well, not project. | ||
# @property | ||
|
@@ -126,22 +133,8 @@ def load_yaml_config(version): | |
parsing consistent between projects. | ||
""" | ||
checkout_path = version.project.checkout_path(version.slug) | ||
env_config = {} | ||
|
||
# Get build image to set up the python version validation. Pass in the | ||
# build image python limitations to the loaded config so that the versions | ||
# can be rejected at validation | ||
build_image = DOCKER_BUILD_IMAGES.get( | ||
version.project.container_image, | ||
DOCKER_BUILD_IMAGES.get(DOCKER_IMAGE, None), | ||
) | ||
if build_image: | ||
env_config = { | ||
'python': build_image['python'], | ||
} | ||
|
||
try: | ||
sphinx_env_config = env_config.copy() | ||
sphinx_env_config = {} | ||
sphinx_env_config.update({ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You can just create the dict at this line instead of |
||
'output_base': '', | ||
'type': 'sphinx', | ||
|
@@ -155,8 +148,9 @@ def load_yaml_config(version): | |
# This is a subclass of ConfigError, so has to come first | ||
raise | ||
except ConfigError: | ||
# Just fall back to defaults | ||
config = BuildConfig( | ||
env_config=env_config, | ||
env_config={}, | ||
raw_config={}, | ||
source_file='empty', | ||
source_position=0, | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -40,17 +40,3 @@ | |
DOCKER_OOM_EXIT_CODE = 137 | ||
|
||
DOCKER_HOSTNAME_MAX_LEN = 64 | ||
|
||
# Build images | ||
DOCKER_BUILD_IMAGES = { | ||
'readthedocs/build:1.0': { | ||
'python': {'supported_versions': [2, 2.7, 3, 3.4]}, | ||
}, | ||
'readthedocs/build:2.0': { | ||
'python': {'supported_versions': [2, 2.7, 3, 3.5]}, | ||
}, | ||
'readthedocs/build:latest': { | ||
'python': {'supported_versions': [2, 2.7, 3, 3.3, 3.4, 3.5, 3.6]}, | ||
}, | ||
} | ||
DOCKER_BUILD_IMAGES.update(getattr(settings, 'DOCKER_BUILD_IMAGES', {})) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think keeping full build image names here on |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -46,54 +46,6 @@ def setUp(self): | |
install_project=False, requirements_file='urls.py') | ||
self.version = get(Version, project=self.project) | ||
|
||
def test_python_supported_versions_default_image_1_0(self, load_config): | ||
load_config.side_effect = create_load() | ||
self.project.container_image = 'readthedocs/build:1.0' | ||
self.project.save() | ||
config = load_yaml_config(self.version) | ||
self.assertEqual(load_config.call_count, 1) | ||
load_config.assert_has_calls([ | ||
mock.call(path=mock.ANY, env_config={ | ||
'python': {'supported_versions': [2, 2.7, 3, 3.4]}, | ||
'type': 'sphinx', | ||
'output_base': '', | ||
'name': mock.ANY | ||
}), | ||
]) | ||
self.assertEqual(config.python_version, 2) | ||
|
||
def test_python_supported_versions_image_2_0(self, load_config): | ||
load_config.side_effect = create_load() | ||
self.project.container_image = 'readthedocs/build:2.0' | ||
self.project.save() | ||
config = load_yaml_config(self.version) | ||
self.assertEqual(load_config.call_count, 1) | ||
load_config.assert_has_calls([ | ||
mock.call(path=mock.ANY, env_config={ | ||
'python': {'supported_versions': [2, 2.7, 3, 3.5]}, | ||
'type': 'sphinx', | ||
'output_base': '', | ||
'name': mock.ANY | ||
}), | ||
]) | ||
self.assertEqual(config.python_version, 2) | ||
|
||
def test_python_supported_versions_image_latest(self, load_config): | ||
load_config.side_effect = create_load() | ||
self.project.container_image = 'readthedocs/build:latest' | ||
self.project.save() | ||
config = load_yaml_config(self.version) | ||
self.assertEqual(load_config.call_count, 1) | ||
load_config.assert_has_calls([ | ||
mock.call(path=mock.ANY, env_config={ | ||
'python': {'supported_versions': [2, 2.7, 3, 3.3, 3.4, 3.5, 3.6]}, | ||
'type': 'sphinx', | ||
'output_base': '', | ||
'name': mock.ANY | ||
}), | ||
]) | ||
self.assertEqual(config.python_version, 2) | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These are some of the tests I was looking for in |
||
def test_python_default_version(self, load_config): | ||
load_config.side_effect = create_load() | ||
config = load_yaml_config(self.version) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With settings re-enabled, this can be pulled from settings. See
settings.rst
anddoc_extensions.py
inconf/