Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save docker image hash to consider when auto wiping the environment #3793

Merged
merged 8 commits into from
Mar 23, 2018

Conversation

humitos
Copy link
Member

@humitos humitos commented Mar 14, 2018

Closes #3744

Now, we can access `self.config.build_image` directly.
At initialization time we have the project and we already know if the
project has the build image override so we can decide at that point
and save it as a instance attribute.

Then we can use this values from other places inside the same class.
The hash is used to know if the environment is obsolete and auto-wipe
it if necessary.
@agjohnson agjohnson added this to the Build stability milestone Mar 16, 2018
Copy link
Contributor

@agjohnson agjohnson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! We'll be careful as we're deploy this, as we'll want to make sure we're not overloading the build servers. I noted this in our deploy checklist though.

# If the user define the Python version just as a major version
# (e.g. ``2`` or ``3``) we won't know exactly which exact version was
# used to create the venv but we can still compare it against the new
# one coming from the project version config.
return any([
env_python_version != self.config.python_full_version,
env_build_image != build_image,
env_build_hash != self.build_env.image_hash,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will force wipe all existing builds, but perhaps that is a good thing. We'll have explicit metadata after the next build of all of these projects.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm...

This won't force the wipe because it will fail in the L134 (https://github.com/rtfd/readthedocs.org/pull/3793/files/05d74aec013e1b211798ba02e91636d331794a1d#diff-31f8391d8643c47c6caedf53930c0b7eR134) when trying to access the build.hash that will produce a KeyError exception and we will return False.

Then, the current build with the old environment will be used but we will be saving the current new attributes and... we have a bug here, right?


@pytest.mark.xfail(
reason='PythonEnvironment needs to be refactored to do not rely on DockerBuildEnvironment',
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be good to lay out in a new ticket

@agjohnson agjohnson merged commit 7ac7dfc into master Mar 23, 2018
@agjohnson agjohnson deleted the humitos/rtdenvironment/save-image-hash branch March 23, 2018 17:22
agjohnson added a commit that referenced this pull request Mar 26, 2018
humitos added a commit that referenced this pull request Mar 29, 2018
…3793)

* Remove obsolete code

Now, we can access `self.config.build_image` directly.

* Move container_image selection to the init

At initialization time we have the project and we already know if the
project has the build image override so we can decide at that point
and save it as a instance attribute.

Then we can use this values from other places inside the same class.

* Save Docker Image hash in readthedocs-environment.json

The hash is used to know if the environment is obsolete and auto-wipe
it if necessary.

* Simplify the class naming

* Save the image hash in the json file

* Lint

* Remove invalid properties from YAML config in tests

* Add test for save_environment_json
agjohnson pushed a commit that referenced this pull request Mar 31, 2018
* Save docker image hash to consider when auto wiping the environment (#3793)

* Remove obsolete code

Now, we can access `self.config.build_image` directly.

* Move container_image selection to the init

At initialization time we have the project and we already know if the
project has the build image override so we can decide at that point
and save it as a instance attribute.

Then we can use this values from other places inside the same class.

* Save Docker Image hash in readthedocs-environment.json

The hash is used to know if the environment is obsolete and auto-wipe
it if necessary.

* Simplify the class naming

* Save the image hash in the json file

* Lint

* Remove invalid properties from YAML config in tests

* Add test for save_environment_json

* Improve docstring

* Handle obsolete cases better

* when the file is corrupted or we don't have access, we return that
  it's OBSOLETE

* when there is a new setting that we need to compare and it's not in
  the JSON file, we return OBSOLETE

* Test case for build image in the config but not in the JSON
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants