Implement sphinx key from v2 config #4482

stsewd · 2018-08-06T21:46:57Z

This PR on top of #4456. It should be merged after.

New commits are from 6756b7f

humitos · 2018-08-15T18:58:24Z

I will wait until the base PR gets merged to review this.

stsewd · 2018-08-25T00:25:10Z

readthedocs/rtd_tests/tests/test_config_integration.py

@@ -232,7 +232,6 @@ def test_requirements_file(self, load_config):
        self.assertEqual(config.python.requirements, '__init__.py')


-@pytest.mark.skip


Tests are un-skipped here

stsewd · 2018-08-25T00:26:19Z

Rebased and updated! I'll try to manually test this by morning monday

stsewd · 2018-08-27T19:06:43Z

Ok, I need to figure out how to keep the previous behavior that creates a conf.py file if no one is found and use that.

stsewd · 2018-08-27T19:32:24Z

I'll like to merge this first #4566, as I'm touching some related code and I want to make sure I'm not breaking that code.

stsewd · 2018-08-27T21:41:45Z

readthedocs/rtd_tests/tests/test_doc_builder.py

-    def test_create_conf_py(
-            self, get_conf_py_path, _, get_config_params,
+    @patch('readthedocs.projects.models.Project.checkout_path')
+    def test_multiple_conf_py(


Silly me, left this test with a duplicated name...

stsewd · 2018-08-27T21:42:46Z

readthedocs/rtd_tests/tests/test_doc_builder.py

@@ -97,14 +114,24 @@ def test_create_conf_py(
        """

        tmp_docs_dir = py.path.local(tempfile.mkdtemp())
-        tmp_docs_dir.join('conf.py').new()
-        tmp_docs_dir.join('test').mkdir().join('conf.py').new()
+        tmp_docs_dir.join('conf.py').write('')


Looks like new() doesn't create the files on disk

agjohnson · 2018-08-27T23:18:01Z

#4456 is merged, this needs a rebase.

agjohnson

Looks good so far! Noted a couple places that would be helpful to ensure our logic isn't changing, or at least is not likely to introduce tricky bugs.

agjohnson · 2018-08-27T23:29:27Z

readthedocs/doc_builder/backends/sphinx.py

+            outfile_path = (
+                self.config.sphinx.configuration or
+                self.project.conf_file(self.version.slug)
+            )


It seems here we're implying that if you set a sphinx configuration file, but it doesn't exist, we just create it? I feel like we should error out if you set up a config file and it doesn't actually exist, it will be confusing to the user for the build to pass in this case.

This is to keep compatibility with the behavior when the user doesn't set a config file and rtd tries to find one.

That part I understand, but this seemed to be creating a 3rd option, where the config was specified but not found, and therefore the config was generated instead. This might not be an issue, I interpreted self.config.sphinx.configuration as defaulting to None if the file is specified but doesn't exist, but that doesn't seem to be the logic used looking at it again.

agjohnson · 2018-08-27T23:33:15Z

readthedocs/doc_builder/backends/sphinx.py

+        config_file = (
+            self.config.sphinx.configuration or
+            self.project.conf_file(self.version.slug)
+        )


Same here. And I feel like we shouldn't be guessing here. If the config file was specified and it exists, that's when we should set the config file path. Likewise, if we're generating a config, we set a config file path when we find out the user doesn't have a config and didn't specify a config file path.

This will cut down on magical decisions that tend to hide errors from us.

agjohnson · 2018-08-27T23:34:30Z

readthedocs/doc_builder/backends/sphinx.py

-        cwd = self.project.conf_dir(self.version.slug)
+        config_file = self.config.sphinx.configuration
+        config_dir = os.path.dirname(config_file) if config_file else None
+        cwd = config_dir or self.project.conf_dir(self.version.slug)


This logic looks duplicated, is there a way to reduce this duplication if so?

I knew you were not going to like it p: haha, so all this is to make sure that we are keeping the previous behavior. If a config file doesn't exist, we create one, and also we try to find one in the whole project. I guess we can do a big refactor to do this step early, before parsing the config file (but not sure if this worth it). Or maybe we can use a config_file property inside the sphinx builder and modify that as we want

Yeah, this could either be a property, or even just simply set the common values on __init__ in base?

agjohnson · 2018-08-27T23:36:16Z

readthedocs/doc_builder/config.py

+        sphinx_configuration = path.join(
+            version.get_conf_py_path(),
+            'conf.py'
+        )


It's not clear to me why this change is needed. Shouldn't the full path have already been returned by version.get_conf_py_path()?

I was bitten by the name here too, this only returns the directory name. There is a test unskipped that covers this case.

huh. is there a reason why this was working then? It seems like it shouldn't have worked at all if it was just the directory.

We didn't have a test, but I added them in https://github.com/rtfd/readthedocs.org/pull/4456/files#diff-5a2cc4147895068b33fe5bf89e4e6e08R589 (skipped)

Also currently we are reading this from the database, that's why the builds don't fail

stsewd · 2018-08-29T03:20:37Z

readthedocs/doc_builder/backends/sphinx.py

-            outfile = codecs.open(outfile_path, encoding='utf-8', mode='a')
+            self.config_file = (
+                self.config_file or
+                self.project.conf_file(self.version.slug)


I need to keep this one, as here we catch and re-raise the exception to the user

stsewd · 2018-08-29T05:56:37Z

V1 is still working, I refactored the code. I'll be testing v2 in the morning.

stsewd · 2018-08-29T17:44:47Z

OK, v2 tested! It works :D

agjohnson · 2018-09-07T16:54:03Z

Changes look good!

stsewd mentioned this pull request Aug 7, 2018

Implement mkdocs key from v2 config #4486

Merged

stsewd mentioned this pull request Aug 15, 2018

Match v1 config interface to new one #4456

Merged

stsewd added 12 commits August 24, 2018 18:47

Test conf.py default value

f7dc3bd

Fix conf.py path

09ea0dc

Don't skip tests

0648f65

Expose config object in builders

e9db42c

Use doctype from config

7fe49ee

Pass python_env in tests

954e277

Fix old test

292f912

Implement sphinx key

19810f4

Implement fail_on_warning

fb48f7c

Skip tests for mkdocs

1083999

Fix raising wrong exception

d35c23c

Fix tests

58adc35

stsewd force-pushed the build-config-v2-phinx branch from 64cd38a to 58adc35 Compare August 25, 2018 00:04

Remove search from tests

819a9fa

stsewd commented Aug 25, 2018

View reviewed changes

Return the absolute path

e6b9dd5

stsewd added the PR: work in progress Pull request is not ready for full review label Aug 27, 2018

stsewd added 5 commits August 27, 2018 15:33

Check for None

db40aa7

Use the config file

b4c22dd

Fix test

6ee2d66

Merge branch 'master' into build-config-v2-phinx

4750efd

Fix test

aa25732

stsewd commented Aug 27, 2018

View reviewed changes

agjohnson added this to the YAML File Completion milestone Aug 27, 2018

agjohnson requested changes Aug 27, 2018

View reviewed changes

stsewd added 2 commits August 28, 2018 21:42

Remove repetition

189726c

Break and fix test

dc99eb8

stsewd commented Aug 29, 2018

View reviewed changes

stsewd removed the PR: work in progress Pull request is not ready for full review label Aug 29, 2018

stsewd requested a review from agjohnson September 4, 2018 16:22

agjohnson approved these changes Sep 7, 2018

View reviewed changes

agjohnson merged commit 8705717 into readthedocs:master Sep 7, 2018

stsewd deleted the build-config-v2-phinx branch September 7, 2018 16:54

pdavide mentioned this pull request Sep 28, 2018

"Edit on GitHub" url broken for sphinx projects #4671

Closed

stsewd mentioned this pull request Nov 8, 2018

Support sphinx warn option readthedocs/readthedocs-build#7

Closed

stsewd mentioned this pull request Dec 8, 2018

Support failing on Sphinx warnings readthedocs/readthedocs-build#25

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement sphinx key from v2 config #4482

Implement sphinx key from v2 config #4482

stsewd commented Aug 6, 2018 •

edited

Loading

humitos commented Aug 15, 2018

stsewd Aug 25, 2018

stsewd commented Aug 25, 2018

stsewd commented Aug 27, 2018

stsewd commented Aug 27, 2018

stsewd Aug 27, 2018

stsewd Aug 27, 2018

agjohnson commented Aug 27, 2018

agjohnson left a comment

agjohnson Aug 27, 2018

stsewd Aug 27, 2018

agjohnson Aug 28, 2018

agjohnson Aug 27, 2018

agjohnson Aug 27, 2018

stsewd Aug 27, 2018 •

edited

Loading

agjohnson Aug 28, 2018

agjohnson Aug 27, 2018

stsewd Aug 28, 2018

agjohnson Aug 28, 2018

stsewd Aug 28, 2018

stsewd Aug 28, 2018

stsewd Aug 29, 2018

stsewd commented Aug 29, 2018

stsewd commented Aug 29, 2018

agjohnson commented Sep 7, 2018

		@@ -232,7 +232,6 @@ def test_requirements_file(self, load_config):
		self.assertEqual(config.python.requirements, '__init__.py')


		@pytest.mark.skip

Implement sphinx key from v2 config #4482

Implement sphinx key from v2 config #4482

Conversation

stsewd commented Aug 6, 2018 • edited Loading

humitos commented Aug 15, 2018

Choose a reason for hiding this comment

stsewd commented Aug 25, 2018

stsewd commented Aug 27, 2018

stsewd commented Aug 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agjohnson commented Aug 27, 2018

agjohnson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stsewd Aug 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stsewd commented Aug 29, 2018

stsewd commented Aug 29, 2018

agjohnson commented Sep 7, 2018

stsewd commented Aug 6, 2018 •

edited

Loading

stsewd Aug 27, 2018 •

edited

Loading