Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop -E from Sphinx call #10701

Closed
mgeier opened this issue Sep 3, 2023 · 6 comments · Fixed by #11073
Closed

Drop -E from Sphinx call #10701

mgeier opened this issue Sep 3, 2023 · 6 comments · Fixed by #11073
Labels
Accepted Accepted issue on our roadmap Improvement Minor improvement to code

Comments

@mgeier
Copy link
Contributor

mgeier commented Sep 3, 2023

A few years ago (see #6966), there was some effort to speed up subsequent builds with different builders by sharing the "doctree" directory.

A remnant of this is still visible since the Sphinx arguments contain -d _build/doctrees.

However, I recently noticed that the Sphinx command now also contain the -E flag, which destroys most of the savings expected from sharing doctrees.

If the -E flag is intentionally used for some reason, the -d argument should probably be removed to avoid confusion.

Details

Expected Result

Different builders re-use the doctrees, sources are only read for the first build, all subsequent builds are very quick.

Actual Result

Each build reads all source files, not re-using previous doctrees.

@humitos
Copy link
Member

humitos commented Sep 3, 2023

Thanks for reporting this. We should probably remove both since there is no shared state between each build anymore.

@humitos
Copy link
Member

humitos commented Sep 4, 2023

@mgeier Hrm... Thinking a little more about this. Since we are building the HTML and then the PDF, aren't these toctrees shared between those commands?

@humitos humitos added the Needed: more information A reply from issue author is required label Sep 4, 2023
@mgeier
Copy link
Contributor Author

mgeier commented Sep 4, 2023

Since we are building the HTML and then the PDF, aren't these toctrees shared between those commands?

I thought that was the intention a few years ago, and this is normally achieved by making sure to specify the same doctree directory with -d, which is the case here.

However, this is all obliterated by the -E flag, which means don't use a saved environment, always read all files.

So even though the doctree directory is technically shared and re-used, because of the -E flag all source files are still read and Sphinx's environment is re-built from scratch.

@humitos humitos added Needed: design decision A core team decision is required and removed Needed: more information A reply from issue author is required labels Sep 5, 2023
@humitos
Copy link
Member

humitos commented Sep 5, 2023

Thanks for the clarification here.

@stsewd @ericholscher what do you think it's the correct movement here?

  • Should we remove both options and keep it working as it's now?
  • Should we remove only -E and get the benefits of sharing the parsed doctree between HTML, ePUB and PDF?

Since the root directory is always clean, I think it makes sense to only remove -E which will speed up some projects. In particular, those with many files and building multiple versions.

@stsewd
Copy link
Member

stsewd commented Sep 5, 2023

+1 on removing the -E option

@ericholscher
Copy link
Member

Yea, I think we don't need it anymore since we aren't reusing build directories across builds.

@humitos humitos added Needed: design decision A core team decision is required Accepted Accepted issue on our roadmap Improvement Minor improvement to code and removed Needed: design decision A core team decision is required labels Sep 12, 2023
@humitos humitos added this to the Build stability milestone Sep 12, 2023
@humitos humitos changed the title Drop -E from Sphinx call Drop -E from Sphinx call Sep 12, 2023
humitos added a commit that referenced this issue Jan 29, 2024
Remove the `-E` from Sphinx command so it can shares the saved environment
between commands from the same build.

This will speed up a little those builds that are building multiple formats.

Closes #10701
humitos added a commit that referenced this issue Jan 30, 2024
Remove the `-E` from Sphinx command so it can shares the saved environment
between commands from the same build.

This will speed up a little those builds that are building multiple formats.

Closes #10701
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Accepted Accepted issue on our roadmap Improvement Minor improvement to code
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants