Latest sphinx clobbers pinned sphinx version (which breaks autosummary)

[chummels]

Details

Starting about 6 months ago, the autosummary functionality in our docs spontaneously stopped working without any changes on our end, and it left us with no API docs (see: https://trident.readthedocs.io/en/latest/reference.html). Upon discussion with devs of other RTD-using projects, it appeared that the problem was in the newest version of Sphinx. Thus, I attempted to pin the version of sphinx used in our RTD build to an older version, similarly to what worked for the other devs. I updated our conda environments.yml file to include an old version of sphinx, since that is what we use in our readthedocs.yml file to specify the build environment. Unfortunately, RTD installs the old version successfully, but then it clobbers this version by installing the latest sphinx immediately afterwards. Is there any way to have RTD not install the latest sphinx version, so that we can get autosummary working again?

If you look in the build (see URL below), you'll see where it correctly installs the old version of sphinx as:
conda env create --quiet --name latest --file environment.yml

but then the following line updates to the latest version of sphinx:
conda install --yes --quiet --name latest mock pillow sphinx sphinx_rtd_theme

• Read the Docs project URL: https://readthedocs.org/projects/trident/
• Build URL (if applicable): https://readthedocs.org/projects/trident/builds/12766973/

Expected Result

Leave the version of sphinx to the one specified in the conda environments.yml file.

Actual Result

Gets clobbered by the newest version of sphinx.

[humitos]

Hi! If you already pinned the versions you want to install in your environment.yml file you may be hitting this issue #3829. There is a workaround for this issue that involved enabling the feature flag (https://docs.readthedocs.io/en/stable/guides/feature-flags.html#feature-flags) CONDA_APPEND_CORE_REQUIREMENTS in your project.

Users can't enable these flags, so I enabled it for your project and triggered a new build at https://readthedocs.org/projects/trident/builds/12774141/. Please report back if your issue is solved with this.

[eric-wieser]

Does the FORCE_SPHINX_FROM_VENV feature flag help here?

Edit: Looks like as of two hours ago that flag is now always on: e5bd237

[stsewd]

Edit: Looks like as of two hours ago that flag is now always on: e5bd237

@eric-wieser that commit isn't live yet, but the flag is always on since some days ago.

[stsewd]

I can see that the CONDA_APPEND_CORE_REQUIREMENTS flag is working. From the logs

Running Sphinx v3.2.1