Reduce Windows runtime duration of some linkcheck tests by enabling timeouts

[jayaddison]

Purpose

This pull request is an experiment to prove/disprove a theory I have about the reason for some recently-added linkcheck builder case sensitivity tests taking unexpectedly long durations of time (4 seconds plus) to run on Windows in GitHub Actions CI.

I think that there may be some network/resource-cleanup side-effects from calling socket.setdefaulttimeout in an earlier test -- and I would like to confirm whether this is the case, and also if it is, then whether we can undo the effects of that (and/or report upstream if it is not already a known problem).

Result: the theory was disproven; the socket.setdefaulttimeout is not the direct cause. The absence of configured linkcheck_timeout settings for the linkcheck-case-check was determined to be the fault.

To reduce the cost of this experiment, non-Windows GitHub Actions CI workflows and jobs are disabled.

References

• Relates to linkcheck: Allow case-insensitive URL comparisons #14046.

[jayaddison]

@AA-Turner @FazeelUsmani I usually focus intensely on pointless annoying quirks/puzzles such as this one until I find some kind of answer -- in this case, I am really, really struggling to figure out why the test_linkcheck_case_sensitivity test case runs slowly -- it only seems to affect Windows.

I've tested a bunch of theories and attempted various things to try to narrow it down -- and I still don't even have a sense of what the factor that affects it is.

I don't know whether it's worth spending much/any time on, but if you do have a few moments, I'd be glad for your help and ideas.

[jayaddison]

Perhaps I've been too focused on what does exist within the scope of this test case -- without stopping to consider what doesn't exist in it, but is found in the other linkcheck test cases.

Specifically: I am thinking of the presence of linkcheck_timeout in the conf.py files for the other linkcheck testroots.

[jayaddison]

Self-quoting, and cc @FazeelUsmani @AA-Turner:

Specifically: I am thinking of the presence of linkcheck_timeout in the conf.py files for the other linkcheck testroots.

Yes, the absence of the linkcheck_timeout configuration was the reason for the long-duration tests; adding it in ad2ebf1 has reduced the runtime of the test_linkcheck_case_sensitivity tests to a more reasonable ~1s or so each.

This may affect a few other linkcheck testroots, so I'll do a quick audit of those and add the config to any others where it's relevant.

[jayaddison]

Requested linkcheck testroots

$ grep -o "testroot='[a-z\-]*'" tests/test_builders/test_build_linkcheck.py | sort | uniq -c
      2 testroot='linkcheck'
      1 testroot='linkcheck-anchors-ignore'
      1 testroot='linkcheck-anchors-ignore-for-url'
      1 testroot='linkcheck-case-check'
     18 testroot='linkcheck-localserver'
      3 testroot='linkcheck-localserver-anchor'
      6 testroot='linkcheck-localserver-https'
      1 testroot='linkcheck-localserver-warn-redirects'
      1 testroot='linkcheck-raw-node'
      1 testroot='linkcheck-too-many-retries'
      6 testroot='root'

Configured linkcheck_timeout values

$ grep -rwl linkcheck_timeout tests/roots | sort
tests/roots/test-linkcheck-anchors-ignore-for-url/conf.py
tests/roots/test-linkcheck-anchors-ignore/conf.py
tests/roots/test-linkcheck-case-check/conf.py
tests/roots/test-linkcheck-documents_exclude/conf.py
tests/roots/test-linkcheck-localserver-anchor/conf.py
tests/roots/test-linkcheck-localserver-https/conf.py
tests/roots/test-linkcheck-localserver-warn-redirects/conf.py
tests/roots/test-linkcheck-localserver/conf.py
tests/roots/test-linkcheck-raw-node/conf.py
tests/roots/test-linkcheck-too-many-retries/conf.py
tests/roots/test-linkcheck/conf.py

So: I think the only other affected testroot -- that would benefit the test suite duration on Windows by enabling linkcheck_timeout -- is the root one.

[jayaddison]

Result from commit 2e81411: one test, test_raw_node, remains puzzlingly slow at ~3s -- however, the overall time for the test_build_linkcheck.py file reduced from ~2m30s to ~1m30s -- so, that's cautiously optimistic news.

I'll rebase the conf.py edits, resolve/hide the outdated comments, and open this for review.

[AA-Turner]

Timeouts were first added in 97f07ca (several subsequent commits tweak the values).

A

[jayaddison]

Thanks @AA-Turner 👍