tokens: speed up duplication of Tokens objects

[oliver-sanders]

Sometimes we need to duplicate a Tokens object.

Currently this works by doing tokenise(detokenise(tokens)) which is really inefficient (I should know better, I've moaned about this in the cycler classes before!).

This PR switches to using the already parsed values held in the tokens object.

Using the example from #5315 with -s TASKS=25 -s CYCLES=1 this reduces the time taken by increment_graph_window by ~50% after the changes from #5319 and #5321 have been applied.

Before (with #5319 & #5321)

After (still with #5319 & #5321)

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to both setup.cfg and conda-environment.yml.
• Tests are included (or explain why tests are not needed).
• CHANGES.md entry included if this is a change that can affect users
• Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
• If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[hjoliver]

Nice!

[datamel]

In the integration tests, the id is currently a posix path which is breaking them with this change. I'm guessing mypy doesn't check tests.

This diff fixes it:

diff --git a/tests/integration/test_reinstall.py b/tests/integration/test_reinstall.py
index 02be406bc..8b13c5574 100644
--- a/tests/integration/test_reinstall.py
+++ b/tests/integration/test_reinstall.py
@@ -80,7 +80,7 @@ def one_run(one_src, test_dir, run_dir):
     )
     return SimpleNamespace(
         path=w_run_dir,
-        id=w_run_dir.relative_to(run_dir),
+        id=str(w_run_dir.relative_to(run_dir)),
     )
 

[hjoliver]

That's a genuine test fail, in tests/integration/test_reinstall.py:

        if args:
            if len(args) > 1:
                raise ValueError()
            if isinstance(args[0], str):
                kwargs = tokenise(args[0], relative)
            else:
>               kwargs = dict(args[0])
E               TypeError: 'PosixPath' object is not iterable

cylc/flow/id.py:113: TypeError

[hjoliver]

Fix?

diff --git a/tests/integration/test_reinstall.py b/tests/integration/test_reinstall.py
index 02be406bc..8b13c5574 100644
--- a/tests/integration/test_reinstall.py
+++ b/tests/integration/test_reinstall.py
@@ -80,7 +80,7 @@ def one_run(one_src, test_dir, run_dir):
     )
     return SimpleNamespace(
         path=w_run_dir,
-        id=w_run_dir.relative_to(run_dir),
+        id=str(w_run_dir.relative_to(run_dir)),
     )

[hjoliver]

Argghhh, @datamel bet me to it!!! (she must have been working late!). Still, same fix suggested - great minds think alike 😁

[oliver-sanders]

Ta for the fix both :)

[oliver-sanders]

Added a collective changelog for all three efficiency changes.

[MetRonnie]

(only linkcheck failing)