Skip to content

fix(tests): bare pytest flags pass through run_tests.sh without a '--' separator - #54008

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-90ecafc5
Jun 28, 2026
Merged

fix(tests): bare pytest flags pass through run_tests.sh without a '--' separator#54008
teknium1 merged 1 commit into
mainfrom
hermes/hermes-90ecafc5

Conversation

@teknium1

@teknium1 teknium1 commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Summary

A bare scripts/run_tests.sh tests/foo.py -q (or -v / -x / -k expr / --tb=long) now works — those pytest flags are forwarded to each per-file pytest invocation without needing a literal -- separator.

Root cause: run_tests_parallel.py only split passthrough args on --, then handed everything before it to its own argparse. Any bare pytest flag hit argparse, which rejected it with error: unrecognized arguments: -q. This contradicted the runner's own docstring ("Common pytest args pass through") and the AGENTS.md examples, so anyone using pytest muscle-memory got an error and had to retry on every run.

Changes

  • scripts/run_tests_parallel.py: argv splitter now routes any token starting with - that isn't one of the runner's own options (-j/--jobs, --paths, --slice, --file-timeout, --generate-slices, --files, --include-integration) straight to pytest. Value-taking pytest flags given space-separated (-k expr, -m mark, -p plugin, -o name=val, …) peel their following token so the value never gets stolen by positional-path discovery. Combined/=-joined forms of our own flags (-j4, --paths=x) stay ours. Docstring updated to match.
  • scripts/run_tests.sh: usage comment reflects bare-flag passthrough.
  • tests/test_run_tests_parallel.py: 4 behavior-contract tests (bare -q runs clean; -k keeps its value and filters to 1 of 2 tests; explicit -- still works and stacks; a positional path stays a discovery root, not a bad flag).

Validation

Case Before After
run_tests.sh tests/foo.py -q error: unrecognized arguments: -q runs clean
-k test_alpha value swallowed → "No test files to run" filters to 1 of 2 tests
tests/foo.py -- --tb=long works still works (stacks with bare flags)
-j4 -q n/a -j4 stays ours, -q → pytest

Targeted suite green via the canonical runner: scripts/run_tests.sh tests/test_run_tests_parallel.py → 5 passed, 0 failed.

Infographic

test-runner-bare-flags

@teknium1
fix(tests): bare pytest flags pass through run_tests.sh without a '--…
337d982 
…' separator

The parallel runner only forwarded pytest args after a literal '--', so a
bare 'scripts/run_tests.sh tests/foo.py -q' (or -v/-x/-k/--tb=long) errored
out with 'unrecognized arguments'. This contradicted the docstring's
promise that common pytest flags pass through, and forced a retry on every
run that used pytest muscle-memory.

Now any token starting with '-' that isn't one of the runner's own options
(-j/--jobs, --paths, --slice, --file-timeout, --generate-slices, --files,
--include-integration) is routed to each per-file pytest invocation
automatically. Value-taking flags given space-separated (-k expr, -m mark,
-p plugin, -o name=val, etc.) keep their value instead of having it stolen
by positional-path discovery. The explicit '--' separator still works and
stacks with bare flags.

- scripts/run_tests_parallel.py: argv splitter routes bare unknown flags to
  pytest; value-flag lookahead; updated docstring.
- scripts/run_tests.sh: usage comment reflects bare-flag passthrough.
- tests/test_run_tests_parallel.py: 4 behavior-contract tests (bare -q runs,
  -k keeps its value/filters, '--' still works, positional path stays a root).
@github-actions

github-actions Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🔎 Lint report: hermes/hermes-90ecafc5 vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 11611 on HEAD, 11611 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 6094 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@teknium1
teknium1 merged commit 2523917 into main Jun 28, 2026
30 checks passed
@teknium1
teknium1 deleted the hermes/hermes-90ecafc5 branch June 28, 2026 05:43
@alt-glitch alt-glitch added type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have labels Jun 28, 2026
@alt-glitch

alt-glitch commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: this is a competing fix for #42189 alongside open PRs #42195 (earliest/canonical) and #42204. This PR routes leading-dash tokens to pytest selectively (peeling values for -k/-m/-p/-o); #42195 and #42204 use a parse_known_args fallback. Same goal, different mechanism — flagging for a maintainer to pick one. Not marking duplicate since the approaches differ.

pai-scaffolde pushed a commit to pai-scaffolde/hermes-agent that referenced this pull request Jun 28, 2026
@teknium1 @pai-scaffolde
fix(tests): bare pytest flags pass through run_tests.sh without a '--…
4b572d2 
…' separator (NousResearch#54008)

The parallel runner only forwarded pytest args after a literal '--', so a
bare 'scripts/run_tests.sh tests/foo.py -q' (or -v/-x/-k/--tb=long) errored
out with 'unrecognized arguments'. This contradicted the docstring's
promise that common pytest flags pass through, and forced a retry on every
run that used pytest muscle-memory.

Now any token starting with '-' that isn't one of the runner's own options
(-j/--jobs, --paths, --slice, --file-timeout, --generate-slices, --files,
--include-integration) is routed to each per-file pytest invocation
automatically. Value-taking flags given space-separated (-k expr, -m mark,
-p plugin, -o name=val, etc.) keep their value instead of having it stolen
by positional-path discovery. The explicit '--' separator still works and
stacks with bare flags.

- scripts/run_tests_parallel.py: argv splitter routes bare unknown flags to
  pytest; value-flag lookahead; updated docstring.
- scripts/run_tests.sh: usage comment reflects bare-flag passthrough.
- tests/test_run_tests_parallel.py: 4 behavior-contract tests (bare -q runs,
  -k keeps its value/filters, '--' still works, positional path stays a root).
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
@teknium1
fix(tests): bare pytest flags pass through run_tests.sh without a '--…
e10ba2a 
…' separator (NousResearch#54008)

The parallel runner only forwarded pytest args after a literal '--', so a
bare 'scripts/run_tests.sh tests/foo.py -q' (or -v/-x/-k/--tb=long) errored
out with 'unrecognized arguments'. This contradicted the docstring's
promise that common pytest flags pass through, and forced a retry on every
run that used pytest muscle-memory.

Now any token starting with '-' that isn't one of the runner's own options
(-j/--jobs, --paths, --slice, --file-timeout, --generate-slices, --files,
--include-integration) is routed to each per-file pytest invocation
automatically. Value-taking flags given space-separated (-k expr, -m mark,
-p plugin, -o name=val, etc.) keep their value instead of having it stolen
by positional-path discovery. The explicit '--' separator still works and
stacks with bare flags.

- scripts/run_tests_parallel.py: argv splitter routes bare unknown flags to
  pytest; value-flag lookahead; updated docstring.
- scripts/run_tests.sh: usage comment reflects bare-flag passthrough.
- tests/test_run_tests_parallel.py: 4 behavior-contract tests (bare -q runs,
  -k keeps its value/filters, '--' still works, positional path stays a root).
Jasper6439 pushed a commit to Jasper6439/hermes-agent that referenced this pull request Jul 5, 2026
@teknium1 Jasper6439
fix(tests): bare pytest flags pass through run_tests.sh without a '--…
05851cf 
…' separator (NousResearch#54008)

The parallel runner only forwarded pytest args after a literal '--', so a
bare 'scripts/run_tests.sh tests/foo.py -q' (or -v/-x/-k/--tb=long) errored
out with 'unrecognized arguments'. This contradicted the docstring's
promise that common pytest flags pass through, and forced a retry on every
run that used pytest muscle-memory.

Now any token starting with '-' that isn't one of the runner's own options
(-j/--jobs, --paths, --slice, --file-timeout, --generate-slices, --files,
--include-integration) is routed to each per-file pytest invocation
automatically. Value-taking flags given space-separated (-k expr, -m mark,
-p plugin, -o name=val, etc.) keep their value instead of having it stolen
by positional-path discovery. The explicit '--' separator still works and
stacks with bare flags.

- scripts/run_tests_parallel.py: argv splitter routes bare unknown flags to
  pytest; value-flag lookahead; updated docstring.
- scripts/run_tests.sh: usage comment reflects bare-flag passthrough.
- tests/test_run_tests_parallel.py: 4 behavior-contract tests (bare -q runs,
  -k keeps its value/filters, '--' still works, positional path stays a root).
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
@teknium1
fix(tests): bare pytest flags pass through run_tests.sh without a '--…
19c187c 
…' separator (NousResearch#54008)

The parallel runner only forwarded pytest args after a literal '--', so a
bare 'scripts/run_tests.sh tests/foo.py -q' (or -v/-x/-k/--tb=long) errored
out with 'unrecognized arguments'. This contradicted the docstring's
promise that common pytest flags pass through, and forced a retry on every
run that used pytest muscle-memory.

Now any token starting with '-' that isn't one of the runner's own options
(-j/--jobs, --paths, --slice, --file-timeout, --generate-slices, --files,
--include-integration) is routed to each per-file pytest invocation
automatically. Value-taking flags given space-separated (-k expr, -m mark,
-p plugin, -o name=val, etc.) keep their value instead of having it stolen
by positional-path discovery. The explicit '--' separator still works and
stacks with bare flags.

- scripts/run_tests_parallel.py: argv splitter routes bare unknown flags to
  pytest; value-flag lookahead; updated docstring.
- scripts/run_tests.sh: usage comment reflects bare-flag passthrough.
- tests/test_run_tests_parallel.py: 4 behavior-contract tests (bare -q runs,
  -k keeps its value/filters, '--' still works, positional path stays a root).
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
@teknium1
fix(tests): bare pytest flags pass through run_tests.sh without a '--…
8de3973 
…' separator (NousResearch#54008)

The parallel runner only forwarded pytest args after a literal '--', so a
bare 'scripts/run_tests.sh tests/foo.py -q' (or -v/-x/-k/--tb=long) errored
out with 'unrecognized arguments'. This contradicted the docstring's
promise that common pytest flags pass through, and forced a retry on every
run that used pytest muscle-memory.

Now any token starting with '-' that isn't one of the runner's own options
(-j/--jobs, --paths, --slice, --file-timeout, --generate-slices, --files,
--include-integration) is routed to each per-file pytest invocation
automatically. Value-taking flags given space-separated (-k expr, -m mark,
-p plugin, -o name=val, etc.) keep their value instead of having it stolen
by positional-path discovery. The explicit '--' separator still works and
stacks with bare flags.

- scripts/run_tests_parallel.py: argv splitter routes bare unknown flags to
  pytest; value-flag lookahead; updated docstring.
- scripts/run_tests.sh: usage comment reflects bare-flag passthrough.
- tests/test_run_tests_parallel.py: 4 behavior-contract tests (bare -q runs,
  -k keeps its value/filters, '--' still works, positional path stays a root).
This was referenced Jul 14, 2026
Closed
Closed
Open
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
@teknium1
fix(tests): bare pytest flags pass through run_tests.sh without a '--…
82c08aa 
…' separator (NousResearch#54008)

The parallel runner only forwarded pytest args after a literal '--', so a
bare 'scripts/run_tests.sh tests/foo.py -q' (or -v/-x/-k/--tb=long) errored
out with 'unrecognized arguments'. This contradicted the docstring's
promise that common pytest flags pass through, and forced a retry on every
run that used pytest muscle-memory.

Now any token starting with '-' that isn't one of the runner's own options
(-j/--jobs, --paths, --slice, --file-timeout, --generate-slices, --files,
--include-integration) is routed to each per-file pytest invocation
automatically. Value-taking flags given space-separated (-k expr, -m mark,
-p plugin, -o name=val, etc.) keep their value instead of having it stolen
by positional-path discovery. The explicit '--' separator still works and
stacks with bare flags.

- scripts/run_tests_parallel.py: argv splitter routes bare unknown flags to
  pytest; value-flag lookahead; updated docstring.
- scripts/run_tests.sh: usage comment reflects bare-flag passthrough.
- tests/test_run_tests_parallel.py: 4 behavior-contract tests (bare -q runs,
  -k keeps its value/filters, '--' still works, positional path stays a root).
This was referenced Jul 29, 2026
Closed
Open
Open
Open
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
@teknium1
fix(tests): bare pytest flags pass through run_tests.sh without a '--…
915c2f8 
…' separator (NousResearch#54008)

The parallel runner only forwarded pytest args after a literal '--', so a
bare 'scripts/run_tests.sh tests/foo.py -q' (or -v/-x/-k/--tb=long) errored
out with 'unrecognized arguments'. This contradicted the docstring's
promise that common pytest flags pass through, and forced a retry on every
run that used pytest muscle-memory.

Now any token starting with '-' that isn't one of the runner's own options
(-j/--jobs, --paths, --slice, --file-timeout, --generate-slices, --files,
--include-integration) is routed to each per-file pytest invocation
automatically. Value-taking flags given space-separated (-k expr, -m mark,
-p plugin, -o name=val, etc.) keep their value instead of having it stolen
by positional-path discovery. The explicit '--' separator still works and
stacks with bare flags.

- scripts/run_tests_parallel.py: argv splitter routes bare unknown flags to
  pytest; value-flag lookahead; updated docstring.
- scripts/run_tests.sh: usage comment reflects bare-flag passthrough.
- tests/test_run_tests_parallel.py: 4 behavior-contract tests (bare -q runs,
  -k keeps its value/filters, '--' still works, positional path stays a root).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@teknium1 @alt-glitch