Research runtime: fail closed when GPT Researcher reports zero sources

[seathatflowsinourveins]

Scope

• What this PR changes: tools/research/gpt_researcher.sh now FAILS CLOSED (exit 3, with a clear stderr message) when a research run reports zero sources, or when its report's sources cannot be counted. Before, an empty report read as success. The review round (7d01a9bc) does four things:

  • refuses a query with a line break (exit 2), because upstream writes the query unescaped into the frontmatter the guard reads;
  • makes the guard split lines at \n only, as upstream joins them;
  • states the exact exit contract;
  • notes that sources_count counts attempted URLs.

  Tests cover zero sources, eight uncountable report shapes, line-break refusal, a U+2028 forgery and exit-status pass-through. The retriever stays duckduckgo; the config bytes and the install-plan row are unchanged.

• Base commit: ecfa11276 (main after trading: NativeStack2604 research runtime, EdgarTools 5.60.0 (runtime and catalog), hash-pinned build backend #703). Head: 7d01a9bc.

• Lane: lane:foundation.

• Owned paths touched: tools/research/gpt_researcher.sh, tests/test_research_entry.py.

Diagnosis, from the pinned sources:

• Probable cause: throttling. This is an inference, not a measurement. The zero-source run of 2026-10-05 (20261005T222529Z-1387309) most probably met search engines that were throttling this host.
  • Of the 18 reports written that day up to and including it, the other 17 had 5-17 sources.
  • The empty run started 23 s after another run.
  • The same query, re-run through this head at 23:41Z, found 9 sources, so the zero was not a property of the query.
  • It is not proven. ddgs returns nothing and raises nothing both on a non-200 response (base.py L68-70, L120-121) and on a 200 without hits (engines/wikipedia.py L47-48), so the empty run's own log cannot tell the two apart.
  • Per-engine probes 6-10 minutes later (22:31-22:35Z; outputs not retained) saw brave 429, duckduckgo 202, google and mojeek 403, and yahoo 200 with results.
• The DNS error is a red herring. The visible wt.wikipedia.org DNS error comes from ddgs's wikipedia engine, which auto mode tries first, not as a fallback. That engine turns gpt-researcher's hard-coded wt-wt region into a subdomain. ddgs surfaces the last exception it recorded; throttled engines return empty without raising.
• sources_count counts the URLs a run tried to read, not the pages it read. skills/researcher.py adds a URL when it is picked for scraping (L813-834), after a scrape whatever its outcome (L1088-1098), and for each prefetched result (L911-913, L922-924). A count above 0 is necessary, not sufficient.

Open condition: this PR fixes detection only — a throttled run now exits 3, and "retry later" is the stopgap — and the next step is the SearXNG measurement decided at docs/decisions/2026-10-01-new-wsl-definitive-defaults.md:80, proposed for the web-research layer sweep in the co-op's report (ledger report-ns2604-coop-20261005T225740Z); no bounded retry or backoff was added because the retriever swallows the search failure (duckduckgo.py L50-52), so a retry would repeat the whole LLM run against the same probably throttled engines with no published window to wait for, and it would hide the failed attempt that docs/acceptance-evidence-policy.md L60-62 asks us to preserve.

SOTA sources

• assafelovic/gpt-researcher@0957c301 (tag v3.7.0, identical to main by GitHub compare):
  • gpt_researcher/retrievers/duckduckgo/duckduckgo.py L30-35 and L49-52: region wt-wt; no backend or region setting reachable from config; a failed search is printed and returned empty;
  • cli.py L209-245 and L332-336: the report frontmatter's sources_count, which is len(researcher.visited_urls);
  • cli.py L220-222: _yaml_quote escapes only backslashes and double quotes. L245 joins the frontmatter at \n;
  • gpt_researcher/skills/researcher.py L813-834, L911-913, L922-924 and L1088-1098: what visited_urls holds. L851+: every configured retriever runs for every sub-query.
• deedy5/ddgs 9.16.0 (the latest on PyPI, 2026-08-26):
  • ddgs/ddgs.py L88-96, L131-132 and L198-223: the auto engine order and last-exception surfacing;
  • ddgs/engines/wikipedia.py L35-39: region to language subdomain. L47-48: a 200 without hits returns empty;
  • ddgs/base.py L65-70 and L116-121: a status other than 200 (429, 202, 403) returns empty without raising;
  • wikipedia engine builds invalid URL for region="wt-wt" (wt.wikipedia.org does not exist) deedy5/ddgs#417, closed as not planned (wt-wt is not a ddgs region); PR Skills trial review 2026-10-25 (window end): per-host checklist #474, open.
• The installed timer's own help, timeout --help (uutils coreutils 0.10.0 on this host), section "Exit status". GNU coreutils' gtimeout documents the same codes (coreutils/coreutils@e6c09ec0 src/timeout.c L322-325):
  • 124: the command timed out;
  • 125: timeout itself fails;
  • 126: the command cannot be invoked;
  • 127: the command is not found.
• Alternatives weighed (details in the decision context):
  • configuring the region or backend: impossible, it is hard-coded;
  • pinning a library version: none fixes it, since 9.16.0 is the latest and the probable cause is server-side;
  • adding scholarly retrievers: off-topic or rate-limited in probes, and they would inflate sources_count while web search is down;
  • SearXNG through the upstream searx retriever: the durable keyless route, owed as a separately measured service, per docs/decisions/2026-10-01-new-wsl-definitive-defaults.md L80.

Evidence-class table

Claim	Evidence class	Command / receipt
The guard rejects 0 sources and uncountable reports, refuses line-break queries, cannot be forged through U+2028, and passes the CLI's own status through	local_integration	python3 -m unittest -v tests.test_research_entry at 7d01a9bc: 11 OK, exit 0. Discriminating controls, method below: against ecfa1127's wrapper, 14 checks fail; against 93f3cbeb's, exactly the 4 new ones fail (exit 1 both)
The head's guard passes every real report on this host except the empty one	local_integration	Guard code extracted from 7d01a9bc and run with the installed venv over all 21 completed run directories under ~/.local/state/new-wsl-native-stack/research/gptr: 20 pass, 1 rejected (20261005T222529Z-1387309, sources_count: 0)
The empty run's exact query passes the head wrapper	native_proven	Run 20261005T234148Z-2924799: bash tools/research/gpt_researcher.sh '<the empty run's query>' at 7d01a9bc (wrapper sha256 5bd4c76c44b03a84dfc676f1cade12eb7f030732c725e39c54c5b4c501dcbb09), 23:41:48Z-23:51:28Z, exit 0. sources_count: 9; report outputs/WSL_Settings_Guide.md sha256 9b95a02c4d9dade77e0d711a42df61fe747017d50f37cb619576aae7edc18b78; gateway session gptr-20261005T234148Z-2924799. Stdout tail below
A real query passed the first guard	native_proven	Run 20261005T224417Z-1716824: "NTP leap smear guidance 2026" through the working-tree wrapper before 93f3cbeb, which had the same guard result for well-formed frontmatter, 22:44:17Z-22:52:03Z, exit 0. sources_count: 7; report sha256 dbbc2f3e5ca3482ffba29365579fa9a9c6495fa702d2e01f69e3d6d469db50d7. Stdout tail below
Probable cause: throttling of this host	source_review (inference)	ddgs's non-raising paths above, and the run directories' start times. The 22:31-22:35Z probes are an unretained observation from after the failure, not evidence of the state at the failure

The receipts are the run directories on this host (report and config.json) and these stdout tails (home shortened to ~):

# 20261005T234148Z-2924799 (head 7d01a9bc), exit 0
Task title: WSL Settings Guide  ->  filename stem: WSL_Settings_Guide
Report written to 'outputs/WSL_Settings_Guide.md'
run directory: ~/.local/state/new-wsl-native-stack/research/gptr/20261005T234148Z-2924799
sources_count: 9

# 20261005T224417Z-1716824 (working tree before 93f3cbeb), exit 0
Task title: NTP Leap Smear 2026  ->  filename stem: NTP_Leap_Smear_2026
Report written to 'outputs/NTP_Leap_Smear_2026.md'
run directory: ~/.local/state/new-wsl-native-stack/research/gptr/20261005T224417Z-1716824
sources_count: 7

The re-run still logged two sub-queries with "No content found", a DuckDuckGo connection reset and the wt.wikipedia.org DNS error. Throttling stays; the guard makes a total failure visible.

Discriminating control (docs/acceptance-evidence-policy.md L42-46). The test module fixes SCRIPT at ROOT/tools/research/gpt_researcher.sh, with ROOT = Path(__file__).resolve().parents[1] (L21-22). Each control therefore builds a temporary mirror directory:

• git show <rev>:tools/research/gpt_researcher.sh written into <mirror>/tools/research/gpt_researcher.sh;
• the current tools/research/gpt-researcher.config.json and tests/test_research_entry.py copied in, plus an empty tests/__init__.py;
• the two evidence JSON files that the configuration test hashes (wave2-records.json, integration-resolutions.json), at their repository paths.

Then it runs cd <mirror> && python3 -m unittest tests.test_research_entry:

control ecfa1127 (wrapper sha256 af4f3a07…, no guard)        -> exit 1, FAILED (failures=14): every guard and line-break check
control 93f3cbeb (wrapper sha256 5327e6df…, the first guard) -> exit 1, FAILED (failures=4): the 3 line-break subtests and the U+2028 forgery

Local commands run

# round 2, at 7d01a9bc
$ bash -n tools/research/gpt_researcher.sh                         -> 0
$ python3 -m unittest -v tests.test_research_entry                 -> 0 (11 tests)
$ (the two discriminating controls above)                          -> 1 and 1, as designed
$ bash tools/research/gpt_researcher.sh --preflight-only           -> 0 (preflight passed)
$ bash tools/research/gpt_researcher.sh $'two\nlines'              -> 2 (The query must be one line: ...)
$ python3 scripts/validate.py                                      -> 0 (status passed)
$ (six neighbouring test modules)                                  -> 0 (666 tests, 22 skipped)
$ bash tools/research/gpt_researcher.sh '<the empty run's query>'  -> 0 (sources_count: 9)
# round 1, at 93f3cbeb
$ bash -n, unittest (9 tests), --preflight-only, validate.py       -> 0 each
$ (six neighbouring test modules)                                  -> 0 (664 tests, 22 skipped)

Decision record

None new. This is a defect fix inside the existing research-runtime decision. SearXNG is the recorded challenger and stays a separate measured adoption.

Host evidence

Not applicable.

Checklist

• No workflow changes.
• No secrets: the runtime keeps env -i and the keyless loopback placeholder.
• No new paid surface.
• Peer-owned files untouched.

After landing, and only at a quiet point agreed with the command center, the co-op updates both installed copies, which still lack the guard:

• ~/.config/new-wsl-native-stack/gpt-researcher.sh, through the research-harnesses row's own install command (expect sha256 5bd4c76c… from this head);
• the shared main checkout, now at 1796303f.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-06T00:00:18.776682Z	7d01a9b	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[seathatflowsinourveins]

Closed as superseded by #774, which changes both of this PR's files and keeps every line it added (command-center PR board ruling, 2026-10-10). Reopen if anything here is still needed.