chore: point agent-eval-harness submodule to opendatahub-io

[ralphbean]

Summary

• Update .gitmodules URL from ralphbean/agent-eval-harness to opendatahub-io/agent-eval-harness
• Advance submodule pointer to latest main (4b540c6, release 1.22.0)

Test plan

• git submodule update --init succeeds with the new URL
• Functional tests pass (./eval/run-functional.sh triage)

🤖 Generated with Claude Code

[qodo-code-review]

PR Summary by Qodo

Point agent-eval-harness submodule to opendatahub-io and bump to v1.22.0

⚙️ Configuration changes 🕐 Less than 5 minutes

AI Description

• Switch eval harness submodule URL from a fork to opendatahub-io upstream.
• Advance the submodule commit pointer to the latest main (v1.22.0).

Diagram

graph TD
  A["Repo checkout"] --> B[".gitmodules"] --> C["eval/.agent-eval-harness (submodule)"] --> D["GitHub: opendatahub-io/agent-eval-harness"]

Loading

High-Level Assessment

This is the correct minimal change: repoint the submodule remote to the upstream org and pin to the desired upstream commit. Alternatives like switching away from submodules (subtree/vendor) are larger-scope decisions not warranted for this PR.

Files changed (2) +2 / -2

Other (2) +2 / -2

.gitmodulesUpdate eval harness submodule remote to upstream +1/-1

Update eval harness submodule remote to upstream

• Replaces the submodule URL for eval/.agent-eval-harness from the ralphbean fork to the opendatahub-io upstream repository.

.gitmodules

.agent-eval-harnessBump submodule pointer to upstream main (v1.22.0) +1/-1

Bump submodule pointer to upstream main (v1.22.0)

• Advances the pinned submodule commit from 296c33f to 4b540c6, aligning the checkout with the latest upstream main/release noted in the PR description.

eval/.agent-eval-harness

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 10:59 AM UTC · Completed 11:04 AM UTC
Commit: ebcb3a5 · View workflow run →

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Context used

✅ Compliance rules (platform): 55 rules

✅ Skills: 4 invoked

  code-review

  code-implementation

  pr-review

  docs-review

1. Submodule URL sync missing ✓ Resolved 🐞 Bug ☼ Reliability

Description

After updating the submodule URL in .gitmodules, previously-initialized clones may still use the old
cached submodule URL from local git config, so git submodule update --init ... alone can keep
fetching from the old remote. This can cause harness checkout failures for existing worktrees even
though fresh clones work.

Code

.gitmodules[3]

+	url = https://github.com/opendatahub-io/agent-eval-harness.git

Relevance

⭐⭐ Medium

No prior repo evidence requiring git submodule sync after .gitmodules URL changes; guidance in PR
#31 omits it.

PR-#31

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The PR changes the submodule remote URL, while the repo’s primary user-facing guidance for fixing
missing harness files only suggests git submodule update --init ... and does not mention syncing
URLs after .gitmodules changes.

.gitmodules[1-3]
eval/run-functional.sh[64-73]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The submodule URL changed, but existing clones that already initialized the submodule may still have the old URL in their local `.git/config`. The current guidance (in eval/run-functional.sh) only tells users to run `git submodule update --init ...`, which may not update the cached URL.

## Issue Context
This PR updates `.gitmodules` to point `eval/.agent-eval-harness` at a new GitHub org. Git does not always automatically propagate `.gitmodules` URL changes into an already-initialized submodule configuration.

## Fix Focus Areas
- eval/run-functional.sh[68-72]
- .gitmodules[1-3]

## Suggested change
Update the error message/instructions in `eval/run-functional.sh` to recommend syncing before updating, e.g.:

```bash
git submodule sync -- eval/.agent-eval-harness
git submodule update --init --recursive eval/.agent-eval-harness
```

(Optionally also mention `git submodule sync --recursive` for users who prefer that.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[fullsend-ai-review]

Looks good to me

Previous run

Review of #146 — chore: point agent-eval-harness submodule to opendatahub-io

Verdict: ✅ Approve

Summary

This PR updates the agent-eval-harness git submodule in two ways:

1. URL migration: .gitmodules URL changed from ralphbean/agent-eval-harness (personal fork) to opendatahub-io/agent-eval-harness (organization repo)
2. Pointer advance: Submodule commit updated from 296c33f to 4b540c6 (release 1.22.0 per PR description)

Review dimensions

Dimension	Result
Correctness	✅ No issues. No stale references to old URL exist outside .gitmodules. All consumers reference the submodule by local path (eval/.agent-eval-harness), not remote URL. CI workflow (functional-tests.yml) uses submodules: true which reads the updated .gitmodules.
Security	✅ No issues. Direction of change is risk-reducing — personal fork to org repo. Commit is pinned to a specific SHA, preventing upstream drift. No secrets, injection patterns, or permission changes.
Intent & coherence	✅ Mechanical change. Submodule URL update and pointer advance — standard housekeeping.
Style & conventions	✅ No issues. Change follows existing .gitmodules format. PR title uses Conventional Commits (chore:) per AGENTS.md.
Docs currency	✅ No stale references. No documentation files reference the old URL or fork.
Cross-repo contracts	⏭ Not applicable. No exported interfaces, schemas, or APIs modified.

Notes

• The test plan appropriately covers git submodule update --init and functional test execution, which will validate that the new remote contains the pinned commit and that expected scripts exist at the new pointer.

[fullsend-ai-review]

🤖 Review · ❌ Terminated · Started 11:48 AM UTC · Ended 11:55 AM UTC
Commit: bd28f41 · View workflow run →

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 11:58 AM UTC · Completed 12:06 PM UTC
Commit: 9ee7452 · View workflow run →

[fullsend-ai-retro]

PR #146 updated the agent-eval-harness git submodule URL from a personal fork (ralphbean) to an organization repo (opendatahub-io) and advanced the submodule pointer to v1.22.0. The fullsend review agent approved the first commit but missed a well-known git operational pitfall that qodo-code-review caught: existing clones need git submodule sync before git submodule update --init when the .gitmodules URL changes. This caused one rework cycle (the author pushed a fix commit). Additionally, the review agent used 85 tool calls for what was a 3-file, 3-line mechanical URL change — well above the trivial scope constraint the pr-review skill defines for this class of change. Two proposals filed.

Proposals filed

• Correctness sub-agent should detect git submodule URL migration pitfalls (in fullsend-ai/agents)
• Review orchestrator triage did not classify submodule URL update as trivial/mechanical change (in fullsend-ai/agents)