feat: HV-1 native campaign driver (Tools/Hv1NativeCampaignRunner) - #87
Merged
Conversation
…sk status
Adds Tools/Hv1NativeCampaignRunner, a headless driver for HV-1 (docs/
NATIVE_RUNTIME_HIVE_VALIDATION_PLAN.md): submits real native-role
(ExecutionKind=NativeAgent) campaign work units to a live Warchief,
pinned via ExcludedWorkerIds so N jobs land on each of two named
workers, polls each to completion, and validates per-job evidence
(ClaimedBy matches the intended target, Attestation.RuntimeName ==
"NativeRoleRuntime", output contains the expected marker, worker-
reported stats present). Writes one evidence report per run.
That per-job evidence (ExecutionAttestation, Metrics) was already
populated server-side in HiveTaskResult on completion but never
exposed over GET /hive/tasks/{id} -- only OutputArtifacts was. Added
Attestation/Metrics to HiveTaskStatusResponse and populated them in
HandleGetTaskAsync so an external polling harness can verify native-
vs-fallback execution without in-process queue access.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughHive task polling now exposes execution attestations and metrics. A new .NET 10 HV-1 native campaign runner submits paired work units, validates worker evidence, and writes reports. Runtime admission accounting was corrected for resident model reservations, with a regression test and validation record added. ChangesHV-1 native validation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Hv1NativeCampaignRunner
participant HiveTaskQueue
participant RuntimeOrchestrator
participant EvidenceReport
Hv1NativeCampaignRunner->>HiveTaskQueue: Submit paired HV-1 work units
HiveTaskQueue-->>Hv1NativeCampaignRunner: Return Attestation and Metrics
Hv1NativeCampaignRunner->>RuntimeOrchestrator: Admit native worker workload
RuntimeOrchestrator-->>Hv1NativeCampaignRunner: Return admission result
Hv1NativeCampaignRunner->>EvidenceReport: Write validated campaign report
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Daemon-based workers (OrchestratorIDE.Daemon) DO populate WorkerCapabilities.NativeModelHashes via WorkerCapabilityDetector, unlike swarmcli --worker which never calls that detector. Made model-hash capability gating opt-in via --gate-model-hash so the driver still works against either deployment shape, and used it in the real HV-1 run to get a live per-job model-hash capability match in the evidence instead of just an echoed value. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…y gap found HardcoreLaptopMSI (8GB): clean 5/5 native jobs, live model-hash capability match, zero fallback. HardcorePC (6GB): blocked at 1/5, reproducible across a fresh process restart -- the first job's VRAM reservation for the Worker role never releases, denying every subsequent job on that card. Root cause is inside NativeRoleRuntime/ AdapterManager's conversation lifecycle, out of scope for this campaign; filed as an open follow-up rather than worked around. Also documents two gaps found en route: swarmcli --worker cannot execute NativeAgent work units at all (switched to OrchestratorIDE. Daemon instead), and that switch triggered a DPAPI/AES-GCM identity collision on both remotes, recovered by re-pairing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Owner
Author
|
@coderabbitai full review |
Contributor
✅ Action performedFull review finished. Your plan includes PR reviews subject to rate limits. More reviews will be available in 58 minutes. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
Tools/Hv1NativeCampaignRunner/Program.cs (1)
33-34: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMove required-arg validation inside the try/catch (or handle separately) for a consistent exit-code contract.
--model-hashis validated beforereportexists and before thetryblock starts. A missing arg throws an unhandled exception instead of following this program's own FAILED-report + exit-code(0/1/2) convention used everywhere else.♻️ Suggested fix
- var modelHash = GetArg(args, "--model-hash") - ?? throw new InvalidOperationException("--model-hash is required (the pinned fleet GGUF's SHA-256)."); + var modelHash = GetArg(args, "--model-hash"); + if (string.IsNullOrWhiteSpace(modelHash)) + { + Console.Error.WriteLine("--model-hash is required (the pinned fleet GGUF's SHA-256)."); + return 1; + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Tools/Hv1NativeCampaignRunner/Program.cs` around lines 33 - 34, Move the required --model-hash validation into the existing try/catch in the main execution flow, or handle its exception through the same reporting path. Ensure a missing argument produces the program’s FAILED report and established exit code (0/1/2) contract instead of an unhandled exception, while preserving normal validation for supplied values.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@Tools/Hv1NativeCampaignRunner/Program.cs`:
- Around line 33-34: Move the required --model-hash validation into the existing
try/catch in the main execution flow, or handle its exception through the same
reporting path. Ensure a missing argument produces the program’s FAILED report
and established exit code (0/1/2) contract instead of an unhandled exception,
while preserving normal validation for supplied values.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 0d356839-bd34-4269-af63-0fc270d819f1
📒 Files selected for processing (4)
OrchestratorIDE/Services/Hive/HiveTaskBundle.csOrchestratorIDE/Services/Hive/HiveTaskQueue.csTools/Hv1NativeCampaignRunner/Hv1NativeCampaignRunner.csprojTools/Hv1NativeCampaignRunner/Program.cs
…(6GB-box HV-1 blocker) EnsureAdmitted built its budget from a live whole-GPU nvidia-smi read (NativeVramProbe) whose ReservedBytes already includes a role's resident model, then charged a full fresh-load EstimateRequiredBytes for that same model on top -- counting one resident model twice (once as used, once as needed). On a card tight enough that two phantom copies don't fit, every sequential native job after the first was denied by a correctly-functioning fail-closed admission check (HardcorePC RTX 3050 6GB, HV-1 2026-07-21: 1/5). The 8GB laptop passed 5/5 only because it had headroom to absorb the double-charge -- the bug was latent there too. Re-admitting a role either reuses its resident executor (loads nothing) or tears it down before building a replacement (old footprint freed first), so its resident bytes must be credited back out of the live baseline -- the exact analogue of the same-role exclusion already applied to other-role ledger entries. Clamped so a probe that under-counts can't drive the budget negative. Cross-role accounting is unchanged; only same-role re-admission is affected. Regression test (THEORC_TEST_GGUF-gated, same precedent as the existing cross-role reservation test): a stateful provider stands in for the live probe (idle, then resident), and a second same-role admission on a budget that fits one model must succeed. Verified red before / green after; full RuntimeOrchestrator/Hive/OrcScheduler/AdapterManager suite 155/155 green with THEORC_TEST_GGUF set. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…eal fleet Corrects the earlier "reservation never releases" framing: the actual bug was a live-probe double-count in RuntimeOrchestrator.EnsureAdmitted (fixed in this same PR), not a conversation-lifecycle leak. Confirmed with a context-size experiment before touching code, fixed, then re-ran the exact 5-jobs/worker/full-context config that produced 1/5 before: HardcorePC 5/5, HardcoreLaptopMSI 5/5, zero fallback. HV-1 is closed for both fleet machines, including the low-VRAM class HV-0 deliberately included to find exactly this kind of gap. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Owner
Author
|
@coderabbitai full review |
Contributor
✅ Action performedFull review finished. |
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
Tools/Hv1NativeCampaignRunner/Program.cs (1)
72-113: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAvoid re-deriving
target/markerfrom the WorkUnitId string; capture them at creation time.
targetandmarkerare already known inside the loop that builds eachWorkUnit(Line 73-95). RebuildingtargetByUnitId/markerByUnitIdafterward viaWorkUnitId.StartsWith(...)string matching is an unnecessary second source of truth that only stays correct because of the-separator convention.♻️ Proposed simplification
- var workUnits = new List<WorkUnit>(); + var workUnits = new List<WorkUnit>(); + var targetByUnitId = new Dictionary<string, string>(); + var markerByUnitId = new Dictionary<string, string>(); foreach (var (target, other) in new[] { (workerA, workerB), (workerB, workerA) }) { for (var i = 1; i <= jobsPerWorker; i++) { var workUnitId = $"hv1-{target}-{i:00}"; var marker = $"HV1-PROOF {workUnitId}"; + targetByUnitId[workUnitId] = target; + markerByUnitId[workUnitId] = marker; workUnits.Add(new WorkUnit { ... }); } } - - var targetByUnitId = workUnits.ToDictionary( - u => u.WorkUnitId, - u => u.WorkUnitId.StartsWith($"hv1-{workerA}-", StringComparison.Ordinal) ? workerA : workerB); - var markerByUnitId = workUnits.ToDictionary( - u => u.WorkUnitId, u => $"HV1-PROOF {u.WorkUnitId}");🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Tools/Hv1NativeCampaignRunner/Program.cs` around lines 72 - 113, Capture each work unit’s target and marker in dedicated dictionaries while constructing the WorkUnit inside the foreach loop, using WorkUnitId as the key. Then remove the later targetByUnitId and markerByUnitId re-derivation based on WorkUnitId.StartsWith, so downstream reporting uses the values recorded at creation time.docs/NATIVE_RUNTIME_HIVE_VALIDATION_PLAN.md (1)
221-223: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winTypo: "HARDCORELAPTOPM" should be "HARDCORELAPTOPMSI".
Every other reference to this worker in the doc uses the full name (e.g. Line 167 "HardcoreLaptopMSI", Line 206 "HardcoreLaptopMSI's").
📝 Proposed fix
-**Decisive re-run, same config that produced the 1/5 failure (full `NativeContextSize=8192`, 5 -jobs/worker, live `--gate-model-hash`): HARDCOREPC 5/5, HARDCORELAPTOPM 5/5, zero fallback.** +**Decisive re-run, same config that produced the 1/5 failure (full `NativeContextSize=8192`, 5 +jobs/worker, live `--gate-model-hash`): HARDCOREPC 5/5, HARDCORELAPTOPMSI 5/5, zero fallback.**🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/NATIVE_RUNTIME_HIVE_VALIDATION_PLAN.md` around lines 221 - 223, Correct the worker name in the decisive re-run summary from “HARDCORELAPTOPM” to “HARDCORELAPTOPMSI”, matching the full worker name used elsewhere in the document.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@docs/NATIVE_RUNTIME_HIVE_VALIDATION_PLAN.md`:
- Around line 221-223: Correct the worker name in the decisive re-run summary
from “HARDCORELAPTOPM” to “HARDCORELAPTOPMSI”, matching the full worker name
used elsewhere in the document.
In `@Tools/Hv1NativeCampaignRunner/Program.cs`:
- Around line 72-113: Capture each work unit’s target and marker in dedicated
dictionaries while constructing the WorkUnit inside the foreach loop, using
WorkUnitId as the key. Then remove the later targetByUnitId and markerByUnitId
re-derivation based on WorkUnitId.StartsWith, so downstream reporting uses the
values recorded at creation time.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 8ce66580-a50f-413c-8237-9fd5b25a7be8
📒 Files selected for processing (7)
OrchestratorIDE.UnitTests/RuntimeOrchestratorTests.csOrchestratorIDE/Core/Runtime/RuntimeOrchestrator.csOrchestratorIDE/Services/Hive/HiveTaskBundle.csOrchestratorIDE/Services/Hive/HiveTaskQueue.csTools/Hv1NativeCampaignRunner/Hv1NativeCampaignRunner.csprojTools/Hv1NativeCampaignRunner/Program.csdocs/NATIVE_RUNTIME_HIVE_VALIDATION_PLAN.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ExcludedWorkerIdsso N jobs land on each of two named workers, polls to completion, and validates per-job evidence (claimant matches target,Attestation.RuntimeName == "NativeRoleRuntime", output contains the expected marker, worker-reported stats present).ExecutionAttestation/MetricsonHiveTaskStatusResponse(GET /hive/tasks/{id}) — this data was already populated server-side inHiveTaskResulton completion but never exposed to an external polling harness, onlyOutputArtifactswas.Tools/Cf6AcceptanceRunnerpattern (link shared Hive contract source files rather than a project reference, JSON evidence report per run).Why zero-fallback is structural here, not just asserted
Every work unit this driver submits uses
ExecutionKind = NativeAgent. PerHiveWorkerAgent.ExecuteTaskAsync, any non-LegacyAgentexecution kind is unconditionally fail-closed — there is no Ollama fallback path even reachable. TheAttestation.RuntimeName == "NativeRoleRuntime"check in the evidence is confirmation that guarantee held in practice, not the mechanism enforcing it.Test plan
dotnet buildon the new tool, the main Avalonia project, SwarmCli, and Cf6AcceptanceRunner — all green, no regressions from theHiveTaskStatusResponsefield addition.dotnet test --filter FullyQualifiedName~Hive— 112/112 passing.🤖 Generated with Claude Code
Summary by CodeRabbit