Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions test/e2e/docs/NATIVE_RUNTIME_QUALIFICATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Native Runtime Activation Qualification

`registry/activation-qualification.ts` is a dormant release gate for a native
container-runtime candidate. It does not register a runtime, add a live target,
alter the production runtime registry, or advertise support.

The compiler has one provider-neutral contract. A candidate supplies an open
provider ID and fixture bindings; the compiler does not branch on Podman, MXC,
or any other engine name. The inert Podman candidate in
`support/native-runtime-qualification-fixtures.ts` proves the intended scope,
while the fake MXC-style test proves that the same contract accepts another
provider without a central switch.

## Required protected matrix

Compilation requires 24 exact cases:

- OpenClaw, Hermes, and DCode;
- Linux `amd64` and `arm64`;
- rootless CPU with host-local Ollama;
- rootless NVIDIA GPU with CDI-backed Ollama, NIM, and vLLM;
- the release installer with Docker unavailable; and
- protected E2E for every case.

Every case must declare installation, Docker-unavailable proof, onboarding, an
agent turn, stop/start, snapshot/restore, rebuild, restart/reconciliation, and
exact cleanup. Removing one case or obligation is a compile error, not a skip.

## Exact evidence

Every compiled case declares the evidence categories that the protected
collector must eventually produce. Those categories preserve the complete
activation target:

- the exact protected workflow revision, run, job, attempt, head SHA, and base
SHA;
- hashed installer script and invocation artifacts with a successful result;
- an exact provider/profile/architecture/acceleration identity and persisted
host-local engine authority;
- immutable agent and probe image references, plus an immutable inference image
reference for provider-managed NIM and vLLM;
- the exact provider-native host, port, network, gateway provider URL, and the
canonical `https://inference.local/v1` application route;
- the serialized host-local inference authority digest, including the exact
provider-owned runtime/container identity and specification digest for NIM
and vLLM;
- the exercised model ID and a hashed inference-result artifact;
- hashed artifacts for every lifecycle obligation, all bound to that same
durable authority;
- a reconciliation receipt proving recovery retained the same authority;
- an NVIDIA CDI `nvidia.com/gpu=all` receipt for GPU cases; and
- exact cleanup proving external Ollama was retained or provider-owned NIM and
vLLM were removed, with no provider-owned runtime IDs remaining.

These are compile-time requirements, not a receipt schema or generated
evidence. This slice intentionally does not export a reporter, artifact
verifier, or process-local evidence brand before a protected workflow consumes
that API. The protected-collector slice must add those pieces together: obtain
authenticated GitHub run/job state independently of worker receipts, verify
every artifact below its downloaded job root, and fail closed on incomplete,
inexact, linked, escaping, conflicting, changing, or oversized evidence. All
cases must bind to one exact protected workflow/head/base source before the
runtime can activate.

## Activation boundary

Keep this contract inert until an implementation PR supplies executable
provider adapters and the entire protected matrix passes on one exact head/base
pair. Public support, installer selection, production registry wiring, and
workflow dispatch are separate activation work and must not infer support from
the existence of this contract.
8 changes: 8 additions & 0 deletions test/e2e/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Direct E2E implementations now live in Vitest. The former
| Expected-state probes | `test/e2e/registry/expected-states.ts` |
| Product-facing setup/onboarding state | `test/e2e/manifests/*.yaml` |
| Migration status and retirement decisions | GitHub issues and pull requests |
| Native runtime activation qualification | `registry/activation-qualification.ts`, `docs/NATIVE_RUNTIME_QUALIFICATION.md` |

## Target Model

Expand Down Expand Up @@ -107,6 +108,13 @@ mechanics in obligation bindings, and support facts in capabilities. A binding
must cover every obligation explicitly; a missing adapter or capability is a
compile error rather than a skip.

The dormant native-runtime activation contract builds on this foundation. It
requires the complete all-agent, multiarch, CPU/GPU, host-local inference,
installer, lifecycle, recovery, cleanup, and exact-evidence matrix before a
provider can be activated. See
[`NATIVE_RUNTIME_QUALIFICATION.md`](./NATIVE_RUNTIME_QUALIFICATION.md). The
contract is not production runtime registration or a support claim.

## How To Run

```bash
Expand Down
292 changes: 292 additions & 0 deletions test/e2e/registry/activation-qualification.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,292 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import {
defineExecutionProfile,
type ExecutionAcceleration,
type ExecutionArchitecture,
type ExecutionCapability,
type ExecutionProfile,
type ExecutionProviderId,
} from "./execution-profile.ts";
import { compareCodeUnits, type RuntimeAgent } from "./scenario.ts";

export const NATIVE_RUNTIME_QUALIFICATION_AGENTS = ["openclaw", "hermes", "dcode"] as const;
export const NATIVE_RUNTIME_QUALIFICATION_ARCHITECTURES = ["amd64", "arm64"] as const;
export const NATIVE_RUNTIME_QUALIFICATION_ACCELERATIONS = ["cpu", "nvidia-gpu"] as const;
export const NATIVE_RUNTIME_QUALIFICATION_INFERENCE = {
cpu: ["ollama"],
"nvidia-gpu": ["ollama", "nim", "vllm"],
} as const satisfies Readonly<Record<ExecutionAcceleration, readonly LocalInferenceProvider[]>>;

export type LocalInferenceProvider = "ollama" | "nim" | "vllm";
export type QualificationObligation =
| "installer.install"
| "runtime.docker-unavailable"
| "agent.onboard"
| "agent.turn"
| "sandbox.stop-start"
| "sandbox.snapshot-restore"
| "sandbox.rebuild"
| "runtime.restart-reconcile"
| "cleanup.exact";
export type QualificationEvidenceKind =
| "protected-run"
| "source-identity"
| "installer-result"
| "docker-unavailable-guard"
| "managed-images"
| "agent-turn"
| "local-inference"
| "lifecycle"
| "recovery"
| "cleanup"
| "nvidia-cdi";

export const NATIVE_RUNTIME_QUALIFICATION_OBLIGATIONS = [
"installer.install",
"runtime.docker-unavailable",
"agent.onboard",
"agent.turn",
"sandbox.stop-start",
"sandbox.snapshot-restore",
"sandbox.rebuild",
"runtime.restart-reconcile",
"cleanup.exact",
] as const satisfies readonly QualificationObligation[];

const BASE_EVIDENCE_KINDS = [
"protected-run",
"source-identity",
"installer-result",
"docker-unavailable-guard",
"managed-images",
"agent-turn",
"local-inference",
"lifecycle",
"recovery",
"cleanup",
] as const satisfies readonly QualificationEvidenceKind[];

const REQUIRED_CAPABILITIES = [
"agent.configure",
"agent.turn",
"evidence.collect",
"sandbox.lifecycle",
"state.observe",
"transport.socket-free",
] as const satisfies readonly ExecutionCapability[];

export interface NativeRuntimeQualificationCase {
id: string;
agent: RuntimeAgent;
profile: ExecutionProfile;
inference: LocalInferenceProvider;
gate: "protected-e2e";
install: "release-installer";
dockerAvailability: "unavailable";
obligations: readonly QualificationObligation[];
evidenceKinds: readonly QualificationEvidenceKind[];
}

export interface NativeRuntimeQualificationDefinition {
id: string;
repository: string;
protectedWorkflow: string;
provider: ExecutionProviderId;
engineName: string;
cases: readonly NativeRuntimeQualificationCase[];
}

export interface CompiledNativeRuntimeQualification {
id: string;
repository: string;
protectedWorkflow: string;
provider: ExecutionProviderId;
engineName: string;
cases: readonly Readonly<NativeRuntimeQualificationCase>[];
}

const REPOSITORY_PATTERN = /^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/u;

function assertSingleLine(value: string, label: string): string {
const normalized = value.trim();
if (!normalized || /[\r\n]/u.test(normalized)) {
throw new Error(`${label} must be a non-empty single-line string`);
}
return normalized;
}

function assertExactSet<T extends string>(
actual: readonly T[],
expected: readonly T[],
label: string,
): void {
const actualSet = new Set(actual);
if (actualSet.size !== actual.length) {
throw new Error(`${label} contains duplicate values`);
}
const missing = expected.filter((value) => !actualSet.has(value));
const unknown = actual.filter((value) => !expected.includes(value));
if (missing.length > 0 || unknown.length > 0) {
throw new Error(
`${label} is incomplete (missing: ${missing.join(", ") || "none"}; unknown: ${unknown.join(", ") || "none"})`,
);
}
}

export function requiredQualificationEvidenceKinds(
acceleration: ExecutionAcceleration,
): readonly QualificationEvidenceKind[] {
return acceleration === "nvidia-gpu"
? Object.freeze([...BASE_EVIDENCE_KINDS, "nvidia-cdi"])
: BASE_EVIDENCE_KINDS;
}

export function qualificationCaseId(input: {
provider: ExecutionProviderId;
agent: RuntimeAgent;
architecture: ExecutionArchitecture;
acceleration: ExecutionAcceleration;
inference: LocalInferenceProvider;
}): string {
const acceleration = input.acceleration === "nvidia-gpu" ? "gpu" : input.acceleration;
return [
input.provider,
input.agent,
"linux",
input.architecture,
acceleration,
input.inference,
].join("-");
}
Comment thread
ericksoa marked this conversation as resolved.

function coverageKey(input: {
agent: RuntimeAgent;
architecture: ExecutionArchitecture;
acceleration: ExecutionAcceleration;
inference: LocalInferenceProvider;
}): string {
return [input.agent, input.architecture, input.acceleration, input.inference].join("|");
}

function requiredCoverageKeys(): string[] {
return NATIVE_RUNTIME_QUALIFICATION_AGENTS.flatMap((agent) =>
NATIVE_RUNTIME_QUALIFICATION_ARCHITECTURES.flatMap((architecture) =>
NATIVE_RUNTIME_QUALIFICATION_ACCELERATIONS.flatMap((acceleration) =>
NATIVE_RUNTIME_QUALIFICATION_INFERENCE[acceleration].map((inference) =>
coverageKey({ agent, architecture, acceleration, inference }),
),
),
),
).sort(compareCodeUnits);
}

function compileCase(
definition: NativeRuntimeQualificationDefinition,
input: NativeRuntimeQualificationCase,
): Readonly<NativeRuntimeQualificationCase> {
const profile = defineExecutionProfile(input.profile);
if (profile.provider !== definition.provider) {
throw new Error(
`Qualification case '${input.id}' profile provider '${profile.provider}' does not match '${definition.provider}'`,
);
}
if (profile.platform !== "linux" || profile.rootMode !== "rootless") {
throw new Error(`Qualification case '${input.id}' must use a rootless Linux profile`);
}
const capabilities = new Set(profile.capabilities);
const missingCapabilities = REQUIRED_CAPABILITIES.filter((value) => !capabilities.has(value));
if (missingCapabilities.length > 0 || capabilities.has("transport.docker-socket")) {
throw new Error(
`Qualification case '${input.id}' must be socket-free and declares invalid capabilities (missing: ${missingCapabilities.join(", ") || "none"})`,
);
}
if (input.gate !== "protected-e2e") {
throw new Error(`Qualification case '${input.id}' must run through protected E2E`);
}
if (input.install !== "release-installer") {
throw new Error(`Qualification case '${input.id}' must exercise the release installer`);
}
if (input.dockerAvailability !== "unavailable") {
throw new Error(`Qualification case '${input.id}' must prove Docker is unavailable`);
}
const allowedInference = NATIVE_RUNTIME_QUALIFICATION_INFERENCE[profile.acceleration];
if (!(allowedInference as readonly string[]).includes(input.inference)) {
throw new Error(
`Qualification case '${input.id}' cannot use ${input.inference} with ${profile.acceleration}`,
);
}
assertExactSet(
input.obligations,
NATIVE_RUNTIME_QUALIFICATION_OBLIGATIONS,
`Qualification case '${input.id}' obligations`,
);
assertExactSet(
input.evidenceKinds,
requiredQualificationEvidenceKinds(profile.acceleration),
`Qualification case '${input.id}' evidence kinds`,
);
const expectedId = qualificationCaseId({
provider: definition.provider,
agent: input.agent,
architecture: profile.architecture,
acceleration: profile.acceleration,
inference: input.inference,
});
if (input.id !== expectedId) {
throw new Error(`Qualification case id '${input.id}' must be '${expectedId}'`);
}
return Object.freeze({
...input,
profile,
obligations: Object.freeze([...input.obligations]),
evidenceKinds: Object.freeze([...input.evidenceKinds]),
});
}

export function compileNativeRuntimeQualification(
input: NativeRuntimeQualificationDefinition,
): CompiledNativeRuntimeQualification {
const id = assertSingleLine(input.id, "Native runtime qualification id");
const repository = assertSingleLine(input.repository, "Native runtime qualification repository");
if (!REPOSITORY_PATTERN.test(repository)) {
throw new Error(`Native runtime qualification repository '${repository}' must be owner/name`);
}
const protectedWorkflow = assertSingleLine(
input.protectedWorkflow,
"Native runtime qualification protected workflow",
);
const engineName = assertSingleLine(input.engineName, "Native runtime engine name");
const cases = input.cases.map((entry) => compileCase(input, entry));
const casesByCoverage = new Map<string, Readonly<NativeRuntimeQualificationCase>>();
for (const entry of cases) {
const key = coverageKey({
agent: entry.agent,
architecture: entry.profile.architecture,
acceleration: entry.profile.acceleration,
inference: entry.inference,
});
if (casesByCoverage.has(key)) {
throw new Error(`Native runtime qualification repeats case coverage '${key}'`);
}
casesByCoverage.set(key, entry);
}
const expectedCoverage = requiredCoverageKeys();
const missing = expectedCoverage.filter((key) => !casesByCoverage.has(key));
const unknown = [...casesByCoverage.keys()].filter((key) => !expectedCoverage.includes(key));
if (missing.length > 0 || unknown.length > 0) {
throw new Error(
`Native runtime qualification coverage is incomplete (missing: ${missing.join(", ") || "none"}; unknown: ${unknown.join(", ") || "none"})`,
);
}
const compiled = Object.freeze({
id,
repository,
protectedWorkflow,
provider: input.provider,
engineName,
cases: Object.freeze([...cases].sort((left, right) => compareCodeUnits(left.id, right.id))),
});
return compiled;
}
Loading
Loading