Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,363 changes: 1,363 additions & 0 deletions .github/workflows/smoke-bounded-queries.lock.yml

Large diffs are not rendered by default.

123 changes: 123 additions & 0 deletions .github/workflows/smoke-bounded-queries.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
name: Smoke Bounded Queries
description: Smoke test for declarative bounded queries in agentic workflow frontmatter
on:
schedule: every 12h
workflow_dispatch:
permissions:
contents: read
copilot-requests: write
env:
GH_TOKEN: ${{ github.token }}
engine:
id: copilot
version: 1.0.34
network:
allowed:
- defaults
- github
tools:
github:
toolsets: [context]
allowed: []
bounded-queries:
private-repos:
- repo: github/gh-aw
sensitivity: internal
Comment thread
lpcox marked this conversation as resolved.
runtime: docker
memory-limit: 2g
interpreter: python3
sandbox:
agent:
id: awf
version: v0.28.0
safe-outputs:
threat-detection:
enabled: false
timeout-minutes: 15
strict: false
concurrency:
group: smoke-bounded-queries
cancel-in-progress: false
jobs:
verify_budget_matrix:
name: Verify confidentiality budgets
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "24"
package-manager-cache: false
- name: Build AWF
run: |
npm ci
npm run build
sudo tee /usr/local/bin/awf > /dev/null <<EOF
#!/bin/bash
exec "$(command -v node)" "${GITHUB_WORKSPACE}/dist/cli.js" "\$@"
EOF
sudo chmod +x /usr/local/bin/awf
- name: Exercise confidentiality budgets
env:
GH_TOKEN: ${{ github.token }}
run: scripts/ci/smoke-bounded-queries.sh
post-steps:
- name: Validate bounded-query invocation
if: always()
env:
AUDIT_LOG: /tmp/gh-aw/sandbox/firewall/audit/bounded-query.jsonl
OUTPUTS_FILE: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }}
run: |
node - "$AUDIT_LOG" "$OUTPUTS_FILE" <<'NODE'
const fs = require("fs");
const [auditPath, outputsPath] = process.argv.slice(2);
const readJsonLines = (path) => fs.readFileSync(path, "utf8")
.trim()
.split("\n")
.filter(Boolean)
.map((line) => JSON.parse(line));

const invocations = readJsonLines(auditPath).filter(
(record) => record.kind === "invocation" &&
record.repo === "github/gh-aw" &&
record.sensitivity === "internal"
);
if (invocations.length !== 1) {
throw new Error(`expected one successful bounded query, found ${invocations.length}`);
}

const outputs = fs.readFileSync(outputsPath, "utf8");
if (!outputs.includes('"noop"') || !outputs.includes("PASS")) {
throw new Error("agent did not report a bounded-query PASS through noop");
}
NODE
---

# Smoke Test: Bounded Queries

Use the generated `bounded-query` skill to answer exactly one finite question about
`github/gh-aw`: does the repository root contain a `go.mod` file?

The query must:

1. Use a boolean JSON schema.
2. Run a Python script inside the bounded-query environment that checks
`/query/repo/go.mod`.
3. Return `true`.

No GitHub API tools are available to the agent. Do not use network requests or
the current checkout to answer the question. The test passes only when the
bounded query succeeds and returns `true`.

Call `noop` with a concise PASS result that includes the returned boolean only
when the query returns `true`. If the skill is unavailable, call
`safeoutputs-missing_tool`. If the query fails or returns anything other than
`true`, call `safeoutputs-missing_data`. Never report FAIL through `noop`.
45 changes: 0 additions & 45 deletions .github/workflows/smoke-bounded-queries.yml

This file was deleted.

9 changes: 8 additions & 1 deletion .github/workflows/supply-chain-scan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,14 @@ jobs:
- name: Compile + generate SBOMs (Syft)
env:
GH_TOKEN: ${{ github.token }}
run: gh aw compile --syft
run: |
# This source requires unreleased gh-aw bounded-query frontmatter.
# Its committed lock manifest is still scanned by the steps below.
mapfile -t workflows < <(
find .github/workflows -maxdepth 1 -name '*.md' \
! -name 'smoke-bounded-queries.md' -print
)
gh aw compile --syft "${workflows[@]}"

- name: Authenticate to GHCR for image scanning
run: echo "${{ github.token }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
Expand Down
8 changes: 3 additions & 5 deletions scripts/ci/postprocess-smoke-workflows.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,9 @@ const codexWorkflowPaths = [
// with a source build, which is incompatible (e.g. the standalone awf bundle
// rejects --build-local: "requires a full repository checkout").
//
// Currently empty: all workflows (including the network-isolation variants)
// build awf/firewall from the repo's own source and use the compiler-default
// mcp gateway version. The full-repo checkout injected by the post-processor
// makes --build-local valid everywhere.
const releaseModeLockFiles = new Set<string>([]);
const releaseModeLockFiles = new Set<string>([
'smoke-bounded-queries.lock.yml',
]);

// Auto-discover all lock files so new workflows are automatically included.
// This avoids the recurring bug where adding a new workflow .md file and
Expand Down
5 changes: 0 additions & 5 deletions scripts/ci/smoke-bounded-queries.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,26 +21,21 @@ run_inside_agent() {
local schema expected_sequence result_kind
case "${SMOKE_SENSITIVITY:-}" in
public)
# The array admits 2^28 results, so each invocation costs 32 bits.
# Public must remain unmetered even after three invocations (96 bits).
schema="$ARRAY_SCHEMA"
expected_sequence="ok ok ok"
result_kind="array"
;;
internal)
# Two 32-bit invocations exactly exhaust the 64-bit internal budget.
schema="$ARRAY_SCHEMA"
expected_sequence="ok ok error"
result_kind="array"
;;
confidential)
# A boolean costs 5 bits, so only one fits in the 8-bit budget.
schema="$BOOLEAN_SCHEMA"
expected_sequence="ok error error"
result_kind="boolean"
;;
sealed)
# The zero-bit sealed budget must reject even the first query.
schema="$BOOLEAN_SCHEMA"
expected_sequence="error error error"
result_kind="boolean"
Expand Down
Loading