Repository navigation
feat(governance): T-14 机器化第一面 + 红队 required check 落地修正(W2-C3 .github#275,ADR-0083) #336
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
f69be05
e91ad65
8e68615
c6a0a01
5776dca
e40c237
8cd468a
d4e7d19
c96c6c6
969f7b1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,6 +22,88 @@ jobs: | |
| # hash 即 CI-Workflows v1 tag 当前指向,升级 v1 须同步换 hash | ||
| uses: Cloudbird-Software/CI-Workflows/.github/workflows/hygiene.yml@61191f87c537f6e887695517121c6e530a838261 # v1 | ||
|
|
||
| t14-spec-suite: | ||
| # T-14 第一面(ADR-0083 决策 1/2,testing.yaml T-14,#263 W2-C3 .github#275): | ||
| # 1) specs/** 变更的 PR 必须携带同目录 suite/(≥1 非空测试文件且含真实 | ||
| # 断言)——缺失即红(合并阻断;fail-closed:files API 失败视同 specs 变更)。 | ||
| # 2) suite 可执行性证明:全部 specs/*/suite 在本 job 真实执行(unittest 风格、 | ||
| # 零第三方依赖)——"存在但摆拍"的红队层兜底(S1'/S2')之外的机器底线。 | ||
| # T-14 第二面(fail-before 逐变更 + 实现 PR 卡测试解析 + holdout 注册校验) | ||
| # 随后续波次落地,本 job 只锁第一面。 | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| needs: hygiene | ||
| if: always() | ||
|
Comment on lines
+33
to
+36
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 将
将 #!/bin/bash
set -euo pipefail
sed -n '25,106p;107,132p' .github/workflows/gate.yml
jq -r '.rules[]
| select(.type == "required_status_checks")
| .parameters.required_status_checks[].context' \
governance/rulesets/main-protection.json🤖 Prompt for AI Agents |
||
| permissions: | ||
| contents: read | ||
| pull-requests: read | ||
| steps: | ||
| - name: hygiene green? | ||
| env: | ||
| NEEDS: ${{ toJSON(needs) }} | ||
| run: | | ||
| echo "$NEEDS" | ||
| if echo "$NEEDS" | jq -e '[to_entries[] | select(.value.result != "success")] | length > 0' >/dev/null; then | ||
| echo "::error::hygiene 未通过(skipped 不算绿——ADR-0032 严格断言)"; exit 1 | ||
| fi | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| - name: T-14 spec PR suite 强制(PR 事件) | ||
| if: github.event_name == 'pull_request' | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| PR_API: "repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}" | ||
| run: | | ||
| set -euo pipefail | ||
| # 分页拉全量文件清单(fail-closed:任何页失败=红——不能因盲而放行) | ||
| PAGE=1; SPECS_HIT=0; SPEC_DIRS="" | ||
| while :; do | ||
| PAGEJSON=$(gh api "$PR_API/files?per_page=100&page=$PAGE") | ||
| if ! jq -e 'type == "array"' <<<"$PAGEJSON" >/dev/null 2>&1; then | ||
| echo "::error::files API 第 $PAGE 页失败——T-14 判定完整性无法保证(fail-closed)"; exit 1 | ||
| fi | ||
|
Comment on lines
+62
to
+65
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 3. Unhandled gh api failure In the new T-14 presence check loop, a gh api failure will abort the step via set -e before emitting the intended ::error::files API 第 N 页失败... message, reducing diagnosability and making “fail-closed with explicit error” behavior inconsistent. This can turn transient API/permission failures into hard-to-debug red checks. Agent Prompt
|
||
| N=$(jq 'length' <<<"$PAGEJSON") | ||
| DIRS=$(jq -r '[.[].filename | select(startswith("specs/")) | split("/")[1] | select(. != "")] | unique | .[]' <<<"$PAGEJSON") | ||
| if [[ -n "$DIRS" ]]; then SPECS_HIT=1; SPEC_DIRS="$(printf '%s\n%s' "$SPEC_DIRS" "$DIRS" | sort -u)"; fi | ||
| [[ $N -lt 100 ]] && break | ||
| PAGE=$((PAGE+1)) | ||
| done | ||
|
Comment on lines
+61
to
+71
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n -C 8 'changed_files|GOT=|GOT=\$\(\(GOT\+N\)\)|3000 上限|SPEC_DIRS' \
.github/workflows/gate.ymlRepository: Cloudbird-Software/.github Length of output: 4332 🏁 Script executed: #!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
p = Path(".github/workflows/gate.yml")
lines = p.read_text().splitlines()
def block(start, end):
return "\n".join(lines[start-1:end])
t14 = block(52, 106)
adr = block(216, 252)
print("T14_HAS_CHANGED_FILES=", ".changed_files" in t14)
print("T14_HAS_GOT_COUNTER=", "GOT=" in t14)
print("ADR_HAS_CHANGED_FILES=", ".changed_files" in adr)
print("ADR_HAS_GOT_COUNTER=", "GOT=" in adr)
print("T14_PAGE_LOOP=", all(x in t14 for x in (
'gh api "$PR_API/files?per_page=100&page=$PAGE"',
'N=$(jq \'length\'',
'[[ $N -lt 100 ]] && break',
)))
print("ADR_COUNT_CHECK=", 'if [ "$GOT" -lt "$CHANGED" ]; then' in adr)
PY
sed -n '20,110p' .github/workflows/gate.ymlRepository: Cloudbird-Software/.github Length of output: 4327 🌐 Web query:
💡 Result: The GitHub REST API endpoint to list files in a pull request (GET /repos/{owner}/{repo}/pulls/{pull_number}/files) is documented to return a maximum of 3000 files [1][2][3]. Key details regarding this limit and API behavior include: Pagination: The response is paginated. By default, it returns 30 files per page, but you can increase this up to a maximum of 100 files per page using the per_page query parameter [1][2]. Maximum Limit: While you can paginate through results, the endpoint will not return more than 3000 files total for a single pull request [1][3]. Verification: To confirm if you have retrieved all files, you can compare the number of files collected via pagination against the changed_files count provided in the base pull request object (returned by GET /repos/{owner}/{repo}/pulls/{pull_number}) [4]. If changed_files exceeds 3000, you will be unable to retrieve the complete list of files through this REST API endpoint [4]. Citations:
为 T-14 文件清单增加截断检测 当 🤖 Prompt for AI Agents |
||
| if [[ $SPECS_HIT -ne 1 ]]; then echo "非 specs/** 变更,T-14 presence 检查跳过"; exit 0; fi | ||
| MISSING=0 | ||
| for d in $SPEC_DIRS; do | ||
| SUITE="specs/$d/suite" | ||
| if [[ ! -d "$SUITE" ]]; then | ||
| echo "::error::T-14:specs/$d 变更但缺 $SUITE/(spec PR 必须携带测试套件——ADR-0083 决策 1)"; MISSING=1; continue | ||
| fi | ||
| if ! ls "$SUITE"/*.py >/dev/null 2>&1; then | ||
| echo "::error::T-14:$SUITE 无 .py 测试文件"; MISSING=1; continue | ||
| fi | ||
| HAS_ASSERT=0 | ||
| for tf in "$SUITE"/*.py; do | ||
| [[ -s "$tf" ]] && grep -q "assert" "$tf" && HAS_ASSERT=1 && break | ||
|
Comment on lines
+82
to
+84
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
mkdir -p "$tmp/suite"
printf '# assert\n' > "$tmp/suite/test_placeholder.py"
(
cd "$tmp"
python3 -m unittest discover -s suite -p 'test_*.py' -v
)Repository: Cloudbird-Software/.github Length of output: 262 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- workflow context ---'
sed -n '55,105p' .github/workflows/gate.yml
printf '%s\n' '--- self-check context ---'
sed -n '75,115p' specs/ISSUE-263/suite/test_ir263_artifacts.py
printf '%s\n' '--- T-14 and related checks ---'
rg -n -C 3 'T-14|real_assertions|HAS_ASSERT|unittest discover|assert.*suite|suite.*assert' .github specs scripts standards docs governance 2>/dev/null || true
printf '%s\n' '--- candidate suite files ---'
find specs -path '*/suite/*.py' -type f -print | sort | head -80Repository: Cloudbird-Software/.github Length of output: 15394 🏁 Script executed: #!/bin/bash
set -euo pipefail
python3 - <<'PY'
import ast
from pathlib import Path
paths = [Path("specs/ISSUE-263/suite/test_ir263_artifacts.py")]
for path in paths:
tree = ast.parse(path.read_text())
test_methods = []
executable_assertions = []
textual_assertions = 0
for node in ast.walk(tree):
if isinstance(node, ast.Constant) and isinstance(node.value, str):
textual_assertions += node.value.count("assert")
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name.startswith("test_"):
test_methods.append(node.name)
for child in ast.walk(node):
if isinstance(child, ast.Assert):
executable_assertions.append(f"{node.name}: assert")
elif (
isinstance(child, ast.Call)
and isinstance(child.func, ast.Attribute)
and child.func.attr.startswith("assert")
):
executable_assertions.append(f"{node.name}: self.{child.func.attr}")
print(path)
print("test methods:", len(test_methods), test_methods)
print("semantic assertions:", len(executable_assertions))
print("textual 'assert' occurrences in string literals:", textual_assertions)
print("semantic assertion samples:", executable_assertions[:8])
placeholder = ast.parse("# assert\n")
print("placeholder test methods:", [
n.name for n in ast.walk(placeholder)
if isinstance(n, ast.FunctionDef) and n.name.startswith("test_")
])
print("placeholder semantic assertions:", [
n for n in ast.walk(placeholder)
if isinstance(n, ast.Assert)
])
PY
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
mkdir -p "$tmp/specs/X/suite"
printf '# assert\n' > "$tmp/specs/X/suite/test_placeholder.py"
printf '%s\n' '--- presence predicate ---'
if ls "$tmp/specs/X/suite"/*.py >/dev/null 2>&1 &&
grep -q "assert" "$tmp/specs/X/suite"/*.py; then
echo "presence: PASS"
else
echo "presence: FAIL"
fi
printf '%s\n' '--- unittest discovery ---'
(cd "$tmp/specs/X" && python3 -m unittest discover -s suite -p 'test_*.py' -v)Repository: Cloudbird-Software/.github Length of output: 1217 将断言验证改为语义检查。
📍 Affects 2 files
🤖 Prompt for AI Agents |
||
| done | ||
| if [[ $HAS_ASSERT -ne 1 ]]; then | ||
| echo "::error::T-14:$SUITE 测试文件无真实断言(须含 assert——摆拍套件见红队 S1'/S2' 攻击面)"; MISSING=1 | ||
| fi | ||
| done | ||
| [[ $MISSING -eq 0 ]] && echo "OK T-14 presence:$SPEC_DIRS 均携带含断言的 suite/" | ||
| exit $MISSING | ||
| - name: T-14 suite 真实执行(全部 specs/*/suite) | ||
| run: | | ||
| set -euo pipefail | ||
| shopt -s nullglob | ||
| suites=(specs/*/suite) | ||
| shopt -u nullglob | ||
| if [[ ${#suites[@]} -eq 0 ]]; then | ||
| echo "::error::specs/ 下无任何 suite/——治理仓 spec 目录存在但测试面丢失(fail-closed)"; exit 1 | ||
| fi | ||
| for s in "${suites[@]}"; do | ||
| echo "-- 执行 $s" | ||
| ( cd "$(dirname "$s")" && python3 -m unittest discover -s suite -p 'test_*.py' -v ) | ||
| done | ||
| echo "OK 全部 specs suite 执行通过" | ||
|
|
||
| gate: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 # testing.yaml T-01 "gate<5min" 原则的硬上限(红队 #18 P2:无 timeout 的 job 失控可挂 6h) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -60,8 +60,7 @@ | |
| "context": "org-gate" | ||
| }, | ||
| { | ||
| "context": "adversary", | ||
| "integration_id": 0 | ||
| "context": "adversary" | ||
| } | ||
| ] | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| """specs/ISSUE-263 自有验收套件(T-14 第一面,ADR-0083)。 | ||
|
|
||
| 对 #263「卡绑定测试与红队守门制度」的治理落盘工件做结构化断言—— | ||
| IR 自身首先受 T-14 约束(此前 specs/ISSUE-263 无 suite/ 是独立验证 | ||
| 发现的违规样本)。unittest 风格、零第三方依赖(yaml 断言用文本锚点, | ||
| 避免 gate job 依赖面扩大)。 | ||
| """ | ||
| import json | ||
| import os | ||
| import re | ||
| import unittest | ||
|
|
||
| ROOT = os.path.normpath(os.path.join(os.path.dirname(__file__), "..", "..", "..")) | ||
| TRANSITIONS = os.path.join(ROOT, "governance", "transitions.yaml") | ||
| TESTING = os.path.join(ROOT, "governance", "policy", "testing.yaml") | ||
| GOVERNANCE = os.path.join(ROOT, "governance", "GOVERNANCE.yaml") | ||
| MAIN_PROT = os.path.join(ROOT, "governance", "rulesets", "main-protection.json") | ||
| EXPECTED = os.path.join(ROOT, "governance", "expected-state.json") | ||
| REPOS = os.path.join(ROOT, "governance", "REPOS.yaml") | ||
|
|
||
|
|
||
| def read(path): | ||
| with open(path, encoding="utf-8") as f: | ||
| return f.read() | ||
|
|
||
|
|
||
| class TestStateMachine(unittest.TestCase): | ||
| """AC-12:T5/T6 入册且语义齐全。""" | ||
|
|
||
| def test_t5_t6_registered(self): | ||
| t = read(TRANSITIONS) | ||
| for tid in ("T5", "T6"): | ||
| self.assertIsNotNone( | ||
| re.search(rf"^\s*- id: {tid}\b", t, re.M), | ||
| f"transitions.yaml 缺 {tid}") | ||
|
|
||
| def test_t5_suite_ready_predicate(self): | ||
| self.assertIn("suite", read(TRANSITIONS), "T5 须含 suite 就绪谓词") | ||
|
|
||
|
|
||
| class TestPolicyClauses(unittest.TestCase): | ||
| """W1-C5 条款入册:T-14/T-15/AR-10。""" | ||
|
|
||
| def test_t14_t15_registered(self): | ||
| t = read(TESTING) | ||
| self.assertRegex(t, r"T-14,\s*name:\s*card_bound_test_required") | ||
| self.assertRegex(t, r"T-15,\s*name:\s*intent_backstop") | ||
|
|
||
| def test_ar10_veto_clause(self): | ||
| g = read(GOVERNANCE) | ||
| self.assertIn("红队守门", g) | ||
| self.assertIn("needs-human", g) | ||
| self.assertIn("EXPECTED_SKIP", g) | ||
|
|
||
|
|
||
| class TestRulesetEnforcement(unittest.TestCase): | ||
| """W4-C3:adversary 为 required check 且无非法 integration_id(ADR-0083)。""" | ||
|
|
||
| def setUp(self): | ||
| self.rs = json.loads(read(MAIN_PROT)) | ||
|
|
||
| def test_adversary_required_no_integration_id(self): | ||
| checks = None | ||
| for rule in self.rs["rules"]: | ||
| if rule["type"] == "required_status_checks": | ||
| checks = rule["parameters"]["required_status_checks"] | ||
| self.assertIsNotNone(checks, "main-protection 缺 required_status_checks") | ||
| contexts = {c["context"]: c for c in checks} | ||
| self.assertIn("adversary", contexts, "adversary 不在 required checks") | ||
| for ctx, entry in contexts.items(): | ||
| self.assertNotIn( | ||
| "integration_id", entry, | ||
| f"required check '{ctx}' 携带 integration_id(非法——ADR-0083 决策 3)") | ||
|
|
||
| def test_gate_and_org_gate_kept(self): | ||
| rule = next(r for r in self.rs["rules"] if r["type"] == "required_status_checks") | ||
| contexts = [c["context"] for c in rule["parameters"]["required_status_checks"]] | ||
| for c in ("gate", "org-gate"): | ||
| self.assertIn(c, contexts) | ||
|
|
||
|
|
||
| class TestExpectedState(unittest.TestCase): | ||
| """W2-C1:验证者 APP 登记面。""" | ||
|
|
||
| def test_verifier_app_registered(self): | ||
| d = json.loads(read(EXPECTED)) | ||
| va = d.get("verifier_app") or {} | ||
| self.assertTrue(isinstance(va.get("id"), int), "verifier_app.id 须为数值") | ||
| self.assertEqual(va.get("slug"), "verifier-app") | ||
|
|
||
| def test_holdout_repo_declared(self): | ||
| self.assertRegex(read(REPOS), r"name:\s*holdout\b") | ||
|
|
||
|
|
||
| class TestSelfSuiteValidity(unittest.TestCase): | ||
| """T-14 自反:本 suite 含真实断言(非摆拍——S1'/S2' 攻击面对象)。""" | ||
|
|
||
| def test_suite_has_real_assertions(self): | ||
| src = read(os.path.abspath(__file__)) | ||
| asserts = len(re.findall(r"\bassert", src)) | ||
| self.assertGreater(asserts, 10, f"断言过少({asserts})——疑似摆拍套件") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| unittest.main() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
2. T-14 not merge-blocking
🐞 Bug≡ CorrectnessAgent Prompt
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools