Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
5b39c41
Add Chrome Releases blog integration to security-audit skill
mattleibow May 29, 2026
60cd397
Add Chrome Releases expectations to security-audit evals
mattleibow May 29, 2026
71346c9
Add Chrome Releases section to HTML security report viewer
mattleibow May 29, 2026
049215e
Add blog post links to Chrome Releases table in HTML viewer
mattleibow May 29, 2026
fd652e7
Sort Chrome Releases CVEs: Skia first, then ANGLE, Canvas, etc.
mattleibow May 29, 2026
34076fc
Add descriptive text to summary cards in HTML viewer
mattleibow May 29, 2026
eaa639b
Compact CVE table: truncated descriptions with click-to-expand
mattleibow May 29, 2026
db69f0a
Make description column fill remaining width with CSS ellipsis
mattleibow May 29, 2026
70abf1e
Give description column 50% width in CVE tables
mattleibow May 29, 2026
89e74ab
Widen source column (150px) and shrink description (35%) for non-Chro…
mattleibow May 29, 2026
ca4ee84
Fix Fixed In column: populate from fixMilestone, hide when empty
mattleibow May 29, 2026
151a002
Sort CVE table rows: affected first, then by severity within each group
mattleibow May 29, 2026
e29edbb
Add Markdown report renderer for AI consumption
mattleibow May 29, 2026
6a71aeb
Simplify Step 11: use generated MD instead of manual in-chat summary
mattleibow May 29, 2026
247640b
Remove description truncation from MD renderer
mattleibow May 29, 2026
5d22ed2
Fix MD tables: sanitize newlines/pipes, move Chrome Releases after CG
mattleibow May 29, 2026
91685b7
Fix version verification table: use 'name' field for dependency column
mattleibow May 29, 2026
f1f0c6f
Fix blocking issues from Opus 4.8 review
mattleibow May 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 94 additions & 33 deletions .agents/skills/security-audit/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,29 +32,32 @@ combined into a single unified report.

## Key References

- **[references/chrome-releases.md](references/chrome-releases.md)** — Chrome Releases blog: RSS query, two-pass extraction (regex + AI review), cross-referencing with NVD
- **[references/skia-cve-resolution.md](references/skia-cve-resolution.md)** — Skia core CVE pipeline (NVD → Bug ID → Commit → Branch → Cherry-pick → Reachability). **The Skia process is fine-grained — read this before auditing Skia.**
- **[references/third-party-deps.md](references/third-party-deps.md)** — Third-party CVE process (libpng, freetype, harfbuzz, etc.): version verification, fix-commit ancestry, known false positives
- **[references/cg-alerts.md](references/cg-alerts.md)** — Component Governance alerts: ADO pipeline queries, Docker container CVEs, fix locations
- **[documentation/dev/dependencies.md](../../../documentation/dev/dependencies.md)** — Dependency list, cgmanifest format, Skia-specific tracking notes
- **[references/report-template.md](references/report-template.md)** — Markdown report format
- **[references/report-template.md](references/report-template.md)** — Markdown format guide (used by `render-security-audit-md.py`)
- **[references/report-schema.md](references/report-schema.md)** — JSON schema for structured output
- **[references/security-audit-schema.json](references/security-audit-schema.json)** — Machine-readable JSON Schema (Draft 2020-12)
- **[scripts/validate-security-audit.py](scripts/validate-security-audit.py)** — Validates report JSON against schema + semantic checks
- **[scripts/render-security-audit.py](scripts/render-security-audit.py)** — Renders JSON → standalone HTML
- **[scripts/render-security-audit-md.py](scripts/render-security-audit-md.py)** — Renders JSON → Markdown (for AI consumption)
- **[scripts/viewer.html](scripts/viewer.html)** — HTML template (Bootstrap 5)

## Workflow

1. Search GitHub issues/PRs (all deps including Skia)
2. Verify dependency versions from submodule/DEPS/headers (NOT cgmanifest.json)
3. Audit Skia core CVEs — see [Skia CVE Resolution](references/skia-cve-resolution.md)
4. Audit third-party dependency CVEs — see [Third-Party Deps](references/third-party-deps.md)
5. Query Component Governance alerts — see [CG Alerts](references/cg-alerts.md)
6. Check false positives
7. Assemble structured JSON report
8. Validate report (`validate-security-audit.py`)
9. Render HTML (`render-security-audit.py`)
10. Present markdown summary to user
2. Query Chrome Releases blog (`query-chrome-releases.py`) — see [Chrome Releases](references/chrome-releases.md)
3. Verify dependency versions from submodule/DEPS/headers (NOT cgmanifest.json)
4. Audit Skia core CVEs — see [Skia CVE Resolution](references/skia-cve-resolution.md)
5. Audit third-party dependency CVEs — see [Third-Party Deps](references/third-party-deps.md)
6. Query Component Governance alerts — see [CG Alerts](references/cg-alerts.md)
7. Check false positives
8. Assemble structured JSON report
9. Validate report (`validate-security-audit.py`)
10. Render HTML (`render-security-audit.py`)
11. Present markdown summary to user

---

Expand All @@ -70,7 +73,50 @@ Search PRs in both `mono/SkiaSharp` and `mono/skia` for dependency updates alrea

---

### Step 2: Verify Dependency Versions
### Step 1.5: Query Chrome Releases Blog

> 🔍 The Chrome Releases blog often discloses Skia CVEs **before NVD** processes them.
> This step provides early detection and cross-validation.

**See [references/chrome-releases.md](references/chrome-releases.md)** for full details on the
data source, script usage, and AI review instructions.

#### Run the script

```bash
python3 .agents/skills/security-audit/scripts/query-chrome-releases.py \
--verbose --output output/ai/chrome-releases-cache.json
```

This takes ~10-30 seconds (fetches RSS feed pages). Cache is reused if < 24 hours old.

#### Two-pass review

1. **Deterministic (regex):** Read `structured_cves[]` from the JSON output. These are
high-confidence CVEs extracted from the known blog format. Each has a CVE ID, severity,
component, bug ID, and milestone already parsed.

2. **AI review (broad):** Scan `posts[].text_content` for anything the regex missed:
- CVE mentions not captured by regex (format variations, line breaks)
- Indirect Skia references ("type confusion in Rendering")
- Wild exploitation notices (highest priority!)
- Related component CVEs (GPU, Compositing) that may involve Skia code

#### Cross-reference with NVD (Step 3)

After the NVD query in Step 3, compare results:

| Chrome Releases | NVD | Interpretation |
|-----------------|-----|----------------|
| ✅ Found | ✅ Found | Normal — use NVD CVSS, Chrome Releases for milestone |
| ✅ Found | ❌ Not found | **Early disclosure** — NVD may be delayed. Use Chrome severity. |
| ❌ Not found | ✅ Found | Vendor bulletin CVE (Android/Huawei) — not in Chrome stable |

Set the `source` field on each CVE object: `"both"`, `"chrome_releases"`, or `"nvd"`.

---

### Step 3: Verify Dependency Versions

> ⚠️ **CRITICAL: Never trust `cgmanifest.json` blindly.** Always verify versions against the
> actual submodule, DEPS file, and source headers. cgmanifest.json is manually maintained
Expand Down Expand Up @@ -156,7 +202,7 @@ Report mismatches as findings.

---

### Step 3: Audit Skia Core CVEs
### Step 4: Audit Skia Core CVEs

> 🛑 Skia is the product, not just a dependency. Every Skia CVE must be resolved to a
> specific fix commit, branch, cherry-pick test, and reachability assessment. Classification
Expand All @@ -176,7 +222,7 @@ process**, including:

---

### Step 4: Audit Third-Party Dependency CVEs
### Step 5: Audit Third-Party Dependency CVEs

For libpng, freetype, harfbuzz, libexpat, brotli, zlib, libjpeg-turbo, libwebp, ANGLE
submodules, etc.
Expand All @@ -191,7 +237,7 @@ submodules, etc.

---

### Step 5: Query Component Governance Alerts
### Step 6: Query Component Governance Alerts

CG scans Docker container images and build-time deps from both ADO pipelines. CG alerts are
invisible to GitHub Issues and NVD searches alone.
Expand All @@ -212,7 +258,7 @@ invisible to GitHub Issues and NVD searches alone.

---

### Step 6: Check False Positives
### Step 7: Check False Positives

Before flagging anything, verify the CVE actually affects SkiaSharp.

Expand All @@ -231,7 +277,7 @@ Before flagging anything, verify the CVE actually affects SkiaSharp.

---

### Step 7: Assemble Structured JSON Report
### Step 8: Assemble Structured JSON Report

> 🛑 **MANDATORY:** The audit MUST produce a JSON file conforming to
> [references/report-schema.md](references/report-schema.md). This is the machine-readable
Expand All @@ -244,7 +290,8 @@ Build the JSON object with these top-level keys:
3. **`versionVerification`** — One entry per dependency with DEPS commit, verified version, cgmanifest version, match boolean
4. **`findings`** — Array of finding objects sorted by priority then severity. **ONE object per dependency** (e.g., one "skia" finding containing ALL Skia CVEs regardless of status). Each has `dependency`, `status`, `cves[]`, `nonChromeCves[]`, `action`, `notes`. The `status` reflects the WORST-case status among the CVEs.
5. **`cgAlerts`** — The complete raw JSON from `query-cg-alerts.py` (full `alerts` array, do not summarize)
6. **`nextSteps`** — Prioritized action items with severity, command, and reason
6. **`chromeReleases`** — Chrome Releases blog data. Transform the script's snake_case output (`cve_id`→`cveId`, `bug_id`→`bugId`, `blog_post_url`→`blogPostUrl`) into `structuredCves[]`. Also copy `blogPostUrl` onto matching CVEs in `findings[].cves[]`. See [report-schema.md](references/report-schema.md#chromereleases--chrome-releases-blog-data-optional) for the full field mapping.
7. **`nextSteps`** — Prioritized action items with severity, command, and reason

> 🛑 **COMPLETENESS REQUIREMENT:** The `findings` array MUST include **every CVE returned
> by the NVD query** (Step 3 of skia-cve-resolution.md). CVEs that are verified as already
Expand All @@ -263,7 +310,7 @@ Save as `output/ai/security-audit-{date}.json`.

---

### Step 8: Validate Report
### Step 9: Validate Report

> 🛑 **MANDATORY:** Always validate before rendering. Fix any errors reported.

Expand All @@ -277,20 +324,27 @@ Warnings are informational — errors must be fixed before proceeding.

---

### Step 9: Render HTML Report
### Step 10: Render HTML + Markdown Reports

> 🛑 **MANDATORY:** Always generate the HTML report. The human needs a readable dashboard.
> 🛑 **MANDATORY:** Always generate both reports.

```bash
python3 .agents/skills/security-audit/scripts/render-security-audit.py \
output/ai/security-audit-{date}.json

python3 .agents/skills/security-audit/scripts/render-security-audit-md.py \
output/ai/security-audit-{date}.json
```

This produces a self-contained HTML file (Bootstrap 5, CDN CSS only) alongside the JSON.
This produces:
- **HTML** — Self-contained dashboard (Bootstrap 5) for human review
- **Markdown** — Comprehensive report for AI consumption and action suggestions

The HTML renders:

- Summary cards with status counts
- Collapsible findings with CVE tables, severity badges, NVD links
- Chrome Releases blog section with above-milestone CVEs by component
- Version verification table with match/mismatch indicators
- Skia upstream verification details with commit links
- Prioritized next steps with severity-coded borders
Expand All @@ -305,24 +359,31 @@ Present the output path to the user:

---

### Step 10: Present Markdown Summary

After generating JSON and HTML, present a concise markdown summary to the user in the
conversation (using the report-template.md format). This is in ADDITION to the JSON+HTML
files, not instead of them.
### Step 11: Present Summary to User

**Priority order** (applies equally to Skia core and third-party deps):
The Markdown report was already generated in Step 10. Present a brief summary in the
conversation pointing to the generated files:

1. 🔴 User-reported + no PR
2. ✅ User-reported + PR ready
3. 🟡 User-reported + PR needs work
4. 🆕 Undiscovered CVEs (proactively found, no user-filed issue)
5. ⚪ False positives
```
✅ Reports generated:
• output/ai/security-audit-{date}.json (structured data)
• output/ai/security-audit-{date}.html (interactive dashboard)
• output/ai/security-audit-{date}.md (full markdown for AI review)

m147 • 2026-05-29 • 102 CVEs • Highest: CRITICAL
🔴 0 attention · 🆕 0 undiscovered · ⚪ 1 FP · ✅ 6 clean
📰 Chrome Releases: 146 Skia-relevant CVEs (16 above current milestone)
```

Within each priority level, sort by severity (CRITICAL > HIGH > MEDIUM > LOW).
Then highlight the **top actionable items** from the report:
- Any `needs_attention` or `undiscovered` findings
- Chrome Releases CVEs above our current milestone (especially Skia/ANGLE)
- Critical/High CG alerts

#### Report quality rules

These rules apply to the JSON assembly (Step 8) and are enforced by the renderers:

1. **Skia bump recommendations must target the highest-severity CVE**, not the lowest. If
there are HIGH CVEs at m146 and a MEDIUM at m133, recommend m146 as the target.
2. **Don't include already-closed GitHub issues** unless directly relevant to an open
Expand Down
26 changes: 25 additions & 1 deletion .agents/skills/security-audit/evals/evals.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,13 @@
"The final markdown lists findings sorted by priority: user-reported+no-PR first, then undiscovered CVEs, then false positives",
"The final markdown includes a Next Steps section with specific actionable recommendations",
"For Skia CVEs, the report includes a Chrome-versioned CVEs table showing CVE ID, Fixed-In milestone, Severity, and Description",
"The report correctly identifies which Skia CVEs are 'potentially affected' (fix milestone > our verified milestone) vs 'already fixed' (fix milestone <= our verified milestone)"
"The report correctly identifies which Skia CVEs are 'potentially affected' (fix milestone > our verified milestone) vs 'already fixed' (fix milestone <= our verified milestone)",
"The audit runs query-chrome-releases.py to fetch Chrome Releases blog data",
"The JSON report contains a top-level 'chromeReleases' key with queriedAt, monthsQueried, postsReviewed, totalCvesExtracted, skiaRelevantCves, and structuredCves fields",
"The chromeReleases.postsReviewed is greater than 0 (blog was actually queried)",
"The chromeReleases.structuredCves array contains CVE objects with cveId, severity, component, bugId, and milestone fields",
"The chromeReleases.skiaRelevantCves count is greater than 0 (Skia-relevant CVEs were found)",
"The audit cross-references Chrome Releases CVEs with NVD results, using the source field ('chrome_releases', 'nvd', or 'both') on CVE objects in findings"
]
},
{
Expand Down Expand Up @@ -64,6 +70,24 @@
"The report uses the verified milestone (not cgmanifest.json) when classifying which Skia CVEs are potentially affected vs already fixed",
"The report ends with prioritized, actionable next steps"
]
},
{
"id": 4,
"prompt": "check the Chrome Releases blog for any Skia security issues we should know about",
"expected_output": "A report showing Skia-relevant CVEs extracted from the Chrome Releases blog, with structured data including CVE IDs, severities, components, bug IDs, and milestones. Should cross-reference with our current milestone to identify what's fixed vs what needs attention.",
"files": [],
"expectations": [
"The audit runs query-chrome-releases.py (or equivalent RSS fetch) to get Chrome Releases blog data",
"The output includes CVEs with 'Skia' as the component (direct Skia CVEs)",
"The output includes CVEs from related components (ANGLE, GPU, Canvas, Fonts, WebGL)",
"Each structured CVE has a severity level (Critical, High, Medium, or Low)",
"Each structured CVE has a milestone number indicating which Chrome version fixed it",
"The audit identifies which CVEs are beyond our current milestone (m147) and thus potentially unpatched",
"The audit identifies CVEs already covered by our milestone as 'already fixed'",
"The output mentions the date range queried (approximately 6 months)",
"The audit uses both regex extraction (Pass 1) and AI review of raw text (Pass 2)",
"If any CVEs mention 'wild exploitation' or 'actively exploited', they are flagged as highest priority"
]
}
]
}
Loading
Loading