Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 19 additions & 14 deletions .github/workflows/daily-firewall-report.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 19 additions & 14 deletions .github/workflows/daily-firewall-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,26 +161,31 @@ Generate a comprehensive daily report of all rejected domains across all agentic

This prevents unnecessary re-analysis of the same data and significantly reduces token usage.

### Step 1: Identify Workflows with Firewall Feature
### Step 1: Collect Recent Firewall-Enabled Workflow Runs

1. List all workflows in the repository
2. For each workflow that has `network.firewall: true` in its frontmatter, note the workflow name
3. Create a list of all firewall-enabled workflows
Use the `logs` command with `--firewall true` to efficiently collect workflow runs that have firewall enabled:

**Example frontmatter structure:**
```yaml
network:
firewall: true
```bash
# Get up to 100 recent workflow runs that used firewall (within past 7 days)
gh aw logs --firewall true --start-date -7d -c 100
```

**Note:** The firewall field is under `network`, not `features`.
This command:
1. Filters runs based on the `steps.firewall` field in `aw_info.json` (e.g., "squid" when enabled)
2. Returns only runs where firewall was enabled
3. Limits to runs from the past 7 days
4. Returns up to 100 matching runs

**Alternative:** To get runs for all firewall-enabled workflows across all time:
```bash
gh aw logs --firewall true -c 100
```

### Step 2: Collect Recent Workflow Runs
### Step 2: Analyze Firewall Logs from Collected Runs

For each firewall-enabled workflow:
1. Get up to 10 workflow runs that occurred within the past 7 days (if there are fewer than 10 runs in that window, include all available; if there are more, include only the most recent 10)
2. For each run ID, use the `audit` tool from the agentic-workflows MCP server with `--json` flag to get detailed firewall information
3. Store the run ID, workflow name, and timestamp for tracking
For each run collected in Step 1:
1. Use the `audit` tool from the agentic-workflows MCP server with `--json` flag to get detailed firewall information
2. Store the run ID, workflow name, and timestamp for tracking

**Using the audit tool:**
```bash
Expand Down
44 changes: 42 additions & 2 deletions pkg/cli/logs.go
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,8 @@ Examples:
` + constants.CLIExtensionPrefix + ` logs --engine claude # Filter logs by claude engine
` + constants.CLIExtensionPrefix + ` logs --engine codex # Filter logs by codex engine
` + constants.CLIExtensionPrefix + ` logs --engine copilot # Filter logs by copilot engine
` + constants.CLIExtensionPrefix + ` logs --firewall true # Filter logs with firewall enabled
` + constants.CLIExtensionPrefix + ` logs --firewall false # Filter logs without firewall
` + constants.CLIExtensionPrefix + ` logs -o ./my-logs # Custom output directory
` + constants.CLIExtensionPrefix + ` logs --branch main # Filter logs by branch name
` + constants.CLIExtensionPrefix + ` logs --branch feature-xyz # Filter logs by feature branch
Expand Down Expand Up @@ -371,6 +373,7 @@ Examples:
verbose, _ := cmd.Flags().GetBool("verbose")
toolGraph, _ := cmd.Flags().GetBool("tool-graph")
noStaged, _ := cmd.Flags().GetBool("no-staged")
firewallFilter, _ := cmd.Flags().GetString("firewall")
parse, _ := cmd.Flags().GetBool("parse")
jsonOutput, _ := cmd.Flags().GetBool("json")
timeout, _ := cmd.Flags().GetInt("timeout")
Expand Down Expand Up @@ -413,7 +416,16 @@ Examples:
}
}

if err := DownloadWorkflowLogs(workflowName, count, startDate, endDate, outputDir, engine, branch, beforeRunID, afterRunID, verbose, toolGraph, noStaged, parse, jsonOutput, timeout); err != nil {
// Validate firewall parameter
if firewallFilter != "" && firewallFilter != "true" && firewallFilter != "false" {
Comment thread
pelikhan marked this conversation as resolved.
Outdated
fmt.Fprintln(os.Stderr, console.FormatError(console.CompilerError{
Type: "error",
Message: fmt.Sprintf("invalid firewall value '%s'. Must be 'true', 'false', or empty", firewallFilter),
}))
os.Exit(1)
}

if err := DownloadWorkflowLogs(workflowName, count, startDate, endDate, outputDir, engine, branch, beforeRunID, afterRunID, verbose, toolGraph, noStaged, firewallFilter, parse, jsonOutput, timeout); err != nil {
fmt.Fprintln(os.Stderr, console.FormatError(console.CompilerError{
Type: "error",
Message: err.Error(),
Expand All @@ -434,6 +446,7 @@ Examples:
logsCmd.Flags().Int64("after-run-id", 0, "Filter runs with database ID after this value (exclusive)")
logsCmd.Flags().Bool("tool-graph", false, "Generate Mermaid tool sequence graph from agent logs")
logsCmd.Flags().Bool("no-staged", false, "Filter out staged workflow runs (exclude runs with staged: true in aw_info.json)")
logsCmd.Flags().String("firewall", "", "Filter runs by firewall usage: 'true' (only runs with firewall), 'false' (only runs without firewall), or '' (all runs)")
logsCmd.Flags().Bool("parse", false, "Run JavaScript parsers on agent logs and firewall logs, writing markdown to log.md and firewall.md")
logsCmd.Flags().Bool("json", false, "Output logs data as JSON instead of formatted console tables")
logsCmd.Flags().Int("timeout", 0, "Maximum time in seconds to spend downloading logs (0 = no timeout)")
Expand All @@ -442,7 +455,7 @@ Examples:
}

// DownloadWorkflowLogs downloads and analyzes workflow logs with metrics
func DownloadWorkflowLogs(workflowName string, count int, startDate, endDate, outputDir, engine, branch string, beforeRunID, afterRunID int64, verbose bool, toolGraph bool, noStaged bool, parse bool, jsonOutput bool, timeout int) error {
func DownloadWorkflowLogs(workflowName string, count int, startDate, endDate, outputDir, engine, branch string, beforeRunID, afterRunID int64, verbose bool, toolGraph bool, noStaged bool, firewallFilter string, parse bool, jsonOutput bool, timeout int) error {
logsLog.Printf("Starting workflow log download: workflow=%s, count=%d, startDate=%s, endDate=%s, outputDir=%s", workflowName, count, startDate, endDate, outputDir)
if verbose {
fmt.Fprintln(os.Stderr, console.FormatInfoMessage("Fetching workflow runs from GitHub Actions..."))
Expand Down Expand Up @@ -615,6 +628,33 @@ func DownloadWorkflowLogs(workflowName string, count int, startDate, endDate, ou
}
}

// Apply firewall filtering if --firewall flag is specified
if firewallFilter != "" {
// Check the firewall field in aw_info.json
awInfoPath := filepath.Join(result.LogsPath, "aw_info.json")
info, err := parseAwInfo(awInfoPath, verbose)
var hasFirewall bool
if err == nil && info != nil {
// Firewall is enabled if steps.firewall is non-empty (e.g., "squid")
hasFirewall = info.Steps.Firewall != ""
}

Copilot AI Nov 4, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parseAwInfo function is called multiple times for the same file within the same loop iteration (engine filter at line 617, staged filter at line 617, and now firewall filter at line 635). Consider parsing the aw_info.json file once per iteration and reusing the parsed result across all filters to avoid redundant file I/O and JSON parsing.

Copilot uses AI. Check for mistakes.

// Check if the run matches the filter
filterRequiresFirewall := firewallFilter == "true"
if filterRequiresFirewall && !hasFirewall {
if verbose {
fmt.Fprintln(os.Stderr, console.FormatInfoMessage(fmt.Sprintf("Skipping run %d: workflow does not use firewall (filtered by --firewall true)", result.Run.DatabaseID)))
}
continue
}
if !filterRequiresFirewall && hasFirewall {
if verbose {
fmt.Fprintln(os.Stderr, console.FormatInfoMessage(fmt.Sprintf("Skipping run %d: workflow uses firewall (filtered by --firewall false)", result.Run.DatabaseID)))
}
continue
}
}

// Update run with metrics and path
run := result.Run
run.TokenUsage = result.Metrics.TokenUsage
Expand Down
Loading
Loading