Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions actions/setup/md/playwright_awf_prompt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
<playwright-awf-policy>
This workflow runs Playwright CLI inside the AWF (agentic workflow firewall)
sandbox. The following rules take precedence over any generic Playwright CLI
skill guidance, including suggestions to `npm install`, use `npx` to fetch
missing packages, or navigate to arbitrary example domains.

- The Playwright CLI and Chromium browser are already staged for this sandbox.
Never install packages, browsers, or system dependencies at runtime, and
never run commands like `npm install -g @playwright/cli`, `npx playwright
install`, or similar. If the CLI or browser appears unavailable, report the
failure instead of trying to install anything.
Comment thread
Copilot marked this conversation as resolved.
Outdated
- If you need to exercise a local application, start its server **inside this
sandbox** (for example with `run` in the background) before using
`playwright-cli`. Bind the server only to `127.0.0.1` (or `localhost`),
preferably on an ephemeral port. Never bind to `0.0.0.0`, publish host
ports, or rely on an external preview/tunnel service.
- Before navigating with `playwright-cli`, poll the loopback URL until it
responds (a readiness/health check) instead of using a fixed sleep.
- Navigate only to the loopback URL of the server you started, or to domains
explicitly present in this workflow's allowed domains list. Do not navigate
to arbitrary public example sites.
- Traffic to `localhost`/`127.0.0.1` is served directly; all other HTTP/HTTPS
browser traffic is routed through the sandbox's proxy and is subject to the
workflow's domain allowlist. Do not attempt to change browser proxy
settings, bypass rules, or the proxy environment variables — they are
enforced by the sandbox, not by this agent.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in aceb416: reworded this section to state the required behavior (keep localhost/127.0.0.1 on the bypass list, route everything else through the proxy) and instruct the agent to fail closed and report the failure if the expected proxy topology isn't in place, rather than implying enforcement already exists.

- Never treat the API/model-provider proxy as a browser proxy; it exists only
to reach the model provider, not for browser navigation.
- Always clean up: close the browser/page, stop any server process you
started, and verify there are no leftover background processes before
finishing the task.
</playwright-awf-policy>
11 changes: 11 additions & 0 deletions docs/src/content/docs/reference/playwright.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,17 @@ network:

Allowing `example.com` automatically allows its subdomains.

### AWF Sandbox Policy

When the workflow runs inside the AWF sandbox (`sandbox.agent` enabled, or the
firewall enabled by default for the configured engine), the compiler injects
an additional policy prompt reinforcing the secure browser topology: bind
local servers to `127.0.0.1` only, wait for a loopback readiness check before
navigating, keep `localhost`/`127.0.0.1` on the proxy bypass list, and never
install packages or browsers at runtime. This guidance takes precedence over
generic Playwright CLI skill suggestions such as `npm install`/`npx` fallback
installation or navigating to arbitrary example domains.

### Browser Support

Chromium (Chrome/Edge), Firefox, and WebKit (Safari) are available.
Expand Down
1 change: 1 addition & 0 deletions pkg/workflow/prompt_constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ const (
prContextPushToPRBranchGuidanceFile = "pr_context_push_to_pr_branch_guidance.md"
tempFolderPromptFile = "temp_folder_prompt.md"
playwrightPromptFile = "playwright_prompt.md"
playwrightAWFPromptFile = "playwright_awf_prompt.md"
markdownPromptFile = "markdown.md"
xpiaPromptFile = "xpia.md"
cacheMemoryPromptFile = "cache_memory_prompt.md"
Expand Down
94 changes: 94 additions & 0 deletions pkg/workflow/prompts_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -778,6 +778,100 @@ This is a test workflow to verify playwright instructions come after temp folder
t.Logf("Successfully verified playwright instructions come after temp folder instructions in generated workflow")
}

func TestPlaywrightAWFPolicyPromptIncludedForCLIModeWithFirewall(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "gh-aw-playwright-awf-prompt-test-*")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)

testFile := filepath.Join(tmpDir, "test-workflow.md")
testContent := `---
on: push
engine: claude
tools:
playwright:
mode: cli
sandbox:
agent:
id: awf
---

# Test Workflow with Playwright CLI and firewall

This is a test workflow with playwright CLI mode and the firewall enabled.
`

if err := os.WriteFile(testFile, []byte(testContent), 0644); err != nil {
t.Fatalf("Failed to create test workflow: %v", err)
}

compiler := NewCompiler()
if err := compiler.CompileWorkflow(testFile); err != nil {
t.Fatalf("Failed to compile workflow: %v", err)
}

lockFile := stringutil.MarkdownToLockFile(testFile)
lockContent, err := os.ReadFile(lockFile)
if err != nil {
t.Fatalf("Failed to read generated lock file: %v", err)
}

lockStr := string(lockContent)

if !strings.Contains(lockStr, `\"file\":\"playwright_awf_prompt.md\"`) {
t.Error("Expected playwright AWF policy prompt file in renderer configuration when CLI mode and firewall are both enabled")
}
}

func TestPlaywrightAWFPolicyPromptNotIncludedWithoutFirewall(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "gh-aw-playwright-awf-no-firewall-test-*")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)

testFile := filepath.Join(tmpDir, "test-workflow.md")
testContent := `---
on: push
engine: claude
tools:
playwright:
mode: cli
features:
dangerously-disable-sandbox-agent: "unit test verifying prompt gating logic"
sandbox:
agent: false
strict: false
---

# Test Workflow with Playwright CLI, firewall disabled

This is a test workflow with playwright CLI mode but the firewall disabled.
`

if err := os.WriteFile(testFile, []byte(testContent), 0644); err != nil {
t.Fatalf("Failed to create test workflow: %v", err)
}

compiler := NewCompiler()
if err := compiler.CompileWorkflow(testFile); err != nil {
t.Fatalf("Failed to compile workflow: %v", err)
}

lockFile := stringutil.MarkdownToLockFile(testFile)
lockContent, err := os.ReadFile(lockFile)
if err != nil {
t.Fatalf("Failed to read generated lock file: %v", err)
}

lockStr := string(lockContent)

if strings.Contains(lockStr, "playwright_awf_prompt.md") {
t.Error("Did not expect playwright AWF policy prompt file when firewall is disabled")
}
}

// ============================================================================
// PR Context Prompt Tests
// ============================================================================
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ jobs:
env:
GH_AW_ACTIONS_DIR: ${{ runner.temp }}/gh-aw/actions
GH_AW_PROMPT: ${{ runner.temp }}/gh-aw/aw-prompts/prompt.txt
GH_AW_PROMPT_CONFIG: "{\"items\":[{\"content_env\":\"GH_AW_PROMPT_CONTENT_0000\"},{\"file\":\"xpia.md\"},{\"file\":\"temp_folder_prompt.md\"},{\"file\":\"markdown.md\"},{\"file\":\"playwright_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0001\"},{\"file\":\"github_mcp_tools_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0002\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0003\"}]}"
GH_AW_PROMPT_CONFIG: "{\"items\":[{\"content_env\":\"GH_AW_PROMPT_CONTENT_0000\"},{\"file\":\"xpia.md\"},{\"file\":\"temp_folder_prompt.md\"},{\"file\":\"markdown.md\"},{\"file\":\"playwright_prompt.md\"},{\"file\":\"playwright_awf_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0001\"},{\"file\":\"github_mcp_tools_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0002\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0003\"}]}"
GH_AW_EXPR_1A3A194A: ${{ github.event.discussion.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'discussion' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
GH_AW_EXPR_463A214A: ${{ github.event.pull_request.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'pull_request' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
GH_AW_EXPR_802A9F6A: ${{ github.event.issue.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'issue' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ jobs:
env:
GH_AW_ACTIONS_DIR: ${{ runner.temp }}/gh-aw/actions
GH_AW_PROMPT: ${{ runner.temp }}/gh-aw/aw-prompts/prompt.txt
GH_AW_PROMPT_CONFIG: "{\"items\":[{\"content_env\":\"GH_AW_PROMPT_CONTENT_0000\"},{\"file\":\"xpia.md\"},{\"file\":\"temp_folder_prompt.md\"},{\"file\":\"markdown.md\"},{\"file\":\"playwright_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0001\"},{\"file\":\"github_mcp_tools_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0002\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0003\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0004\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0005\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0006\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0007\"}]}"
GH_AW_PROMPT_CONFIG: "{\"items\":[{\"content_env\":\"GH_AW_PROMPT_CONTENT_0000\"},{\"file\":\"xpia.md\"},{\"file\":\"temp_folder_prompt.md\"},{\"file\":\"markdown.md\"},{\"file\":\"playwright_prompt.md\"},{\"file\":\"playwright_awf_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0001\"},{\"file\":\"github_mcp_tools_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0002\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0003\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0004\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0005\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0006\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0007\"}]}"
GH_AW_EXPR_1A3A194A: ${{ github.event.discussion.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'discussion' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
GH_AW_EXPR_463A214A: ${{ github.event.pull_request.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'pull_request' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
GH_AW_EXPR_802A9F6A: ${{ github.event.issue.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'issue' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
Expand Down
12 changes: 12 additions & 0 deletions pkg/workflow/unified_prompt_step.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,18 @@ func (c *Compiler) collectPromptSections(data *WorkflowData) []PromptSection {
Content: playwrightPromptFile,
IsFile: true,
})

// 3b. AWF-aware Playwright policy (CLI mode + firewall/AWF sandbox enabled).
// This reinforces the secure loopback/proxy topology and takes precedence
// over generic Playwright CLI skill guidance such as runtime npm installs
// or navigation to arbitrary example domains.
if isPlaywrightCLIMode(data.Tools) && isFirewallEnabled(data) {
unifiedPromptLog.Print("Adding playwright AWF policy section")
sections = append(sections, PromptSection{
Content: playwrightAWFPromptFile,
IsFile: true,
})
}
}

// 4. Trial mode note (if in trial mode)
Expand Down
Loading