fix(sdk): redact plugin and extension source credentials at the serialization boundary (#3752)

[haya-asif]

HUMAN:

This PR fixes a credential leak where plugin/extension source URLs containing embedded tokens (e.g.https://oauth2:TOKEN@host/repo.git) were written verbatim to persisted state (.installed.json, conversation state) and logs. The fix redacts them at the model serialization boundary while keeping the raw value in memory for fetching.

---

AGENT:

Why

Closes #3752. Credential-bearing plugin/extension source URLs (e.g.
https://oauth2:TOKEN@host/repo.git) leaked into persisted state
(.installed.json, meta.json, conversation state, the auto plugins tag) and
logs. The first attempt hand-rolled redaction and broke private-plugin clones;
this version routes the source field through the SDK's existing secret
convention instead.

Summary

• New serialize_credential_url / validate_credential_url in
  utils/pydantic_secrets.py (beside serialize_secret / validate_secret),
  branching on resolve_expose_mode: redact (default → mask only the
  userinfo, https://****@host), plaintext (trusted dumps → real URL),
  encrypted (a cipher → Fernet token, decrypted on load).
• PluginSource.source and InstallationInfo.source wire these via the
  standard field_validator(mode="before") + field_serializer(when_used="always") pair.
• ResolvedPluginSource keeps redacting at construction (unchanged from main).
• RemoteConversation sends plugins with expose_secrets=True; the auto
  plugins tag is redacted; .installed.json is persisted with an expose
  context so ExtensionManager.update() can re-clone.

Maps to the review:

• (1) agent-server clones private plugins — the server now receives the real
  source (and ${VAR}), and meta.json is encrypted at rest, not cleartext.
• (2) ExtensionManager.update() re-clones from .installed.json (round-trip preserved).
• (3) ResolvedPluginSource.source redacted in memory (attribute read is safe).

Issue Number

Closes #3752.

How to Test

uv run pytest tests/sdk/git/test_url_redaction.py tests/sdk/extensions/installation \
  tests/sdk/conversation/remote/test_remote_conversation.py \
  tests/workspace/test_cloud_workspace_automation_tags.py
uv run ruff check . && uv run pyright

New regression tests cover each path: default redaction, expose, the cipher
encrypt-at-rest round-trip on PluginSource and InstallationInfo (the
credential becomes a gAAAAA… token and decrypts back), the remote-send
payload, the plugins tag, and the .installed.json → update() re-clone.
Full tests/sdk + tests/agent_server green; ruff + pyright clean.

Type

• Bug fix
• Feature
• Refactor
• Breaking change
• Docs / chore

Notes

source uses a bespoke serialize_credential_url rather than SecretStr
because it is a polymorphic URL (github:owner/repo, local path, ${VAR})
needing structure-preserving (userinfo-only) redaction, not a full **********
mask. .installed.json stores the source in plaintext at rest (the extensions
layer has no cipher) — same posture as before this PR, and required for update().

[WaseemSabir]

lgtm!

[VascoSch92]

Hey @haya-asif

thanks for the PR.

However, there are a couple of things to fix.

Perhaps, take a moment to explore a little the repo and understand how secrets are handled.

---

Agent-server clones private plugins with https://****@… (first run and resume)

StoredConversation extends StartConversationRequest (plugins: list[PluginSource], request.py:171). The agent-server builds it with request.model_dump(mode="json", context={"expose_secrets": True}) (conversation_service.py:761), persists it via stored.model_dump_json() to meta.json (event_service.py:126), and later clones from stored.plugins (event_service.py:790 → _ensure_plugins_loaded).

The new serializer fires on every one of those dumps and ignores expose_secrets — the exact flag the server passes to keep values intact. Worse, it redacts ${MY_TOKEN} placeholders, which are not secrets (they're expanded from the secret registry at fetch time) and never leaked. So _expand_secret_refs has nothing to expand and git clones https://****@… → 403.

from openhands.sdk.plugin.types import PluginSource

  # ${MY_TOKEN} is the recommended way to clone a private plugin: it is NOT a
  # secret and is expanded from the secret registry at fetch time.
  src = "https://x-token-auth:${MY_TOKEN}@host/repo.git"
  
  # The agent-server builds StoredConversation via this exact call
  # (conversation_service.py:761) and later clones from stored.plugins.
  print(PluginSource(source=src).model_dump(mode="json", context={"expose_secrets": True})["source"])

  # PR #3786 → 'https://****@host/repo.git'   (placeholder gone → git clone 403s)
  # main     → 'https://x-token-auth:${MY_TOKEN}@host/repo.git'  (works)

---

ExtensionManager.update() re-clones private extensions with https://****@…

update() reloads the source from .installed.json and re-installs with it (manager.py:312-325): current_info = self.get(name) (which reads disk via InstallationMetadata.load_from_dir) then self.install(source=current_info.source, …).

After the PR, the persisted source is redacted, and extensions don't support ${VAR} expansion, so the re-fetch clones https://****@… and fails.

  from pathlib import Path
  from openhands.sdk.extensions.installation.info import InstallationInfo

  src = "https://oauth2:SUPER_SECRET@github.com/org/ext.git"
  info = InstallationInfo(name="ext", source=src, install_path=Path("/tmp/ext"))

  # This value is what lands in .installed.json, is reloaded by manager.get(),
  # and is passed straight to install() inside update() (manager.py:312-325).
  print(info.model_dump()["source"])
  
  # PR #3786 → 'https://****@github.com/org/ext.git'   (update() re-clones this → 403)
  # main     → 'https://oauth2:SUPER_SECRET@github.com/org/ext.git'  (works)

---

ResolvedPluginSource.source is now raw in memory → new attribute-access leak

the PR moved redaction from construction-time to the serializer, changing from_plugin_source from source=redact_url_credentials(plugin_source.source) to source=plugin_source.source.

The model now holds the raw URL and only redacts on model_dump. Any direct attribute read therefore leaks, exactly the f-string/attribute bypass the issue warned about.

The PR's audit missed examples/05_skills_and_plugins/02_loading_plugins/main.py:114, which prints resolved.source directly. On main this printed ****.

  from openhands.sdk.plugin.types import PluginSource, ResolvedPluginSource

  resolved = ResolvedPluginSource.from_plugin_source(
      PluginSource(source="https://oauth2:SUPER_SECRET@github.com/org/repo.git"),
      resolved_ref="deadbeef",  
  )

  # Any direct attribute read leaks. Real sink:
  # examples/05_skills_and_plugins/02_loading_plugins/main.py:114
  #   print(f"  - {resolved.source} @ {resolved.resolved_ref}")
  print(resolved.source)
  
  # PR #3786 → 'https://oauth2:SUPER_SECRET@github.com/org/repo.git'  (raw → leaked)
  # main     → 'https://****@github.com/org/repo.git'                 (redacted, safe)

[haya-asif]

Would appreciate another look @VascoSch92 — thanks for the detailed review.

[VascoSch92]

For me is (almost) good.

There are just two things that we should tests to be sure we don't have regression in the future:

1. ExtensionManager.update() re-clones a credentialed source. This is the exact regression-#2 path and the sole justification for keeping .installed.json plaintext. Right now nothing proves the credential actually reaches the clone; a silent break here reintroduces the private-extension failure.
2. Plugin-load log regression guard, issue #3752 explicitly required it, and a serializer can't catch f-string/attribute leaks. Without it, anyone re-adding spec.source to that log silently reopens the credential leak the PR exists to close.

Otherwise, @jpshackelford and @enyst WDYT?

[simonrosenberg]

Since the PR was stale I took the liberty of resolving the issue here #3786

Thanks @haya-asif for initiating the effort!