fix(url_safety): allow NAT64 well-known prefix (RFC 6052)

[SergoVashakmadze]

Problem

is_safe_url fails closed on legitimate public URLs when the client machine is on a NAT64-aware network (dual-stack lite, 464XLAT cellular, or any IPv6-only with CLAT).

Python's IPv6Address.is_reserved returns True for the entire NAT64 Well-Known Prefix 64:ff9b::/96 (RFC 6052). The existing _is_blocked_ip check treats any reserved IPv6 as an SSRF target, so web_extract rejects those URLs with:

Blocked: URL targets a private or internal network address

But NAT64 addresses are the IPv6 representation of public IPv4 services reachable via an upstream NAT64 translator — the embedded IPv4 address in the low 32 bits is public. Blocking them is a false positive.

How I hit it

Consumer dual-stack-lite ISP returns both a public IPv4 and a NAT64 synthesized IPv6 for any AAAA lookup. e.g. blogs.nvidia.com:

>>> socket.getaddrinfo("blogs.nvidia.com", None, socket.AF_UNSPEC, socket.SOCK_STREAM)
[(2,  ..., ('2.19.248.139', 0)),
 (10, ..., ('64:ff9b::213:f88b', 0, 0, 0))]

is_safe_url iterates addresses and bails on the first blocked one, so the NAT64 entry short-circuits the public IPv4.

Affects sec.gov, investor.nvidia.com, blogs.nvidia.com, Google/Cloudflare DNS, and countless others depending on network topology.

Fix

Explicit allow for 64:ff9b::/96 at the top of _is_blocked_ip, before the generic is_private / is_reserved check. Mirrors the existing _CGNAT_NETWORK explicit handling.

_NAT64_NETWORK = ipaddress.ip_network("64:ff9b::/96")

def _is_blocked_ip(ip):
    if isinstance(ip, ipaddress.IPv6Address) and ip in _NAT64_NETWORK:
        return False
    ...

Security posture is preserved: the NAT64 translator itself is a public gateway, so an attacker using a NAT64 IPv6 to reach an internal IPv4 would need to get Hermes onto the NAT64 operator's network AND configure their private IPv4 into the NAT64 mapping — infeasible in the threat model the SSRF check is defending against. (Truly internal IPv4s behind CGNAT/VPN are separately blocked via _CGNAT_NETWORK and the standard private ranges.)

Tests

All 61 tests in tests/tools/test_url_safety.py pass, including new coverage:

• TestIsBlockedIp.test_allowed_ips — 5 new parametrized IPs across the 64:ff9b::/96 range (nvidia, Google DNS, Cloudflare, boundary cases)
• test_nat64_url_allowed_end_to_end — dual-stack hostname returning both a public IPv4 and a NAT64 IPv6
• test_nat64_only_resolution_allowed — IPv6-only resolution returning only a NAT64 address (DS-Lite scenario)

Ran with:

python -m pytest tests/tools/test_url_safety.py -x
# 61 passed in 1.33s

Context

Found while building a Hermes Agent hackathon submission — web_extract kept erroring on legitimate public URLs because the dev machine is on a DS-Lite home ISP. Spent a bit figuring out this wasn't a Firecrawl auth / quota issue. Hopefully saves someone else the debugging.

Repo of the project that surfaced this: https://github.com/SergoVashakmadze/IoMarkets-Hermes (Hermes Creative Hackathon 2026).

[Bartok9]

Salvaged onto current main as #51758 with credit to you, @SergoVashakmadze. The NAT64 false-positive block is still live on main. One hardening change worth flagging: your version returns False for the entire 64:ff9b::/96 prefix, but the low 32 bits are attacker-controllable, so 64:ff9b::169.254.169.254 (cloud metadata) / 64:ff9b::10.0.0.1 (RFC1918) would have been allowed — a fail-open SSRF bypass. The salvage instead extracts the embedded IPv4 and re-checks it against the IPv4 policy (mirroring the existing ::ffff: handling), so public targets pass while private/metadata ones stay blocked. Happy to defer if you would rather refresh this yourself. 🙏

[SergoVashakmadze]

Superseded by #51758, which re-targets this onto current main and closes a fail-open SSRF gap in my original: I returned False for the entire 64:ff9b::/96 prefix, but the embedded low-32-bit IPv4 is attacker-controllable, so metadata/RFC1918/loopback targets would have been allowed. The salvage re-checks the embedded IPv4 against the v4 policy instead. Thanks @Bartok9 for the salvage and the credit — closing in favor of #51758.