Format conformance: RFC 5321 address-literals, IDNA2008/Bidi idn-hostname rules, and the latest JSON-Schema-Test-Suite - #908
Merged
Conversation
…ormats idn-email had no address-literal handling at all: dotted-decimal literals were rejected by the letter-after-dot rule in MatchDecodedHostname, while bracketed IPv6 (and garbage like user@[] or user@[IPv6:zzz]) passed with no validation because MatchIdnHostname does not reject '[', ':' or ']'. RFC 6531 extends only the local part, so idn-email must accept exactly the address-literals email does. email validated IPv4 literals with the RFC 3986 dec-octet parser, which forbids the leading zeros that RFC 5321 Snum (1*3DIGIT, 0-255) permits. Extract a shared MatchAddressLiteral helper backed by a dedicated Snum matcher (decimal only, exactly four 1-3 digit octets, each <= 255), wire it into both MatchEmail and MatchIdnEmail, and never fall through to the IDN hostname path for a domain starting with '['. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MatchDecodedHostname only rejected characters via the disallowed-IDN rune list and the Unicode contextual rules, so ASCII symbols such as '[', ':', '_' and '!' fell through and were accepted. This made "[]" and "[IPv6:2001:db8::1]" validate as idn-hostname, and leaked through the punycode path of the plain hostname format (xn--4gbwdl.foo_bar validated even though foo_bar alone did not). Reject any ASCII rune that is not a letter, digit, '.' or '-' in decoded hostname labels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mats TypeIdnEmail rejected every address-literal because the domain is normalized with IdnMapping.GetAscii under UseStd3AsciiRules, which throws on '[' and ']'; the exception was caught and reported invalid. Branch bracketed domains to a new Validate.IsIdnEmailAddressLiteralMatch helper (shared by Validate and ValidateWithoutCoreType; the Warning variants delegate) that validates IPv4 literals against an RFC 5321 Snum pattern and IPv6 literals the same way the V4 ipv6 format does. EmailPattern validated IPv4 literals with RFC 3986 dec-octets, which forbid the leading zeros Snum permits; replace all twelve octet occurrences (six per pattern copy, including the IPv6-embedded IPv4 forms) with a Snum-based subpattern. Also align the GeneratedRegex copy of EmailPattern with the runtime fallback: it ended the TLD with [a-z0-9] instead of [a-zA-Z0-9], so user@EXAMPLE.COM was rejected on net8.0+ but accepted on net481. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ormance gaps Updating the submodule from 60755c1 (24 upstream commits, +441 lines of format tests across draft7/2019-09/2020-12) and regenerating the V5 test classes surfaced 65 failing test instances. All are fixed: - Implement the RFC 5893 Bidi rules in MatchDecodedHostname using the runtime-derived StrongBidiCategory data: per-label direction from the first character, no RTL characters in LTR labels and vice versa, and no mixing of European and Arabic-Indic digits, enforced only when the domain is a Bidi domain name. - Enforce the ZWNJ contextual rule (RFC 5892 Appendix A.1) at every occurrence: permitted after a virama or in an Arabic joining context; previously ZWNJ was never checked at all. - Reject non-letter, non-digit, non-mark code points in U-labels except the RFC 5892 §2.6 PVALID exceptions and the contextual code points already given their own rules. This catches A-labels that decode to disallowed code points such as xn--7a (U+00A1). - Reject U-labels whose A-label form exceeds 63 octets, counting RFC 3492 Punycode output octets without producing the encoding. - Fix the idn-hostname total-length limit off-by-one (254 -> 253). - Accept RFC 6531 quoted local parts containing non-ASCII in idn-email. - Reject trailing content after a complete UUID: Utf8Parser.TryParse consumed 36 bytes and the discarded bytesConsumed let suffixes pass. - Percent-encode absolute keyword location fragments (RFC 3986) in JsonSchemaAnnotationProducer, e.g. #/patternProperties/%5Ea. The suite has expected the encoded form since April 2025; the committed annotation classes were stale and masked the mismatch until this regeneration. Both suite projects are fully green (7891 + 8105), and the whole solution passes 94049 tests with a warning-free build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Contributor
Code Coverage Summary Report - Windows (net10.0)Summary
CoverageCorvus.Json.CodeGeneration - 80.5%
Corvus.Json.CodeGeneration.CSharp - 81.7%
Corvus.Json.ExtendedTypes - 74.9%
Corvus.Json.JsonReference - 76.7%
Corvus.Text.Json - 94.4%
Corvus.Text.Json.AsyncApi - 64.4%
Corvus.Text.Json.AsyncApi.Amqp - 0.6%
Corvus.Text.Json.AsyncApi.AzureServiceBus - 0%
Corvus.Text.Json.AsyncApi.CodeGeneration - 93.2%
Corvus.Text.Json.AsyncApi.Kafka - 0.6%
Corvus.Text.Json.AsyncApi.Mqtt - 0.9%
Corvus.Text.Json.AsyncApi.Nats - 0.4%
Corvus.Text.Json.AsyncApi.Polly - 100%
Corvus.Text.Json.AsyncApi.Testing - 96.6%
Corvus.Text.Json.AsyncApi.WebSocket - 0%
Corvus.Text.Json.CodeGeneration - 93.2%
Corvus.Text.Json.JMESPath - 93.8%
Corvus.Text.Json.JMESPath.CodeGeneration - 97.7%
Corvus.Text.Json.Jsonata - 91.6%
Corvus.Text.Json.Jsonata.CodeGeneration - 88.5%
Corvus.Text.Json.JsonLogic - 95.1%
Corvus.Text.Json.JsonLogic.CodeGeneration - 92.8%
Corvus.Text.Json.JsonPath - 95.2%
Corvus.Text.Json.JsonPath.CodeGeneration - 96.1%
Corvus.Text.Json.OpenApi - 95.5%
|
Contributor
Code Coverage Summary Report - Linux (net10.0)Summary
CoverageCorvus.Json.CodeGeneration - 80.5%
Corvus.Json.CodeGeneration.CSharp - 81.7%
Corvus.Json.ExtendedTypes - 74.9%
Corvus.Json.JsonReference - 76.7%
Corvus.Text.Json - 94.5%
Corvus.Text.Json.AsyncApi - 64.4%
Corvus.Text.Json.AsyncApi.Amqp - 0.6%
Corvus.Text.Json.AsyncApi.AzureServiceBus - 0%
Corvus.Text.Json.AsyncApi.CodeGeneration - 93.2%
Corvus.Text.Json.AsyncApi.Kafka - 0.6%
Corvus.Text.Json.AsyncApi.Mqtt - 0.9%
Corvus.Text.Json.AsyncApi.Nats - 0.4%
Corvus.Text.Json.AsyncApi.Polly - 100%
Corvus.Text.Json.AsyncApi.Testing - 96.6%
Corvus.Text.Json.AsyncApi.WebSocket - 0%
Corvus.Text.Json.CodeGeneration - 93.2%
Corvus.Text.Json.JMESPath - 93.8%
Corvus.Text.Json.JMESPath.CodeGeneration - 97.7%
Corvus.Text.Json.Jsonata - 91.6%
Corvus.Text.Json.Jsonata.CodeGeneration - 88.5%
Corvus.Text.Json.JsonLogic - 95.1%
Corvus.Text.Json.JsonLogic.CodeGeneration - 92.8%
Corvus.Text.Json.JsonPath - 95.2%
Corvus.Text.Json.JsonPath.CodeGeneration - 96.1%
Corvus.Text.Json.OpenApi - 95.5%
Corvus.Text.Json.OpenApi.CodeGeneration - 98.5%
|
Contributor
Code Coverage Summary Report - Windows (net481)Summary
CoverageCorvus.Json.CodeGeneration - 80.6%
Corvus.Json.CodeGeneration.CSharp - 81.4%
Corvus.Json.ExtendedTypes - 72.6%
Corvus.Json.JsonReference - 73.8%
Corvus.Text.Json - 93.2%
Corvus.Text.Json.CodeGeneration - 86.2%
Corvus.Text.Json.JMESPath - 93.8%
Corvus.Text.Json.JMESPath.CodeGeneration - 97.7%
Corvus.Text.Json.Jsonata - 91.8%
Corvus.Text.Json.Jsonata.CodeGeneration - 88.6%
Corvus.Text.Json.JsonLogic - 95.1%
Corvus.Text.Json.JsonLogic.CodeGeneration - 92.8%
Corvus.Text.Json.JsonPath - 95.2%
Corvus.Text.Json.JsonPath.CodeGeneration - 96.1%
Corvus.Text.Json.Patch - 97.5%
Corvus.Text.Json.Toon - 88.2%
Corvus.Text.Json.Validator - 95.5%
Corvus.Text.Json.Yaml - 90.8%
Corvus.Toon.SystemTextJson - 90%
Corvus.Yaml.SystemTextJson - 88.7%
|
Contributor
|
This was referenced Aug 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #904. Fixes #907.
Summary
A format-conformance release across five commits:
emailandidn-email(V5) —idn-emailhad no address-literal handling (valid IPv4/IPv6 literal domains rejected, bracketed garbage accepted);emailvalidated IPv4 literals with the RFC 3986 dec-octet grammar instead of RFC 5321 Snum (leading zeros). Both now share an address-literal validator backed by a Snum matcher."[]"validated asidn-hostname, and the punycode path ofhostnameleaked symbols (xn--4gbwdl.foo_bar). Decoded labels now reject any ASCII character that is not a letter, digit,.or-.TypeIdnEmailrejected all literals (STD3 IdnMapping throws on brackets),EmailPatternoctets were dec-octet based, and the[GeneratedRegex]copy rejected uppercase-ending TLDs on net8.0+ only (divergence from the net481 fallback).be54236+ conformance fixes — 24 upstream commits of new format tests surfaced 65 failing test instances, all fixed: RFC 5893 Bidi rules, the ZWNJ contextual rule (previously entirely unchecked), the U-label→A-label 63-octet limit via an allocation-free RFC 3492 length counter, the 253-length off-by-one, disallowed decoded code points (xn--7a), RFC 6531 quoted non-ASCII local parts, UUID trailing content, and RFC 3986 percent-encoding of annotation schema locations (a pre-existing mismatch masked by stale generated classes).Verification
JsonSchemaMatchingStringTests(V5) andFormatValidationTests(V4), each confirmed failing before its fix and passing after.TreatWarningsAsErrors).corvusjsonCLI: all four idn-email format does not validate RFC 5321 address-literals; email rejects leading-zero Snum octets #904 idn-email cases plus negative probes produce the expected verdicts.🤖 Generated with Claude Code
https://claude.ai/code/session_013LfPNoJeAg1RgBKL6xZgQc