Skip to content

Size the buffers of the Base64 OperationStatus overloads by the data - #697

Merged
SimonCropp merged 1 commit into
mainfrom
perf/base64-status-buffers
Oct 12, 2026
Merged

SimonCropp merged 1 commit into
mainfrom
perf/base64-status-buffers

Conversation

@SimonCropp

Copy link
Copy Markdown
Owner

The OperationStatus overloads of the Base64 polyfill go through a temporary UTF-8 buffer:

  • EncodeToChars(source, destination, out, out) allocated new byte[destination.Length], so encoding 32 bytes into a 64K char buffer allocated 65 KB
  • DecodeFromChars(source, destination, out, out) allocated new byte[source.Length]

Both now size the buffer by the data, use the stack up to 256 bytes and ArrayPool<byte>.Shared above that, so neither allocates.

For the encode the buffer is Math.Min(destination.Length, encoded length of source): Base64.EncodeToUtf8 never writes more than the encoded length of the source, so capping the buffer at that gives the same status, consumed and written counts as a buffer the size of the destination.

The loops live in separate helpers from the stackalloc: the JIT does not tier up a method that has both, which measured several times slower in a cold process.

Separate from #688 (which changes the other overloads); the two touch different members.

Tests

  • both overloads against Convert for every length from 0 to 400 (all remainders, stack and pooled buffers), with a destination one element larger than needed (sentinel checked) and one 5000 elements larger
  • destination too small by one for every length: status, consumed, written and the partial output
  • not-final blocks for a short and a long (pooled) input: partial trailing group, destination too small, exact fit
  • empty destination, empty source
  • invalid and non-ASCII characters after a short and a long valid prefix

The new tests pass against the previous implementation and against the real net11 Base64. On net4x System.Memory reports NeedMoreData for a not-final block with nothing left over, so that one status is checked as Done or NeedMoreData.

Measurements

Stopwatch / GC.GetAllocatedBytesForCurrentThread, best of 9 rounds after warm-up, Release.

net10.0

before after
EncodeToChars(32B, 64K dest, out, out) 1,994 ns / 65,560 B 25 ns / 0 B
EncodeToChars(1KB, 64K dest, out, out) 2,448 ns / 65,560 B 574 ns / 0 B
DecodeFromChars(32B, dest, out, out) 34 ns / 72 B 28 ns / 0 B
DecodeFromChars(1KB, dest, out, out) 661 ns / 1,392 B 444 ns / 0 B

net48

before after
EncodeToChars(32B, 64K dest, out, out) 2,953 ns / 65,560 B 112 ns / 0 B
EncodeToChars(1KB, 64K dest, out, out) 5,006 ns / 65,560 B 1,821 ns / 0 B
DecodeFromChars(32B, dest, out, out) 122 ns / 72 B 144 ns / 0 B
DecodeFromChars(1KB, dest, out, out) 1,959 ns / 1,392 B 1,987 ns / 0 B

(The machine was busy, so differences of a few tens of ns are noise.)

@SimonCropp SimonCropp modified the milestones: 11.4.4, 11.4.5 Oct 12, 2026
@SimonCropp
SimonCropp merged commit 650bbf2 into main Oct 12, 2026
7 checks passed
@SimonCropp
SimonCropp deleted the perf/base64-status-buffers branch October 12, 2026 12:08
This was referenced Oct 12, 2026
This was referenced Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant