rlp: backport RawList[T] lazy decoding primitives (upstream #33755, #33834, #33840, #33841) by chris-mercer · Pull Request #36 · ethereumclassic/core-geth

chris-mercer · 2026-03-21T06:51:47Z

RLP: Backport lazy decoding primitives for future p2p refactor

Backports upstream RLP lazy decoding types (rlp.RawList[T], iterator improvements) that enable a future full backport of delayed p2p message decoding from go-ethereum PR #33835.

These primitives allow RLP lists to be decoded lazily — the list content is stored as raw bytes and individual elements are decoded on demand. This is the foundation upstream uses for their full CVE-2026-26313 fix (delayed message decoding), where BlockBody fields become rlp.RawList[*Transaction] instead of []*Transaction.

Changes

Commit 1 — rlp.RawList[T] (upstream #33755)

rlp/raw.go: Add RawList[T] generic type that stores raw RLP bytes and decodes elements lazily
rlp/encode.go: Add RawList encoding support
rlp/iterator.go: Add NewListIterator for iterating over raw RLP lists
rlp/raw_test.go: Tests for RawList encode/decode round-trip
trie/node.go: Update rawNode to use rlp.RawValue (required by refactor)

Commit 2 — AppendRaw (upstream #33834)

rlp/raw.go: Add AppendRaw method for building RawList incrementally

Commit 3 — Element count validation (upstream #33840)

rlp/raw.go: Validate and cache element count in RawList
rlp/iterator.go: Update iterator to use cached count
rlp/raw_test.go: Tests for count validation

Commit 4 — Iterator.Count fix (upstream #33841)

rlp/iterator.go: Add back Iterator.Count with fixes for edge cases
rlp/iterator_test.go: Tests for count behavior

Commit 5 — Go 1.26 compatibility

rlp/iterator.go, rlp/raw.go: Fix return types for Go 1.26 iterator semantics (value vs pointer)

Why this is separate from the CVE fix

PR #35 fixes CVE-2026-26313 with a minimal pre-decode item count validation approach that works on Go 1.21 with zero dependencies. This PR provides the infrastructure for a future full backport of upstream's delayed decoding architecture, which requires Go generics (Go 1.26+).

Verification

go build ./rlp/...              # ✅
go vet ./rlp/...                # ✅
go test ./rlp/... -count=1      # ✅ all tests pass

References

upstream #33755 — rlp.RawList initial implementation
upstream #33834 — AppendRaw method
upstream #33840 — element count validation
upstream #33841 — Iterator.Count fix
upstream #33835 — full delayed decoding refactor (future backport target)

Road to Olympia — Core-Geth Modernization March

Developed by White B0x Inc. for Ethereum Classic DAO LLC

🤖 Generated with Claude Code

Merge Order

Dependencies: PR security: fixes EOL runtime vulnerabilities. Go 1.21 → 1.26, blst v0.3.11 → v0.3.16 #10 (Go 1.26 upgrade — required for Go generics)
Part of: Core-Geth Modernization March — Wave 1 (Security infrastructure)

Go 1.21 reached end-of-life in August 2024 with no security patches. This is the primary ETC client — cannot ship on EOL toolchain. Changes: - go.mod: Go 1.21 → 1.24 - Dockerfiles: golang:1.22-alpine → golang:1.24-alpine - CI workflows: all 8 workflow files updated to Go 1.24 - Remove fjl/memsize dependency (incompatible with Go 1.23+ due to runtime.stopTheWorld linkname restriction, no upstream fix available) - Fix Go 1.24 non-constant format string vet errors in 7 test files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

blst v0.3.11 used `typedef _Bool bool` which C23-aware GCC in golang:1.24-alpine rejects (bool is a keyword in C23). v0.3.14 includes explicit C23 harmonization and Go 1.24 support. Also added -std=gnu11 to gotool.go CGO_CFLAGS as defense-in-depth against future C23 typedef issues from any CGO dependency. The Dockerfile ENV approach was ineffective because gotool.go unconditionally overrides CGO_CFLAGS. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Go 1.24 reached end-of-life on 2026-02-11. Go 1.26.1 is the current stable release. The x/tools package required updating for Go 1.26 compatibility, which cascaded to x/crypto, x/net, x/sys, x/sync, x/text, and x/mod. Changes: - go.mod: Go 1.24 → 1.26 - golang.org/x/tools v0.21 → v0.43 (Go 1.26 compat) - golang.org/x/crypto v0.31 → v0.49 (cascading dep) - golang.org/x/net v0.33 → v0.52 (cascading dep) - golang.org/x/sys v0.28 → v0.42 (cascading dep) - golang.org/x/sync v0.10 → v0.20 (cascading dep) - golang.org/x/text v0.21 → v0.35 (cascading dep) - golang.org/x/mod v0.17 → v0.34 (cascading dep) - All 8 CI workflows: go-version 1.24 → 1.26 - Both Dockerfiles: golang:1.24-alpine → golang:1.26-alpine Verified: go build ./..., go vet ./..., go mod tidy (clean) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Latest stable release (Sep 2024). Includes lifted limitation for 384-bit modular inversion and hardened operations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Brings x/time to current release. Used in p2p/dnsdisc rate limiting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

This adds a new type wrapper that decodes as a list, but does not actually decode the contents of the list. The type parameter exists as a marker, and enables decoding the elements lazily. RawList can also be used for building a list incrementally.

This is helpful when building a list from already-encoded elements.

This changes `RawList` to ensure the count of items is always valid. Lists with invalid structure, i.e. ones where an element exceeds the size of the container, are now detected during decoding of the `RawList` and thus cannot exist. Also remove `RawList.Empty` since it is now fully redundant, and `Iterator.Count` since it returns incorrect results in the presence of invalid input. There are no callers of these methods (yet).

I removed `Iterator.Count` in #33840, because it appeared to be unused and did not provide the documented invariant: the returned count should always be an upper bound on the number of iterations allowed by `Next`. In order to make `Count` work, the semantics of `CountValues` has to change to return the number of items up and including the invalid one. I have reviewed all callsites of `CountValues` to assess if changing this is safe. There aren't that many, and the only call that doesn't check the error and return is in the trie node parser, `trie.decodeNodeUnsafe`. There, we distinguish the node type based on the number of items, and it previously returned an error for item count zero. In order to avoid any potential issue that could result from this change, I'm adding an error check in that function, though it isn't necessary.

Update NewListIterator to return Iterator by value instead of pointer, and newIterator to accept initialOffset parameter. Aligns with upstream geth commit ac85a6f25 changes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Backports upstream RLP “lazy decoding” primitives (notably rlp.RawList[T]) and iterator improvements to enable future delayed p2p message decoding refactors, alongside the Go 1.26 toolchain bump and related dependency/build cleanups.

Changes:

Add generic rlp.RawList[T] with lazy element decoding, append support, and encoding helpers.
Rework rlp.Iterator/NewListIterator, add Count/Offset behavior and additional iterator tests/fuzzing.
Bump repo to Go 1.26 and update CI/Docker/deps; remove memsize integration and fix a handful of vet-format issues.

Reviewed changes

Copilot reviewed 26 out of 27 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`trie/node.go`	Handle `CountValues` errors when decoding trie nodes.
`rlp/raw.go`	Introduce `RawList[T]`; adjust `CountValues` error-count semantics.
`rlp/encode.go`	Add `EncodeToRawList[T]` helper to produce `RawList[T]`.
`rlp/iterator.go`	Replace old list iterator with `Iterator` value type; add `Count`/`Offset`.
`rlp/raw_test.go`	Add extensive `RawList` tests; update `CountValues` expectations for error cases.
`rlp/iterator_test.go`	Add iterator offset/count tests, error-case tests, and fuzzing for `Count`.
`metrics/json_test.go`	Fix `t.Fatalf` formatting to satisfy vet.
`core/rawdb/accessors_chain_test.go`	Fix `t.Fatalf` formatting to satisfy vet.
`cmd/geth/logging_test.go`	Fix `t.Logf` formatting to satisfy vet.
`cmd/evm/t8n_test.go`	Fix `t.Logf` formatting to satisfy vet.
`accounts/abi/unpack_test.go`	Fix `t.Fatalf` formatting bug in error path.
`cmd/rlpdump/main.go`	Fix `Fprintf` formatting to satisfy vet.
`internal/debug/flags.go`	Remove memsize UI handler wiring and import.
`cmd/geth/main.go`	Remove memsize tracking registration.
`internal/build/gotool.go`	Extend `CGO_CFLAGS` for portability and C compiler compatibility.
`go.mod`	Bump Go version to 1.26; update deps (remove memsize, update blst/x/*).
`go.sum`	Sync checksums with updated module versions.
`Dockerfile`	Use `golang:1.26-alpine` builder image.
`Dockerfile.alltools`	Use `golang:1.26-alpine` builder image.
`.github/workflows/test-linux.yml`	Run CI jobs with Go 1.26.
`.github/workflows/release-packages.yml`	Run release packaging with Go 1.26.
`.github/workflows/go-generate-check.yml`	Run generate check with Go 1.26.
`.github/workflows/evmc.yml`	Run EVMC workflow with Go 1.26.
`.github/workflows/bench-vm.yml`	Run VM benchmarks with Go 1.26.
`.github/workflows/bench-trie.yml`	Run trie benchmarks with Go 1.26.
`.github/workflows/bench-core.yml`	Run core benchmarks with Go 1.26.
`.github/workflows/audit-bootnodes.yml`	Run bootnode audit with Go 1.26.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- RawList.Items: check it.Err() inside the loop since Next() returns true on parse errors (with Value set to nil). Without this check, DecodeBytes would receive nil data after a parse error. - CountValues: document that on error it returns the count of values successfully parsed so far. - Iterator.Err: fix doc comment — Next returns true (not false) on the error step; subsequent calls return false. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

chris-mercer and others added 10 commits March 20, 2026 12:06

build: upgrade blst v0.3.14 → v0.3.16

61557db

Latest stable release (Sep 2024). Includes lifted limitation for 384-bit modular inversion and hardened operations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

deps: upgrade golang.org/x/time v0.3.0 → v0.15.0

cf3e9bd

Brings x/time to current release. Used in p2p/dnsdisc rate limiting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

rlp: add AppendRaw method to RawList (#33834)

6fc23d5

This is helpful when building a list from already-encoded elements.

rlp: fix Iterator return types for Go 1.26 compatibility

a3b0672

Update NewListIterator to return Iterator by value instead of pointer, and newIterator to accept initialOffset parameter. Aligns with upstream geth commit ac85a6f25 changes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

chris-mercer requested a review from Copilot March 21, 2026 07:42

Copilot started reviewing on behalf of chris-mercer March 21, 2026 07:42 View session

Copilot AI reviewed Mar 21, 2026

View reviewed changes

Comment thread rlp/raw.go

Comment thread rlp/raw.go

Comment thread rlp/iterator.go Outdated

chris-mercer mentioned this pull request Mar 21, 2026

security: fixes p2p memory DoS via crafted response messages (CVE-2026-26313) #35

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rlp: backport RawList[T] lazy decoding primitives (upstream #33755, #33834, #33840, #33841)#36

rlp: backport RawList[T] lazy decoding primitives (upstream #33755, #33834, #33840, #33841)#36
chris-mercer wants to merge 11 commits into
ethereumclassic:masterfrom
chris-mercer:rlp/rawlist-backport

chris-mercer commented Mar 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

chris-mercer commented Mar 21, 2026

RLP: Backport lazy decoding primitives for future p2p refactor

Changes

Why this is separate from the CVE fix

Verification

References

Merge Order

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants