feat: p2p exchange wrapper

[tac0turtle]

Overview

This pr was an idea from @renaynay in which we create an exchange wrapper so that the exchange can check our store before making public requests for data. This would help the edge case i ran into today where the p2p store was missing 40 blocks randomly

[github-actions]

The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	⏩ skipped	✅ passed	✅ passed	Nov 20, 2025, 12:21 PM

[claude]

Claude finished @tac0turtle's task —— View job

---

Comprehensive Code Review ✅

I've completed a thorough review of the P2P exchange wrapper implementation. This is a well-designed solution to prevent unnecessary P2P network requests by checking the local DA store first. Here are my findings:

---

🎯 Overall Assessment

Strengths:

• Clean separation of concerns with the wrapper pattern
• Good test coverage (70.83% for new code)
• Proper use of Go generics for type safety
• Non-breaking changes to the API

Priority Issues:

1. ⚠️ Performance: Using GetBlockByHash instead of a header-only lookup for header sync (as noted by @julienrbrt)
2. ⚠️ Test Coverage: Missing coverage for critical error paths in sync_service.go
3. ⚠️ Error Handling: Silent error swallowing could mask issues

---

1. Code Quality & Architecture

✅ Positives:

• Wrapper Pattern: The exchangeWrapper cleanly implements the decorator pattern without modifying the underlying header.Exchange interface
• Dependency Injection: Function-based getters (storeGetter, storeGetterByHeight) provide excellent flexibility and testability
• Zero-value Safety: Proper checks for nil store and getters prevent panics

⚠️ Issues:

pkg/sync/sync_service.go:102-104 - Performance Issue (Critical)

getter := func(ctx context.Context, s store.Store, hash header.Hash) (*types.SignedHeader, error) {
    h, _, err := s.GetBlockByHash(ctx, hash)  // ← Fetches full block (header + data)
    return h, err
}

Problem: For header sync, this fetches the entire block (header + data) when only the header is needed.

Root Cause: The store.Store interface (pkg/store/types.go:49-71) provides:

• ✅ GetHeader(ctx, height) - Returns header only
• ✅ GetBlockByHash(ctx, hash) - Returns header + data
• ❌ Missing: GetHeaderByHash(ctx, hash) - Would return header only by hash

Recommendation - Option 1 (Preferred): Add GetHeaderByHash to store.Store interface:

// In pkg/store/types.go
type Reader interface {
    // ... existing methods ...
    
    // GetHeaderByHash returns the header with given block header hash, or error if it's not found in Store.
    GetHeaderByHash(ctx context.Context, hash []byte) (*types.SignedHeader, error)
}

// Then in pkg/sync/sync_service.go
getter := func(ctx context.Context, s store.Store, hash header.Hash) (*types.SignedHeader, error) {
    return s.GetHeaderByHash(ctx, hash)
}

Recommendation - Option 2 (Quick fix): Document current approach and accept the trade-off:

getter := func(ctx context.Context, s store.Store, hash header.Hash) (*types.SignedHeader, error) {
    // Note: GetBlockByHash fetches both header and data because store.Store
    // doesn't provide GetHeaderByHash. This is less efficient but ensures
    // correctness. Consider adding GetHeaderByHash in a future optimization.
    h, _, err := s.GetBlockByHash(ctx, hash)
    return h, err
}

Impact:

• I/O overhead: Reading block data from disk unnecessarily
• Memory allocation: Deserializing unused block data
• Cache pollution: Loading unused data into memory

---

pkg/sync/exchange_wrapper.go:20-29 - Silent Error Handling

func (ew *exchangeWrapper[H]) Get(ctx context.Context, hash header.Hash) (H, error) {
    if ew.daStore != nil && ew.getter != nil {
        if h, err := ew.getter(ctx, ew.daStore, hash); err == nil && !h.IsZero() {
            return h, nil
        }  // ← Error is silently ignored
    }
    return ew.Exchange.Get(ctx, hash)
}

Problem: Store errors are silently ignored, making debugging difficult if the store has issues.

Recommendation: Add optional logging (requires passing logger to wrapper):

func (ew *exchangeWrapper[H]) Get(ctx context.Context, hash header.Hash) (H, error) {
    if ew.daStore != nil && ew.getter != nil {
        if h, err := ew.getter(ctx, ew.daStore, hash); err == nil && !h.IsZero() {
            return h, nil
        } else if err != nil && ew.logger != nil {
            // Log at trace/debug level since fallback is normal behavior
            ew.logger.Trace().Err(err).Str("hash", hash.String()).Msg("DA store lookup failed, falling back to P2P")
        }
    }
    return ew.Exchange.Get(ctx, hash)
}

Note: This is optional - only add if observability becomes important.

---

2. Error Handling & Resilience

✅ Positives:

• Proper nil checks prevent panics
• Zero-value checks prevent returning empty headers
• Fallback to P2P exchange ensures availability

⚠️ Issues:

Ambiguous Error Scenarios: The current implementation treats all store errors the same:

• Store is unavailable (connectivity)
• Data not found (expected miss)
• Store corruption (serious issue)

Recommendation: Consider distinguishing between error types if store errors become problematic in production. This is a future optimization, not a blocking issue.

---

3. Security

✅ No Security Issues Found

• No credential handling
• No input validation issues (context and hash are safe types)
• No injection vulnerabilities
• Proper use of context for cancellation

---

4. Performance & Resource Efficiency

⚠️ Critical Issue: Header Fetch Efficiency

Problem: As mentioned in #1, the header sync getter fetches full blocks unnecessarily.

Impact:

• I/O overhead: Reading more data than needed from disk/DA layer
• Memory allocation: Deserializing full block data when only headers are needed
• Network efficiency: The whole point of this PR is to optimize P2P traffic, but we're still doing unnecessary I/O on the store side

Measurement Needed: Profile the difference between:

• Current: GetBlockByHash (header + data) - reads both from store
• Proposed: GetHeaderByHash (header only) - reads only header from store

For Data Sync: The current approach is correct because you need the full block:

// pkg/sync/sync_service.go:82-85 - This is CORRECT ✅
getter := func(ctx context.Context, s store.Store, hash header.Hash) (*types.Data, error) {
    _, d, err := s.GetBlockByHash(ctx, hash)  // Need full block for data sync
    return d, err
}

---

✅ Good Practices:

• No unnecessary goroutines created
• No blocking operations without context
• Efficient short-circuit evaluation (err == nil && !h.IsZero())

---

5. Testing & Reliability

✅ Positives:

• Excellent unit test coverage for the wrapper itself (exchange_wrapper_test.go)
• Tests cover hit/miss/unconfigured scenarios
• Proper use of mocks

⚠️ Issues:

Missing Test Coverage (from Codecov report):
The sync_service.go changes have 14 uncovered lines. Based on the diff, the likely uncovered areas are:

1. Error paths in store lookup (lines 82-108):

   • What happens when GetBlockByHash returns an error?
   • What happens when it returns a zero-value header?

2. Integration scenarios:

   • No test verifying the wrapper actually reduces P2P calls
   • No test for concurrent access patterns

Recommendations:

1. Add Integration Test:

func TestExchangeWrapper_ReducesP2PCalls(t *testing.T) {
    // Setup: Store with data at height 10
    store := setupStoreWithBlock(t, 10)
    mockExchange := extmocks.NewMockExchange[*types.SignedHeader](t)
    // Exchange should NOT be called when data is in store
    
    wrapper := &exchangeWrapper[*types.SignedHeader]{
        Exchange:       mockExchange,
        daStore:        store,
        getterByHeight: headerGetterByHeight,
    }
    
    // When: Request height 10 (in store)
    h, err := wrapper.GetByHeight(ctx, 10)
    // Then: Success, no P2P calls
    assert.NoError(t, err)
    assert.NotNil(t, h)
    mockExchange.AssertNotCalled(t, "GetByHeight")
    
    // When: Request height 11 (not in store)
    mockExchange.On("GetByHeight", ctx, uint64(11)).Return(header11, nil)
    h, err = wrapper.GetByHeight(ctx, 11)
    // Then: P2P exchange called exactly once
    assert.NoError(t, err)
    mockExchange.AssertNumberOfCalls(t, "GetByHeight", 1)
}

2. Add Concurrency Test:

func TestExchangeWrapper_ConcurrentAccess(t *testing.T) {
    wrapper := setupWrapper(t)
    var wg sync.WaitGroup
    for i := 0; i < 100; i++ {
        wg.Add(1)
        go func(height uint64) {
            defer wg.Done()
            _, _ = wrapper.GetByHeight(context.Background(), height)
        }(uint64(i % 10))
    }
    wg.Wait()
    // Verify no panics and reasonable behavior
}

---

Test Code Quality:

exchange_wrapper_test.go:19 - Improve test data:

expectedHeader := &types.SignedHeader{} // Just a dummy

Recommendation: Use more realistic test data to catch zero-value bugs:

expectedHeader := &types.SignedHeader{
    Header: types.Header{
        Height:  10,
        ChainID: "test-chain",
        // ... other required fields
    },
    Signature: []byte("test-sig"),
}

---

6. Documentation & Developer Experience

⚠️ Missing Documentation:

exchange_wrapper.go - Lacks package and type documentation:

Recommendations:

// exchangeWrapper wraps a header.Exchange to check the local DA store
// before making P2P network requests. This prevents redundant network
// traffic when data is already available locally.
//
// This wrapper addresses the issue where the P2P store may be missing
// blocks that exist in the DA layer, as observed in production where
// 40 blocks were randomly missing from the P2P store.
//
// The wrapper silently falls back to the underlying Exchange if:
//   - The DA store is not configured (nil)
//   - The store lookup fails (any error)
//   - The store returns a zero-value header
//
// Concurrency: Safe for concurrent use as it has no mutable state.
type exchangeWrapper[H header.Header[H]] struct {
    header.Exchange[H]
    daStore        store.Store
    getter         storeGetter[H]
    getterByHeight storeGetterByHeight[H]
}

---

sync_service.go:82-88 - Document the trade-off:

getter := func(ctx context.Context, s store.Store, hash header.Hash) (*types.Data, error) {
    // GetBlockByHash is used because the data sync needs both header and data.
    // We discard the header here as the caller only needs the data portion.
    _, d, err := s.GetBlockByHash(ctx, hash)
    return d, err
}

---

7. Concurrency Safety

✅ Positives:

• exchangeWrapper has no mutable state (only reads from store)
• Function-based getters are naturally concurrent-safe
• Proper use of atomic.Bool for storeInitialized in SyncService

ℹ️ Assumption:

• The implementation assumes store.Store is concurrent-safe (not verified in this PR but reasonable assumption for a store interface)

---

8. Code Style & Go Idioms

✅ Excellent:

• Proper use of generics with type constraints
• Idiomatic error handling
• Clean function signatures
• Good separation of concerns

ℹ️ Minor:

• Consistent naming conventions throughout
• Follows Go proverbs (e.g., "Accept interfaces, return concrete types" - not applicable here due to generics)

---

Summary & Recommendations

🔴 Must Fix (Blocking):

1. Performance: Address the GetBlockByHash vs header-only lookup issue for header sync (pkg/sync/sync_service.go:102-104)
   • Option A: Add GetHeaderByHash(ctx, hash) to store.Store interface (preferred)
   • Option B: Document the current trade-off and accept it (quick fix)

🟡 Should Fix (Strongly Recommended):

2. Test Coverage: Add integration test showing P2P call reduction and store cache hit
3. Documentation: Add godoc comments to exchangeWrapper type and key functions
4. Test Data: Use realistic test headers instead of zero-value structs

🟢 Nice to Have (Optional):

5. Add trace logging for store lookup failures (requires logger injection)
6. Add concurrency test for wrapper
7. Add benchmark comparing store vs P2P performance
8. Consider error type differentiation (store error vs not found)

---

Conclusion

This is a well-architected solution to a real production problem (40 missing blocks in P2P store). The wrapper pattern is clean, the dependency injection is flexible, and the fallback behavior is robust.

The main concern is the performance issue where header sync fetches full blocks instead of just headers. This defeats part of the optimization goal of this PR. I recommend either:

1. Adding GetHeaderByHash to the store interface (best solution), or
2. Documenting the current trade-off if adding the method is out of scope for this PR

Overall Recommendation: ✅ Approve after addressing the GetHeaderByHash performance issue

The code is production-ready once the performance concern is addressed. The missing test coverage and documentation are improvements but not blockers.

---

Additional Context from Review Comment:
@julienrbrt's comment about "GetHeader is less expensive" is spot-on and points to the core performance issue identified above.

---

[codecov]

Codecov Report

❌ Patch coverage is 70.83333% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.88%. Comparing base (9d4c64c) to head (210ffcc).

Files with missing lines	Patch %	Lines
pkg/sync/sync_service.go	56.25%	10 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2855      +/-   ##
==========================================
- Coverage   64.89%   64.88%   -0.02%     
==========================================
  Files          81       82       +1     
  Lines        7242     7271      +29     
==========================================
+ Hits         4700     4718      +18     
- Misses       1998     2007       +9     
- Partials      544      546       +2     

Flag	Coverage Δ
combined	64.88% <70.83%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.