Skip to content

docs: comprehensive caching and resolve package documentation #1433

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jensneuse merged 3 commits into from
Mar 6, 2026
Merged

docs: comprehensive caching and resolve package documentation #1433

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7761077
docs: comprehensive caching and resolve package documentation
jensneuse Mar 6, 2026
894480c
merge: resolve conflict keeping execution/engine/CLAUDE.md
jensneuse Mar 6, 2026
70ed709
docs: address review feedback — add fence language tags, fix type ass…
jensneuse Mar 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
368 changes: 61 additions & 307 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,313 +1,67 @@
# Entity Caching Reference
# graphql-go-tools

GraphQL Federation entity caching system with L1 (per-request) and L2 (external) caches.
GraphQL Router / API Gateway framework for Go. Federation-first, with query planning, parallel resolution, and entity caching.

## Architecture Overview
Module: `github.com/wundergraph/graphql-go-tools` (Go 1.25, go.work workspace)

| Cache | Storage | Scope | Key Fields | Thread Safety |
|-------|---------|-------|------------|---------------|
| **L1** | `sync.Map` in Loader | Single request | `@key` only | sync.Map |
| **L2** | External (LoaderCache) | Cross-request | `@key` only | Atomic stats |
## Data Flow

**Key Principle**: Both L1 and L2 use only `@key` fields for stable entity identity.

## Key Files

| File | Purpose |
|------|---------|
| `v2/pkg/engine/resolve/loader.go` | L1/L2 cache core: `prepareCacheKeys`, `tryL1CacheLoad`, `tryL2CacheLoad`, `populateL1Cache` |
| `v2/pkg/engine/resolve/loader_json_copy.go` | Shallow copy for self-referential entities |
| `v2/pkg/engine/resolve/caching.go` | `RenderCacheKeys`, `EntityQueryCacheKeyTemplate`, `RootQueryCacheKeyTemplate` |
| `v2/pkg/engine/resolve/context.go` | `CachingOptions`, `CacheStats`, tracking methods |
| `v2/pkg/engine/resolve/fetch.go` | `FetchCacheConfiguration`, `FetchInfo.ProvidesData` |
| `v2/pkg/engine/plan/visitor.go` | `configureFetchCaching()`, `isEntityBoundaryField` |
| `v2/pkg/engine/plan/federation_metadata.go` | `EntityCacheConfiguration`, `RootFieldCacheConfiguration` |
| `v2/pkg/engine/datasource/graphql_datasource/graphql_datasource.go` | `buildCacheKeyVariable()`, cache key template building |
| `execution/engine/config_factory_federation.go` | `SubgraphCachingConfig`, per-subgraph configuration |
| `execution/engine/federation_caching_test.go` | E2E caching tests |
| `v2/pkg/engine/resolve/l1_cache_test.go` | L1 cache unit tests |

## Core Types

### Cache Key Templates
```go
// Entity caching - same @key-only keys for both L1 and L2
type EntityQueryCacheKeyTemplate struct {
Keys *ResolvableObjectVariable // @key fields only (no @requires)
}
func (e *EntityQueryCacheKeyTemplate) RenderCacheKeys(a arena.Arena, ctx *Context, items []*astjson.Value, prefix string) ([]*CacheKey, error)

// Root field caching - same template for L1 and L2
type RootQueryCacheKeyTemplate struct {
RootFields []QueryField // TypeName + FieldName + Args
}
```

### Configuration Types
```go
// Per-subgraph caching config (explicit opt-in)
type SubgraphCachingConfig struct {
SubgraphName string
EntityCaching plan.EntityCacheConfigurations // For _entities queries
RootFieldCaching plan.RootFieldCacheConfigurations // For root queries
}

type EntityCacheConfiguration struct {
TypeName string // e.g., "User"
CacheName string
TTL time.Duration
IncludeSubgraphHeaderPrefix bool
}

type RootFieldCacheConfiguration struct {
TypeName string // e.g., "Query"
FieldName string // e.g., "topProducts"
CacheName string
TTL time.Duration
IncludeSubgraphHeaderPrefix bool
}
```

### Cache Stats (Thread Safety)
```go
type CacheStats struct {
L1Hits int64 // Main thread only (non-atomic)
L1Misses int64 // Main thread only (non-atomic)
L2Hits *atomic.Int64 // Goroutine-safe (atomic)
L2Misses *atomic.Int64 // Goroutine-safe (atomic)
}
```

## Enabling Caching

### Runtime Options
```go
ctx.ExecutionOptions.Caching = CachingOptions{
EnableL1Cache: true, // Per-request entity cache
EnableL2Cache: true, // External cache
}
```

### Per-Subgraph Configuration (L2 only)
```go
subgraphCachingConfigs := engine.SubgraphCachingConfigs{
{
SubgraphName: "products",
RootFieldCaching: plan.RootFieldCacheConfigurations{
{TypeName: "Query", FieldName: "topProducts", CacheName: "default", TTL: 30 * time.Second},
},
},
{
SubgraphName: "accounts",
EntityCaching: plan.EntityCacheConfigurations{
{TypeName: "User", CacheName: "default", TTL: 30 * time.Second},
},
},
}

opts := []engine.FederationEngineConfigFactoryOption{
engine.WithSubgraphEntityCachingConfigs(subgraphCachingConfigs),
}
```

## Cache Flow

### Sequential Execution (`tryCacheLoad`)
1. `prepareCacheKeys()` - Generate L1 and L2 cache keys
2. `tryL1CacheLoad()` - Check L1 (main thread)
3. `tryL2CacheLoad()` - Check L2 (main thread)
4. Fetch if needed, then `populateL1Cache()` and `updateL2Cache()`

### Parallel Execution (`resolveParallel`)
1. **Main thread**: `prepareCacheKeys()` + `tryL1CacheLoad()` for all nodes
2. **Goroutines**: `tryL2CacheLoad()` + fetch via `loadFetchL2Only()`
3. **Main thread**: Merge results, populate L1 cache

**Rationale**: L1 is cheap (in-memory), check on main thread to skip goroutine work early. L2/fetch are expensive, run in parallel.

## Self-Referential Entity Fix

**Problem**: When `User.friends` returns the same `User` entity, L1 cache causes pointer aliasing → stack overflow on merge.

**Solution**: `shallowCopyProvidedFields()` in `loader_json_copy.go` creates copies based on `ProvidesData` schema.

```go
// In tryL1CacheLoad:
ck.FromCache = l.shallowCopyProvidedFields(cachedValue, info.ProvidesData)
```

## ProvidesData and Validation

`FetchInfo.ProvidesData` describes what fields a fetch provides. Used by:
- `validateItemHasRequiredData()` - Check if cached entity is complete
- `shallowCopyProvidedFields()` - Copy only required fields

**Critical**: For nested entity fetches, `ProvidesData` must contain entity fields (`id`, `username`), NOT the parent field (`author`).

## configureFetchCaching Logic

```go
func configureFetchCaching(internal, external) FetchCacheConfiguration {
// 1. Always preserve CacheKeyTemplate for L1
result := FetchCacheConfiguration{CacheKeyTemplate: external.Caching.CacheKeyTemplate}

// 2. Check global disable
if v.Config.DisableEntityCaching { return result }

// 3. Determine fetch type FIRST
if external.RequiresEntityFetch || external.RequiresEntityBatchFetch {
// Entity fetch: all rootFields same type, use first
entityTypeName := internal.rootFields[0].TypeName
cacheConfig := fedConfig.EntityCacheConfig(entityTypeName)
} else {
// Root field fetch: need exactly 1 rootField
if len(internal.rootFields) != 1 { return result }
cacheConfig := fedConfig.RootFieldCacheConfig(rootField.TypeName, rootField.FieldName)
}
}
```

## Unit Testing

```go
// Standard test setup
ctrl := gomock.NewController(t)
defer ctrl.Finish()

ds := NewMockDataSource(ctrl)
ds.EXPECT().Load(gomock.Any(), gomock.Any(), gomock.Any()).
DoAndReturn(func(ctx context.Context, headers any, input []byte) ([]byte, error) {
return []byte(`{"data":{...}}`), nil
}).Times(1)

loader := &Loader{caches: map[string]LoaderCache{"default": cache}}

// REQUIRED: Disable singleFlight for unit tests
ctx := NewContext(context.Background())
ctx.ExecutionOptions.DisableSubgraphRequestDeduplication = true
ctx.ExecutionOptions.Caching = CachingOptions{EnableL1Cache: true, EnableL2Cache: true}

// REQUIRED: Always use arena
ar := arena.NewMonotonicArena(arena.WithMinBufferSize(1024))
resolvable := NewResolvable(ar, ResolvableOptions{})
resolvable.Init(ctx, nil, ast.OperationTypeQuery)

err := loader.LoadGraphQLResponseData(ctx, response, resolvable)
out := fastjsonext.PrintGraphQLResponse(resolvable.data, resolvable.errors)
```

### FakeLoaderCache
Test mock in `cache_load_test.go` with TTL support and operation logging.

### Assertions

**IMPORTANT**: Always use exact assertions in cache tests. Never use vague comparisons.

```go
// GOOD: Exact values - always preferred
assert.Equal(t, 3, hitCount, "should have exactly 3 L1 hits")
assert.Equal(t, int64(12), l1HitsInt, "should have exactly 12 L1 hits")
assert.Equal(t, 2, accountsCalls, "should call accounts subgraph exactly twice")

// BAD: Never use vague comparisons
assert.GreaterOrEqual(t, hitCount, 1) // DON'T DO THIS
assert.Greater(t, l1HitsInt, int64(0)) // DON'T DO THIS
assert.LessOrEqual(t, calls, 5) // DON'T DO THIS
```

Exact assertions catch regressions that vague assertions miss. If the expected value changes, update the test to reflect the new exact value.

### Snapshot Comments

**IMPORTANT**: Every event line in a `CacheAnalyticsSnapshot` assertion MUST have a brief comment explaining **why** that event occurred. Focus on causation, not field values.

```go
// GOOD: explains the "why"
L2Reads: []resolve.CacheKeyEvent{
{CacheKey: keyUser, Kind: resolve.CacheKeyMiss, ...}, // First request, L2 empty
{CacheKey: keyUser, Kind: resolve.CacheKeyHit, ...}, // Populated by Request 1
},

// BAD: restates the field value
{CacheKey: keyUser, Kind: resolve.CacheKeyMiss, ...}, // this is a miss
```

## Federation Test Setup

Test services: `accounts`, `products`, `reviews` in `execution/federationtesting/`

### Testing Entity Caching vs @provides
```graphql
type Review {
# @provides - gateway trusts subgraph, NO entity resolution
author: User! @provides(fields: "username")

# No @provides - gateway MUST resolve via _entities
# Use for testing L1/L2 caching
authorWithoutProvides: User!
}
```

### Run Tests
```bash
go test -run "TestL1Cache" ./v2/pkg/engine/resolve/... -v
go test -run "TestFederationCaching" ./execution/engine/... -v
go test -race ./execution/engine/... -v # Race detector
```

## astjson API Reference

```go
// Create values on arena
astjson.ObjectValue(arena)
astjson.ArrayValue(arena)
astjson.StringValue(arena, string)
astjson.StringValueBytes(arena, []byte)
astjson.NumberValue(arena, string)
astjson.TrueValue(arena)
astjson.FalseValue(arena)
astjson.NullValue // Global constant (not a function)

// Manipulate
value.Set(arena, key, val)
value.SetArrayItem(arena, idx, val)
value.Get(keys...)
value.GetArray()
value.GetStringBytes()
value.MarshalTo([]byte)
value.Type() // TypeNull, TypeTrue, TypeObject, etc.
```

## LoaderCache Interface

```go
type LoaderCache interface {
Get(ctx context.Context, keys []string) ([]*CacheEntry, error)
Set(ctx context.Context, entries []*CacheEntry, ttl time.Duration) error
Delete(ctx context.Context, keys []string) error
}

type CacheEntry struct {
Key string
Value []byte // JSON-encoded entity
}
```

## Always use exact assertions

Use `assert.Equal` with exact expected values. Never use `Contains`, `GreaterOrEqual`, `LessOrEqual`, or any vague comparison.
For objects or slices, always compare against a fully defined expected value, not just a subset.

```go
// CORRECT
assert.Equal(t, 3, len(log), "should have exactly 3 cache operations")
assert.Equal(t, 1, tracker.GetCount(host), "should call subgraph exactly once")
assert.Equal(t, int64(12), stats.L1Hits, "should have exactly 12 L1 hits")

// WRONG — hides regressions
assert.GreaterOrEqual(t, len(log), 1)
assert.Greater(t, stats.L1Hits, int64(0))
assert.Contains(t, log[0].Keys, expectedKey)
```text
parse → normalize → validate → plan → resolve → response
```

If the expected value changes due to a code change, update the test to the new exact value.
## Package Map

### Core (v2/pkg/)

| Package | Purpose |
|---------|---------|
| `ast` | GraphQL AST representation |
| `astparser` | GraphQL parser (schema + operations) |
| `astnormalization` | AST normalization passes |
| `astvalidation` | Schema and query validation |
| `astvisitor` | AST visitor pattern for tree walking |
| `astprinter` | AST to string serialization |
| `asttransform` | AST transformations |
| `astimport` | AST import/merge utilities |
| `fastjsonext` | JSON manipulation extensions (astjson API) |
| `federation` | Federation composition utilities |
| `errorcodes` | Error code definitions |

### Engine (v2/pkg/engine/)

| Package | Purpose |
|---------|---------|
| `plan` | Query planning, federation metadata, cache configuration types |
| **`resolve`** | **Resolution engine: fetching, caching, rendering** → see [resolve/CLAUDE.md](v2/pkg/engine/resolve/CLAUDE.md) |
| `datasource/graphql_datasource` | GraphQL subgraph datasource adapter |
| `postprocess` | Response post-processing passes (L1 cache optimization, fetch tree building) |

### Execution (execution/)

| Package | Purpose |
|---------|---------|
| `engine` | Federation engine config factory (`SubgraphCachingConfig`, `WithSubgraphEntityCachingConfigs`), E2E tests |
| `federationtesting` | Test federation services: accounts, products, reviews |
| `graphql` | GraphQL execution utilities |

## Key Architectural Decisions

- **Federation-first**: designed for federated GraphQL with entity resolution and `@key`/`@provides`/`@requires`
- **Arena-based allocation**: JSON values live on arena memory (no GC pressure), released per-request
- **Parallel resolution**: fetch tree with Sequence/Parallel nodes, 4-phase parallel execution with L1/L2 caching
- **Two-pass rendering**: pre-walk (validate, collect errors) + print-walk (render JSON)

## Entity Caching

Two-level entity caching system (L1 per-request + L2 external). See:
- [v2/pkg/engine/resolve/CLAUDE.md](v2/pkg/engine/resolve/CLAUDE.md) — full resolve package reference (resolution pipeline + caching internals)
- [ENTITY_CACHING_INTEGRATION.md](ENTITY_CACHING_INTEGRATION.md) — router integration guide (public APIs, configuration, examples)

## Testing Conventions

- **Exact assertions only**: use `assert.Equal` with exact expected values, never `GreaterOrEqual`, `Contains`, or vague comparisons
- **Snapshot comments**: every event line in `CacheAnalyticsSnapshot` assertions must explain **why** that event occurred
- **Cache log rule**: every `ClearLog()` must have `GetLog()` + assertions before the next `ClearLog()`
- **Federation test services**: `accounts`, `products`, `reviews` in `execution/federationtesting/`
- Run: `go test ./v2/pkg/engine/resolve/... -v` and `go test ./execution/engine/... -v`
Loading