-
Notifications
You must be signed in to change notification settings - Fork 162
docs: comprehensive caching and resolve package documentation #1433
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,313 +1,67 @@ | ||
| # Entity Caching Reference | ||
| # graphql-go-tools | ||
|
|
||
| GraphQL Federation entity caching system with L1 (per-request) and L2 (external) caches. | ||
| GraphQL Router / API Gateway framework for Go. Federation-first, with query planning, parallel resolution, and entity caching. | ||
|
|
||
| ## Architecture Overview | ||
| Module: `github.com/wundergraph/graphql-go-tools` (Go 1.25, go.work workspace) | ||
|
|
||
| | Cache | Storage | Scope | Key Fields | Thread Safety | | ||
| |-------|---------|-------|------------|---------------| | ||
| | **L1** | `sync.Map` in Loader | Single request | `@key` only | sync.Map | | ||
| | **L2** | External (LoaderCache) | Cross-request | `@key` only | Atomic stats | | ||
| ## Data Flow | ||
|
|
||
| **Key Principle**: Both L1 and L2 use only `@key` fields for stable entity identity. | ||
|
|
||
| ## Key Files | ||
|
|
||
| | File | Purpose | | ||
| |------|---------| | ||
| | `v2/pkg/engine/resolve/loader.go` | L1/L2 cache core: `prepareCacheKeys`, `tryL1CacheLoad`, `tryL2CacheLoad`, `populateL1Cache` | | ||
| | `v2/pkg/engine/resolve/loader_json_copy.go` | Shallow copy for self-referential entities | | ||
| | `v2/pkg/engine/resolve/caching.go` | `RenderCacheKeys`, `EntityQueryCacheKeyTemplate`, `RootQueryCacheKeyTemplate` | | ||
| | `v2/pkg/engine/resolve/context.go` | `CachingOptions`, `CacheStats`, tracking methods | | ||
| | `v2/pkg/engine/resolve/fetch.go` | `FetchCacheConfiguration`, `FetchInfo.ProvidesData` | | ||
| | `v2/pkg/engine/plan/visitor.go` | `configureFetchCaching()`, `isEntityBoundaryField` | | ||
| | `v2/pkg/engine/plan/federation_metadata.go` | `EntityCacheConfiguration`, `RootFieldCacheConfiguration` | | ||
| | `v2/pkg/engine/datasource/graphql_datasource/graphql_datasource.go` | `buildCacheKeyVariable()`, cache key template building | | ||
| | `execution/engine/config_factory_federation.go` | `SubgraphCachingConfig`, per-subgraph configuration | | ||
| | `execution/engine/federation_caching_test.go` | E2E caching tests | | ||
| | `v2/pkg/engine/resolve/l1_cache_test.go` | L1 cache unit tests | | ||
|
|
||
| ## Core Types | ||
|
|
||
| ### Cache Key Templates | ||
| ```go | ||
| // Entity caching - same @key-only keys for both L1 and L2 | ||
| type EntityQueryCacheKeyTemplate struct { | ||
| Keys *ResolvableObjectVariable // @key fields only (no @requires) | ||
| } | ||
| func (e *EntityQueryCacheKeyTemplate) RenderCacheKeys(a arena.Arena, ctx *Context, items []*astjson.Value, prefix string) ([]*CacheKey, error) | ||
|
|
||
| // Root field caching - same template for L1 and L2 | ||
| type RootQueryCacheKeyTemplate struct { | ||
| RootFields []QueryField // TypeName + FieldName + Args | ||
| } | ||
| ``` | ||
|
|
||
| ### Configuration Types | ||
| ```go | ||
| // Per-subgraph caching config (explicit opt-in) | ||
| type SubgraphCachingConfig struct { | ||
| SubgraphName string | ||
| EntityCaching plan.EntityCacheConfigurations // For _entities queries | ||
| RootFieldCaching plan.RootFieldCacheConfigurations // For root queries | ||
| } | ||
|
|
||
| type EntityCacheConfiguration struct { | ||
| TypeName string // e.g., "User" | ||
| CacheName string | ||
| TTL time.Duration | ||
| IncludeSubgraphHeaderPrefix bool | ||
| } | ||
|
|
||
| type RootFieldCacheConfiguration struct { | ||
| TypeName string // e.g., "Query" | ||
| FieldName string // e.g., "topProducts" | ||
| CacheName string | ||
| TTL time.Duration | ||
| IncludeSubgraphHeaderPrefix bool | ||
| } | ||
| ``` | ||
|
|
||
| ### Cache Stats (Thread Safety) | ||
| ```go | ||
| type CacheStats struct { | ||
| L1Hits int64 // Main thread only (non-atomic) | ||
| L1Misses int64 // Main thread only (non-atomic) | ||
| L2Hits *atomic.Int64 // Goroutine-safe (atomic) | ||
| L2Misses *atomic.Int64 // Goroutine-safe (atomic) | ||
| } | ||
| ``` | ||
|
|
||
| ## Enabling Caching | ||
|
|
||
| ### Runtime Options | ||
| ```go | ||
| ctx.ExecutionOptions.Caching = CachingOptions{ | ||
| EnableL1Cache: true, // Per-request entity cache | ||
| EnableL2Cache: true, // External cache | ||
| } | ||
| ``` | ||
|
|
||
| ### Per-Subgraph Configuration (L2 only) | ||
| ```go | ||
| subgraphCachingConfigs := engine.SubgraphCachingConfigs{ | ||
| { | ||
| SubgraphName: "products", | ||
| RootFieldCaching: plan.RootFieldCacheConfigurations{ | ||
| {TypeName: "Query", FieldName: "topProducts", CacheName: "default", TTL: 30 * time.Second}, | ||
| }, | ||
| }, | ||
| { | ||
| SubgraphName: "accounts", | ||
| EntityCaching: plan.EntityCacheConfigurations{ | ||
| {TypeName: "User", CacheName: "default", TTL: 30 * time.Second}, | ||
| }, | ||
| }, | ||
| } | ||
|
|
||
| opts := []engine.FederationEngineConfigFactoryOption{ | ||
| engine.WithSubgraphEntityCachingConfigs(subgraphCachingConfigs), | ||
| } | ||
| ``` | ||
|
|
||
| ## Cache Flow | ||
|
|
||
| ### Sequential Execution (`tryCacheLoad`) | ||
| 1. `prepareCacheKeys()` - Generate L1 and L2 cache keys | ||
| 2. `tryL1CacheLoad()` - Check L1 (main thread) | ||
| 3. `tryL2CacheLoad()` - Check L2 (main thread) | ||
| 4. Fetch if needed, then `populateL1Cache()` and `updateL2Cache()` | ||
|
|
||
| ### Parallel Execution (`resolveParallel`) | ||
| 1. **Main thread**: `prepareCacheKeys()` + `tryL1CacheLoad()` for all nodes | ||
| 2. **Goroutines**: `tryL2CacheLoad()` + fetch via `loadFetchL2Only()` | ||
| 3. **Main thread**: Merge results, populate L1 cache | ||
|
|
||
| **Rationale**: L1 is cheap (in-memory), check on main thread to skip goroutine work early. L2/fetch are expensive, run in parallel. | ||
|
|
||
| ## Self-Referential Entity Fix | ||
|
|
||
| **Problem**: When `User.friends` returns the same `User` entity, L1 cache causes pointer aliasing → stack overflow on merge. | ||
|
|
||
| **Solution**: `shallowCopyProvidedFields()` in `loader_json_copy.go` creates copies based on `ProvidesData` schema. | ||
|
|
||
| ```go | ||
| // In tryL1CacheLoad: | ||
| ck.FromCache = l.shallowCopyProvidedFields(cachedValue, info.ProvidesData) | ||
| ``` | ||
|
|
||
| ## ProvidesData and Validation | ||
|
|
||
| `FetchInfo.ProvidesData` describes what fields a fetch provides. Used by: | ||
| - `validateItemHasRequiredData()` - Check if cached entity is complete | ||
| - `shallowCopyProvidedFields()` - Copy only required fields | ||
|
|
||
| **Critical**: For nested entity fetches, `ProvidesData` must contain entity fields (`id`, `username`), NOT the parent field (`author`). | ||
|
|
||
| ## configureFetchCaching Logic | ||
|
|
||
| ```go | ||
| func configureFetchCaching(internal, external) FetchCacheConfiguration { | ||
| // 1. Always preserve CacheKeyTemplate for L1 | ||
| result := FetchCacheConfiguration{CacheKeyTemplate: external.Caching.CacheKeyTemplate} | ||
|
|
||
| // 2. Check global disable | ||
| if v.Config.DisableEntityCaching { return result } | ||
|
|
||
| // 3. Determine fetch type FIRST | ||
| if external.RequiresEntityFetch || external.RequiresEntityBatchFetch { | ||
| // Entity fetch: all rootFields same type, use first | ||
| entityTypeName := internal.rootFields[0].TypeName | ||
| cacheConfig := fedConfig.EntityCacheConfig(entityTypeName) | ||
| } else { | ||
| // Root field fetch: need exactly 1 rootField | ||
| if len(internal.rootFields) != 1 { return result } | ||
| cacheConfig := fedConfig.RootFieldCacheConfig(rootField.TypeName, rootField.FieldName) | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Unit Testing | ||
|
|
||
| ```go | ||
| // Standard test setup | ||
| ctrl := gomock.NewController(t) | ||
| defer ctrl.Finish() | ||
|
|
||
| ds := NewMockDataSource(ctrl) | ||
| ds.EXPECT().Load(gomock.Any(), gomock.Any(), gomock.Any()). | ||
| DoAndReturn(func(ctx context.Context, headers any, input []byte) ([]byte, error) { | ||
| return []byte(`{"data":{...}}`), nil | ||
| }).Times(1) | ||
|
|
||
| loader := &Loader{caches: map[string]LoaderCache{"default": cache}} | ||
|
|
||
| // REQUIRED: Disable singleFlight for unit tests | ||
| ctx := NewContext(context.Background()) | ||
| ctx.ExecutionOptions.DisableSubgraphRequestDeduplication = true | ||
| ctx.ExecutionOptions.Caching = CachingOptions{EnableL1Cache: true, EnableL2Cache: true} | ||
|
|
||
| // REQUIRED: Always use arena | ||
| ar := arena.NewMonotonicArena(arena.WithMinBufferSize(1024)) | ||
| resolvable := NewResolvable(ar, ResolvableOptions{}) | ||
| resolvable.Init(ctx, nil, ast.OperationTypeQuery) | ||
|
|
||
| err := loader.LoadGraphQLResponseData(ctx, response, resolvable) | ||
| out := fastjsonext.PrintGraphQLResponse(resolvable.data, resolvable.errors) | ||
| ``` | ||
|
|
||
| ### FakeLoaderCache | ||
| Test mock in `cache_load_test.go` with TTL support and operation logging. | ||
|
|
||
| ### Assertions | ||
|
|
||
| **IMPORTANT**: Always use exact assertions in cache tests. Never use vague comparisons. | ||
|
|
||
| ```go | ||
| // GOOD: Exact values - always preferred | ||
| assert.Equal(t, 3, hitCount, "should have exactly 3 L1 hits") | ||
| assert.Equal(t, int64(12), l1HitsInt, "should have exactly 12 L1 hits") | ||
| assert.Equal(t, 2, accountsCalls, "should call accounts subgraph exactly twice") | ||
|
|
||
| // BAD: Never use vague comparisons | ||
| assert.GreaterOrEqual(t, hitCount, 1) // DON'T DO THIS | ||
| assert.Greater(t, l1HitsInt, int64(0)) // DON'T DO THIS | ||
| assert.LessOrEqual(t, calls, 5) // DON'T DO THIS | ||
| ``` | ||
|
|
||
| Exact assertions catch regressions that vague assertions miss. If the expected value changes, update the test to reflect the new exact value. | ||
|
|
||
| ### Snapshot Comments | ||
|
|
||
| **IMPORTANT**: Every event line in a `CacheAnalyticsSnapshot` assertion MUST have a brief comment explaining **why** that event occurred. Focus on causation, not field values. | ||
|
|
||
| ```go | ||
| // GOOD: explains the "why" | ||
| L2Reads: []resolve.CacheKeyEvent{ | ||
| {CacheKey: keyUser, Kind: resolve.CacheKeyMiss, ...}, // First request, L2 empty | ||
| {CacheKey: keyUser, Kind: resolve.CacheKeyHit, ...}, // Populated by Request 1 | ||
| }, | ||
|
|
||
| // BAD: restates the field value | ||
| {CacheKey: keyUser, Kind: resolve.CacheKeyMiss, ...}, // this is a miss | ||
| ``` | ||
|
|
||
| ## Federation Test Setup | ||
|
|
||
| Test services: `accounts`, `products`, `reviews` in `execution/federationtesting/` | ||
|
|
||
| ### Testing Entity Caching vs @provides | ||
| ```graphql | ||
| type Review { | ||
| # @provides - gateway trusts subgraph, NO entity resolution | ||
| author: User! @provides(fields: "username") | ||
|
|
||
| # No @provides - gateway MUST resolve via _entities | ||
| # Use for testing L1/L2 caching | ||
| authorWithoutProvides: User! | ||
| } | ||
| ``` | ||
|
|
||
| ### Run Tests | ||
| ```bash | ||
| go test -run "TestL1Cache" ./v2/pkg/engine/resolve/... -v | ||
| go test -run "TestFederationCaching" ./execution/engine/... -v | ||
| go test -race ./execution/engine/... -v # Race detector | ||
| ``` | ||
|
|
||
| ## astjson API Reference | ||
|
|
||
| ```go | ||
| // Create values on arena | ||
| astjson.ObjectValue(arena) | ||
| astjson.ArrayValue(arena) | ||
| astjson.StringValue(arena, string) | ||
| astjson.StringValueBytes(arena, []byte) | ||
| astjson.NumberValue(arena, string) | ||
| astjson.TrueValue(arena) | ||
| astjson.FalseValue(arena) | ||
| astjson.NullValue // Global constant (not a function) | ||
|
|
||
| // Manipulate | ||
| value.Set(arena, key, val) | ||
| value.SetArrayItem(arena, idx, val) | ||
| value.Get(keys...) | ||
| value.GetArray() | ||
| value.GetStringBytes() | ||
| value.MarshalTo([]byte) | ||
| value.Type() // TypeNull, TypeTrue, TypeObject, etc. | ||
| ``` | ||
|
|
||
| ## LoaderCache Interface | ||
|
|
||
| ```go | ||
| type LoaderCache interface { | ||
| Get(ctx context.Context, keys []string) ([]*CacheEntry, error) | ||
| Set(ctx context.Context, entries []*CacheEntry, ttl time.Duration) error | ||
| Delete(ctx context.Context, keys []string) error | ||
| } | ||
|
|
||
| type CacheEntry struct { | ||
| Key string | ||
| Value []byte // JSON-encoded entity | ||
| } | ||
| ``` | ||
|
|
||
| ## Always use exact assertions | ||
|
|
||
| Use `assert.Equal` with exact expected values. Never use `Contains`, `GreaterOrEqual`, `LessOrEqual`, or any vague comparison. | ||
| For objects or slices, always compare against a fully defined expected value, not just a subset. | ||
|
|
||
| ```go | ||
| // CORRECT | ||
| assert.Equal(t, 3, len(log), "should have exactly 3 cache operations") | ||
| assert.Equal(t, 1, tracker.GetCount(host), "should call subgraph exactly once") | ||
| assert.Equal(t, int64(12), stats.L1Hits, "should have exactly 12 L1 hits") | ||
|
|
||
| // WRONG — hides regressions | ||
| assert.GreaterOrEqual(t, len(log), 1) | ||
| assert.Greater(t, stats.L1Hits, int64(0)) | ||
| assert.Contains(t, log[0].Keys, expectedKey) | ||
| parse → normalize → validate → plan → resolve → response | ||
| ``` | ||
|
|
||
| If the expected value changes due to a code change, update the test to the new exact value. | ||
| ## Package Map | ||
|
|
||
| ### Core (v2/pkg/) | ||
|
|
||
| | Package | Purpose | | ||
| |---------|---------| | ||
| | `ast` | GraphQL AST representation | | ||
| | `astparser` | GraphQL parser (schema + operations) | | ||
| | `astnormalization` | AST normalization passes | | ||
| | `astvalidation` | Schema and query validation | | ||
| | `astvisitor` | AST visitor pattern for tree walking | | ||
| | `astprinter` | AST to string serialization | | ||
| | `asttransform` | AST transformations | | ||
| | `astimport` | AST import/merge utilities | | ||
| | `fastjsonext` | JSON manipulation extensions (astjson API) | | ||
| | `federation` | Federation composition utilities | | ||
| | `errorcodes` | Error code definitions | | ||
|
|
||
| ### Engine (v2/pkg/engine/) | ||
|
|
||
| | Package | Purpose | | ||
| |---------|---------| | ||
| | `plan` | Query planning, federation metadata, cache configuration types | | ||
| | **`resolve`** | **Resolution engine: fetching, caching, rendering** → see [resolve/CLAUDE.md](v2/pkg/engine/resolve/CLAUDE.md) | | ||
| | `datasource/graphql_datasource` | GraphQL subgraph datasource adapter | | ||
| | `postprocess` | Response post-processing passes (L1 cache optimization, fetch tree building) | | ||
|
|
||
| ### Execution (execution/) | ||
|
|
||
| | Package | Purpose | | ||
| |---------|---------| | ||
| | `engine` | Federation engine config factory (`SubgraphCachingConfig`, `WithSubgraphEntityCachingConfigs`), E2E tests | | ||
| | `federationtesting` | Test federation services: accounts, products, reviews | | ||
| | `graphql` | GraphQL execution utilities | | ||
|
|
||
| ## Key Architectural Decisions | ||
|
|
||
| - **Federation-first**: designed for federated GraphQL with entity resolution and `@key`/`@provides`/`@requires` | ||
| - **Arena-based allocation**: JSON values live on arena memory (no GC pressure), released per-request | ||
| - **Parallel resolution**: fetch tree with Sequence/Parallel nodes, 4-phase parallel execution with L1/L2 caching | ||
| - **Two-pass rendering**: pre-walk (validate, collect errors) + print-walk (render JSON) | ||
|
|
||
| ## Entity Caching | ||
|
|
||
| Two-level entity caching system (L1 per-request + L2 external). See: | ||
| - [v2/pkg/engine/resolve/CLAUDE.md](v2/pkg/engine/resolve/CLAUDE.md) — full resolve package reference (resolution pipeline + caching internals) | ||
| - [ENTITY_CACHING_INTEGRATION.md](ENTITY_CACHING_INTEGRATION.md) — router integration guide (public APIs, configuration, examples) | ||
|
|
||
| ## Testing Conventions | ||
|
|
||
| - **Exact assertions only**: use `assert.Equal` with exact expected values, never `GreaterOrEqual`, `Contains`, or vague comparisons | ||
| - **Snapshot comments**: every event line in `CacheAnalyticsSnapshot` assertions must explain **why** that event occurred | ||
| - **Cache log rule**: every `ClearLog()` must have `GetLog()` + assertions before the next `ClearLog()` | ||
| - **Federation test services**: `accounts`, `products`, `reviews` in `execution/federationtesting/` | ||
| - Run: `go test ./v2/pkg/engine/resolve/... -v` and `go test ./execution/engine/... -v` | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Specify a fence language for the data-flow block.
The fenced block should include a language tag (
text) to pass MD040.Suggested doc fix
Verify each finding against the current code and only fix it if needed.
In
@CLAUDE.mdaround lines 9 - 11, Add a language tag to the fenced data-flowblock so the code fence reads with a
textspecifier: replace thetriple-backtick fence around the line "parse → normalize → validate → plan →
resolve → response" with a
text fenced block (i.e., open withtext andclose with ```) to satisfy MD040; look for the fenced block containing the
data-flow string in CLAUDE.md.