docs(capture): NFS flush ceiling is metadata-RPC count, not fsync; add durability-vs-response-timing - #201
Merged
Conversation
…sync; add durability-vs-response-timing section
…und on 64MB), drop unattested ~90 files/s
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes the filesystem-consumer docs piece from #174 (the only part not already merged via #195/#198/#200).
What changed
docs/features/capture_consumers.md:status: "pending". Clients should wait for the expected file set (layers × hooks), not a fixed sleep. Points at thecapture_waitrequest flag for servers that want to block.fsynctable row and the new subsection with our own NFS measurements.Why this differs from #174's wording
#174 attributed the per-small-file NFS ceiling to
fsyncbeing a synchronous COMMIT round-trip. Our filesystem investigation found the governing cost is metadata-RPC count (create/open/close/rename), notfsync—fsync/atomic_publishtoggles are near-no-ops on a sync NFS export, while thepacked/shardedlayouts (which collapse metadata RPCs per request) are the real lever. Measured ~90 small files/s (~30 MB/s at ~350 KB/file) vs. ~93% of raw disk bound for large contiguous writes. The docs now state that cause.