Extract ready times from existing chains

Teaches the merge batcher to extract ready times from the existing chains, and maintaining the chain invariant after extracting data. This reduces the effort to maintain data that is not yet ready, by maintaining a frontier per chain block that allows us to efficiently decide that a block needs to be inspected or not. Signed-off-by: Moritz Hoffmann <[email protected]>
TimelyDataflow · Mar 1, 2024 · 1d6a5e5 · 1d6a5e5
1 parent 0f93140
commit 1d6a5e5
Show file tree

Hide file tree

Showing 3 changed files with 531 additions and 296 deletions.
diff --git a/examples/hello.rs b/examples/hello.rs
@@ -51,7 +51,7 @@ fn main() {
 
         // Load up graph data. Round-robin among workers.
         for _ in 0 .. (edges / peers) + if index < (edges % peers) { 1 } else { 0 } {
-            input.update((rng1.gen_range(0, nodes), rng1.gen_range(0, nodes)), 1)
+            input.update_at((rng1.gen_range(0, nodes), rng1.gen_range(0, nodes)), 1_000_000, 1)
         }
 
         input.advance_to(1);