Crash in nested scope #107
Comments
|
Whoa, that is a pretty minimal repro. Let me take a look today and I'll try and track down the issue. |
|
Ok, I think I have the answer. The high-level bit is that about a year ago differential dataflow changed, and You are using timely dataflow loop construction, not differential dataflow loop construction, and there are a few (one? I think) additional things you have to do to maintain differential dataflow's invariants. Specifically, as the data go around the loop, you need to increment the corresponding coordinate of the In the current .inner
.map(|(x,t,d)| (x, Product::new(t.outer, t.inner+1), d))
.connect_loop(self.feedback);If I put this into your example, as impl<'a, G: Scope, D: Default+Data+Hashable> Drop for Variable<'a, G, D> where G::Timestamp: Lattice+Ord {
fn drop(&mut self) {
if let Some(feedback) = self.feedback.take() {
self.current.distinct()
.inner
.map(|(x,t,d)| (x, Product::new(t.outer, t.inner+1), d))
.connect_loop(feedback);
}
}
}the example seems to run fine. |
|
Btw, pushed a change to dataflog.rs and marked as "fixed", but totally re-open if you think this is unacceptable. :D |
|
Thank you so much for the quick fix! I confirm that the fix also works in the context of the more complex program where I originally ran into this. |
* Rust template, wip * Rust template wip * datalog template: wip * rust template wip * wip * wip * wip * wip * wip * wip * initial implementation of program.rs * first test passes * performance improvement: batch multiple updates in one message * two more tests, one failing * more tests, bug fixes * only rollback input relations * tests * use channel instead of cond variable * recursion test (currently failing); refactored tests * applied Frank's fix to Variable implementation see https://github.com/frankmcsherry/differential-dataflow/issues/107 * fix incorrect use of struct Variable test_recursion now passes * improved performance * better antijoin interface; +1 test * some documentation + cleanup * replace many cfg(test) annotations with one * performance improvements - buffer messages between the client and dataflow threads - advance epoch once per update * remove unused file * another unused file * Mihai's review * refactor: move progDependencyGraph to DatalogProgram.hs * wip * DatalogProgram.progExpandMultiheadRules * added from_XXX methods to Uint; implemented struct Int for unbounded signed integers * explore Rust syntax for writing arrange functions * Datalog->Rust compiler wip * wip * wip * wip * wip * add Cargo.lock to .gitignore * mkProg * addArrangement * wip * wip * disallow variable declarations without assignment This creates an undefined behavior and complicates compilation to Rust, which does not allow this * wip * generate complete Rust project * bug fixes * implemented bit slicing and concatenation * bug fixes * more fixes * replace integer constants with bitvectors based on type inference * more Rust compilation bugs * Add imports arguments to compile. Can be used to either import additional Rust files that implement extern functions or inline the implementation directly in the generated code. * more compilation issues function.dl now compiles * more compilation fixes * run cargo test from Spec.hs * fix to previous commit * FlatMap support * implement __builtin_2str * support FlatMap in Compile.hs * all tests compile * add instructions to install Rust to README * generate Value:: enums for relations that are not used in any rules * pass update callback as argument to generated Rust code
I am using the
Variableimplementation from thedataflog.rsexample to construct recursive relations. However, my program crashes any time I use a nested scope. Here is the minimal failing example. The definition ofstruct Variableis copied fromdataflog.rsverbatim. The actual dataflow consists of collection1 and collection2, with collection2 being an exact copy of collection1.Note that there is no recursion here, so this does not actually require a nested scope. I only use it demonstrate the failure. It still crashes if I use an actual recursive definition.
Here is the stack trace:
The text was updated successfully, but these errors were encountered: