Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions docs/design/dfx-testing-story-example/dfx.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"canisters": {
"cleanSheets": {
"_comment:": "we list `main` service code versus its `unit-tests` (each is a single Motoko function to run) versus its `functional-tests` (each is an `install-and-go`), versus `e2e-tests` that each involve more than a single `install-and-go`, and require a `test-script` over several canisters."

"main": "src/cleanSheets.mo"
"unit-tests": {
"all": "test/main.mo"
},
"func-tests": {
"functionalTest1": "test/simpleAdaptonDivByZero.mo",
"functionalTest2": "test/simpleExcelEmulation.mo",
},
"e2e-tests": {
"AliceBob": {
"_comment:": "test-script installs and scripts bots `alice` and `bob`"
"test-script": "test/aliceBob/test-script"
"aliceCanister": "test/aliceBob/aliceCanister.mo"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those (already defined canisters) necessary? I see the need for a test-script key I just don't see the need to repeat canisters that are already listed in the /canisters key.

Copy link
Contributor Author

@matthewhammer matthewhammer Feb 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ideas here are that

  • some canisters are just for tests, and are only defined for the tests that require them, and
  • separate name scoping of canister names within e2e tests and their test scripts, but not between distinct e2e tests; they are independent, by assumption. So, distinct e2e tests do not share canister names, and can redefine them to point at different files, with different implementations (I don't do that here, though).

So, the aliceCanister, bobCanister and charlieCanister keys are only defined for the e2e test that defines them. So, they have to be defined if they are reused, as they are here. The idea was to show exactly this kind of sharing, given the restricted scoping of the canister names.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there reasons to scope them? It seems this just causes more copy-paste.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separate namespaces.

I expect that separate namespaces will generally be more useful than one big shared global one, where you have to "mimic" the separation I want here by introducing long names for everything. In terms of implementation complexity, separate namespaces is no more complex either; there's no implementation-based reason not to support them.

In terms of user value: If I have two test scripts that introduce two cansiters called testBot, implemented in two different directories, it's the filesystem structure that distinguishes them as files; as canisters that appear in test scripts, we need to distinguish them to give them the same short name, like testBot, and know that this same short name is never ambiguous. Separate namespaces (much like the scoping rules of any PL with namespaces/modules) is the answer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this just causes more copy-paste.

I don't follow this concern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing some boiler-plate copy-paste; aliceCanister seems to always point to test/aliceBob/aliceCanister.mo, so maybe instead of doing the name => main_file map, we could just list the canisters we want to expose. Maybe if we move those test definitions to their own files so they free up dfx.json would also be better. WDYT?

"bobCanister": "test/aliceBob/bobCanister.mo"
}
"AliceBobCharlie": {
"_comment:": "reuse bots `alice` and `bob`, and includes bot `charlie`"
"test-script": "test/aliceBobCharlie/test-script"
"aliceCanister": "test/aliceBob/aliceCanister.mo"
"bobCanister": "test/aliceBob/bobCanister.mo"
"charlieCanister": "test/aliceBobCharlie/charlieCanister.mo"
}
}
},
}
}
1 change: 1 addition & 0 deletions docs/design/dfx-testing-story-example/src/adapton.mo
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
(OMITTING EXAMPLE CODE)
1 change: 1 addition & 0 deletions docs/design/dfx-testing-story-example/src/cleanSheets.mo
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
(OMITTING EXAMPLE CODE)
1 change: 1 addition & 0 deletions docs/design/dfx-testing-story-example/src/eval.mo
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
(OMITTING EXAMPLE CODE)
1 change: 1 addition & 0 deletions docs/design/dfx-testing-story-example/src/main.mo
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
(EXAMPLE CODE OMITTED)
1 change: 1 addition & 0 deletions docs/design/dfx-testing-story-example/src/types.mo
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
(OMITTING EXAMPLE CODE)
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
(EXAMPLE CODE OMITTED)
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
(EXAMPLE CODE OMITTED)
10 changes: 10 additions & 0 deletions docs/design/dfx-testing-story-example/test/aliceBob/test-script
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
install bobCanister
==> #ok(bobId)
install aliceCanister
==> #ok(aliceId)
call bobId addFriend aliceId
Copy link
Contributor

@hansl hansl Feb 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With inter-canister calls, could you elaborate on your rationale for using a DSL instead of actual Motoko?

In Motoko:

let result = await bob.addFriend(aliceId);
assert(result == Ok); // or some Motoko

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is scoping.

The intention of this Alice canister code is that it does not import (or know about) Bob's code statically, and vice versa. They are "test bots" written independently, and only linked dynamically, by a test script.

I realize that this tiny example could mention those names using aliases; but, the point here was to illustrate a multi-canister test that does not use aliases, and still has canisters interact.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does bobId come from? Is it a local identifier? How is its name derived?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, bobId is a test script variable, bound by the result pattern #ok(bobId). This pattern will not match an error/trap result, only a successful one.

In general, these patterns may have variables, which may appear free in subsequent actions, as illustrated here with bobId.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha so ==> #ok(abc) is also a declaration for abc? I need to read the rest of this document, sorry ^_^

==> #ok(())
call aliceId addFriend BobId
==> #ok(())
call bobId collaborateWithAlice
==> #ok(())
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
(EXAMPLE CODE OMITTED)
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
install bobCanister ==> #ok(bobId)
install aliceCanister ==> #ok(aliceId)
install charlieCanister ==> #ok(charlieId)
# comment out these calls, to try alternatives where Charlie starts all interactions
# call bobId addFriend aliceId ==> #ok(())
# call bobId addFriend charlieId ==> #ok(())
# call aliceId addFriend bobId ==> #ok(())
# call aliceId addFriend charlieId ==> #ok(())
call charlieId addFriend bobId ==> #ok(())
call charlieId addFriend aliceId ==> #ok(())
call charlieId collaborateWithAliceAndBob ==> #ok(())
208 changes: 208 additions & 0 deletions docs/design/dfx-testing-story-example/test/simpleAdaptonDivByZero.mo
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
import R "mo:stdlib/result";
import P "mo:stdlib/prelude";

import T "../src/types";
import A "../src/adapton";
import E "../src/eval";

/*

This file reproduces the intro example from the Adapton Rust docs,
found here:

https://docs.rs/adapton/0/adapton/#demand-driven-change-propagation

The DCG update behavior asserted here (see uses of
`assertLogEventLast` below) matches the cleaning/dirtying behavior
described in the link above, and in other documents and papers.

In particular, the same example is used in a recorded adapton talk,
from the http://adapton.org website. The link above has slides with
still pictures from the talk.

*/

actor SimpleAdaptonDivByZero {

public func go() {
let ctx : T.Adapton.Context = A.init(true);

// "cell 1 holds 42":
let cell1 : T.Adapton.NodeId = assertOkPut(A.put(ctx, #nat(1), #nat(42)));
A.assertLogEventLast
(ctx, #put(#nat(1), #nat(42), []));

// "cell 2 holds 2":
let cell2 : T.Adapton.NodeId = assertOkPut(A.put(ctx, #nat(2), #nat(2)));
A.assertLogEventLast
(ctx, #put(#nat(2), #nat(2), []));

// "cell 3 holds a suspended closure for this expression:
//
// get(cell1) / get(cell2)
//
// ...and it is still unevaluated".
//
let cell3 : T.Adapton.NodeId = assertOkPut(
A.putThunk(ctx, #nat(3),
E.closure(
null,
#strictBinOp(#div,
#get(#refNode(cell1)),
#get(#refNode(cell2))
)))
);

// "cell 4 holds a suspended closure for this expression:
//
// if (get(cell2) == 0) { 0 }
// else { get(cell3) }
//
// ...and it is still unevaluated".
//
let cell4 : T.Adapton.NodeId = assertOkPut(
A.putThunk(ctx, #nat(4),
E.closure(
null,
#ifCond(#strictBinOp(#eq,
#get(#refNode(cell2)),
#nat(0)),
#nat(0),
#get(#refNode(cell3)))))
);

// demand division:
let res1 = assertOkGet(A.get(ctx, cell4));
A.assertLogEventLast(ctx,
#get(#nat(4), #ok(#nat(21)),
[
#evalThunk(
#nat(4), #ok(#nat(21)),
[#get(#nat(2), #ok(#nat(2)), []),
#get(#nat(3), #ok(#nat(21)),
[
#evalThunk(
#nat(3), #ok(#nat(21)),
[
#get(#nat(1), #ok(#nat(42)), []),
#get(#nat(2), #ok(#nat(2)), [])
])
])
])
])
);

// "cell 2 holds 0":
ignore A.put(ctx, #nat(2), #nat(0));
A.assertLogEventLast
(ctx,
#put(#nat(2), #nat(0),
[
#dirtyIncomingTo(
#nat(2),
[
#dirtyEdgeFrom(
#nat(3),
[
#dirtyIncomingTo(
#nat(3),
[
#dirtyEdgeFrom(
#nat(4),
[
#dirtyIncomingTo(#nat(4), [])
])
])
]),
#dirtyEdgeFrom(
#nat(4),
[
#dirtyIncomingTo(
#nat(4), [])
])
])
])
);

// re-demand division:
let res2 = assertOkGet(A.get(ctx, cell4));
A.assertLogEventLast
(ctx,
#get(#nat(4), #ok(#nat(0)),
[
#cleanThunk(
#nat(4), false,
[
#cleanEdgeTo(
#nat(2), false, [])
]),
#evalThunk(
#nat(4), #ok(#nat(0)),
[
#get(#nat(2), #ok(#nat(0)), [])
])
]));

// "cell 2 holds 2":
ignore A.put(ctx, #nat(2), #nat(2));
A.assertLogEventLast
(ctx,
#put(#nat(2), #nat(2),
[
#dirtyIncomingTo(
#nat(2),
[
#dirtyEdgeFrom(
#nat(4),
[
#dirtyIncomingTo(
#nat(4), [])
])
])
]));

// re-demand division:
let res3 = assertOkGet(A.get(ctx, cell4));
A.assertLogEventLast
(ctx,
#get(#nat(4), #ok(#nat(21)),
[
#cleanThunk(
#nat(4), false,
[
#cleanEdgeTo(#nat(2), false, [])
]),
#evalThunk(
#nat(4), #ok(#nat(21)),
[
#get(#nat(2), #ok(#nat(2)), []),
#get(#nat(3), #ok(#nat(21)),
[
#cleanThunk(
#nat(3), true,
[
#cleanEdgeTo(#nat(1), true, []),
#cleanEdgeTo(#nat(2), true, [])
])
])
])
]));
};


func assertOkPut(r:R.Result<T.Adapton.NodeId, T.Adapton.PutError>) : T.Adapton.NodeId {
switch r {
case (#ok(id)) { id };
case _ { P.unreachable() };
}
};

func assertOkGet(r:R.Result<T.Adapton.Result, T.Adapton.GetError>) : T.Eval.Result {
switch r {
case (#ok(res)) { res };
case _ { P.unreachable() };
}
};
};

//SimpleAdaptonDivByZero.go();
Copy link
Contributor Author

@matthewhammer matthewhammer Feb 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a common class of test: it's a single canister, and a single call (e.g., go() like here.)

So to be clear, this class is so common that I don't even want to write a script for any of them. The idea is that mentioning them in dfx.json is enough to get the test system to call their go function (or whatever we call it).

I call each a "functional test" since each one is much more than a unit test, but not as complex as a multi-canister test, or one that requires multiple calls. Within one call, we can do a pretty full test of a non-actor library's functionality, as illustrated here.

(Thanks for the idea @nikclayton-dfinity!)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that comment load bearing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the question.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has that comment semantic meaning?

Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
import R "mo:stdlib/result";
import P "mo:stdlib/prelude";
import Debug "mo:stdlib/debug";

import T "../src/types";
import A "../src/adapton";
import E "../src/eval";

actor simpleExcelEmulation {

public func go() {

let sheetExp : T.Eval.Exp =
#sheet(
#text("S"),
[
[ #nat(1), #nat(2) ],
[ #strictBinOp(#add,
#cellOcc(0,0),
#cellOcc(0,1)),
#strictBinOp(#mul,
#nat(2),
#cellOcc(1,0)) ]
]);

// Adapton maintains our dependence graph
let actx : T.Adapton.Context = A.init(true);

// create the initial Sheet datatype from the DSL expression above
let s : T.Sheet.Sheet = {
switch (E.evalExp(actx, null, sheetExp)) {
case (#ok(#sheet(s))) s;
case _ { P.unreachable() };
}};

// Demand that the sheet's results are fully refreshed
ignore E.Sheet.refresh(actx, s);
ignore E.Sheet.refresh(actx, s);
A.assertLogEventLast(
actx,
#get(#tagTup(#text("S"), [#nat(1), #nat(1), #text("out")]), #ok(#nat(6)), [])
);

// Update the sheet by overwriting (0,0) with a new formula:
ignore E.Sheet.update(actx, s, 0, 0,
#strictBinOp(#add, #nat(666), #cellOcc(0,1)));

// Demand that the sheet's results are fully refreshed
ignore E.Sheet.refresh(actx, s);
ignore E.Sheet.refresh(actx, s);
assert (s.errors.len() == 0);
A.assertLogEventLast(
actx,
#get(#tagTup(#text("S"), [#nat(1), #nat(1), #text("out")]), #ok(#nat(1_340)), [])
);

// Update the sheet, creating a cycle at (0,0):
ignore E.Sheet.update(actx, s, 0, 0, #cellOcc(0,0));
ignore E.Sheet.refresh(actx, s);
assert (s.errors.len() != 0);

// Update the sheet, removing the cycle:
ignore E.Sheet.update(actx, s, 0, 0,
#strictBinOp(#add, #nat(666), #cellOcc(0,1)));

// Demand that the sheet's results are fully refreshed
ignore E.Sheet.refresh(actx, s);
ignore E.Sheet.refresh(actx, s);
assert (s.errors.len() == 0);
A.assertLogEventLast(
actx,
#get(#tagTup(#text("S"), [#nat(1), #nat(1), #text("out")]), #ok(#nat(1_340)), [])
);
};
};

//simpleExcelEmulation.go()
Loading