User transactable attributes may halt progress if sources dictate times

[bachdavi]

Just ran into this.
0561422 introduced advance_domain_to_source() which enables sources to dictate times.
Internally we use the domain probe to gather frontier information about all attributes in one domain.
This frontier information in turn is then used to advance() all kinds of things.
We probe source created as well as user created attributes.
Source attribute sessions are auto_flush as soon as the source downgrades its capability.
User created attribute sessions are not.
This leads to the problem that the frontier is stuck on the transaction of the user.
advance_source_to_domain() is unable to advance based on the domains frontier information, as it is held back by the user created attribute.

One solution is to simply not probe the user created and transactable attributes.
Transactions to the user attributes are given to its session at the correct current epoch.
One thing that arrises is that then user input is bound to source timestamps...

[bachdavi]

A quick fix here a73c867
There seems to be no case where we want user created attributes to be probed.
Maybe there exists a case where source attributes do not want to be probed, basically saying they yield their authority to other sources / user advances.
So one could also make it configurable in AttributeConfig.

[comnik]

This should be addressed in f09ec28. I've added some simple unit tests to the domain module, but more are required to gain confidence.

[comnik]

More tests added in 491554a. The logic seems correct (now, had to fix a bunch of edge cases...), and the interesting case is 491554a#diff-02e97a1fbd4aa8f2c9eac9bb8a89a414R57, which stalls everything as expected.

Domain's now expose advance_epoch, advance, and probed_source_count. Naming still sub-optimal. For domains without any probed sources, advance_epoch will forward the epoch and advance all input handles and all traces accordingly.

For domains with probed sources, advance will take the domain frontier into account and forward the epoch to unblock sources (thus ensuring that the domain frontier is always a lower bound), and advance all traces up to the domain frontier. advance is intended to be used after a bunch of work has been done, s.t. the domain frontier has actually had the chance to change.

As discussed, we should move away from checking anything in the server loop and simply always step a bunch, and advance after. In addition, we should call advance_epoch at some interval, which will determine the minimal result granularity in domains where no sources are present.
Thanks to event_driven, over-stepping shouldn't be a problem. With those changes we should be able to close #55 and never stall sources with input handles.