Skip to content

Latest commit

 

History

History
53 lines (36 loc) · 2.43 KB

FORMULAS-BACKLOG-AND-FEATURE-IDEAS.md

File metadata and controls

53 lines (36 loc) · 2.43 KB
Raw

  • improve formula failure reporting a. [x] report the first failure b. [x] make sure the fact count only increases once per formula

  • make number of generated facts per formula dynamically bindable a. [x] throw an exception if this value is set < 1

  • figure out how to make syntax validation errors show something more sensible than the error message you'd see for a problem with a fact validation problem

  • don't run more tests than need be if there is already failure in this formula's batch.

  • syntax validate thata formula only has one check in it
    a. [x] need to make this more thorough... right now the only test of this feature checks a simple provided case, but needs to work with against-background, background and other more interesting cases b. [ ] it is more thorough now, but let's make it seriously thoough :)

  • formula macro calls a (constantly []) version of shrink on failures

  • add future-formula (and variant names)

  • cleaner syntax for overriding number of trials per formula. Use the num-trials var just for global changes or changes to be visible for groups of formulas.

  • validate that opt-map is only used with valid keys.

  • validate that :num-trials is 1+

  • Work with Meikel Brandmeyer to combine ClojureCheck's Generators with Shrink. implement shrinking. Report only the first fully shrunken failure [ ] 'shrink' depends on domain of 'generate'

  • ability to override shrink function on a per generator basis a. [ ] ablity to not shrink at all on a per generator basis (make a nice syntactic sugar for this... as it is one of the cases of the above.

  • fix strange error if you run (formula [a 1] 1 =>) ... since the formula macro splices in :formula :formula-in-progress possibly solution is to not using fact macro inside of formula, but instead do something like tabular

  • if line numbers shift, then ensure that they always report correctly -- so far I don't know if this even needs to change, since it seems to work fine. Think about it and decide if tests to prevent regressions are useful here.

  • consider implementing with @marick's metaconstant syntax a. [ ] if we do metaconstant style, implement generator overriding

  • figure out what part of t-formulas is registering as a lot more than 1 report per formula. (100???)