Releases: timgent/data-flare
Releases · timgent/data-flare
Changes in package visibility to support data-flare API and UI development
QcTypes made public
v0.1.13 made QcType public (#117)
Min/Max metrics & checks, identical table checks, schema checks
- MinValueMetric, MaxValueMetric, minValueCheck, maxValueCheck added
- dfsMatchUnordered and dfsMatchOrdered check helpers added
- dsSchemasMatch check added
Distinctness metric, compliance empty DS fix, runBlocking method
- Distinctness metric added (docs still pending)
- runBlocking method added for running check suite and waiting for the result before returning (for those who don't want to have to deal with a Future)
- Compliance metric bug fixed where it was giving a value of 0 for empty dataset, rather than 1
Cross-build to Scala 2.12 and improve error handling
- If arbChecks or metric based checks throw exceptions they are now caught, and details of those returned in the check results for the relevant checks
- Deequ support removed due to issues it would cause for building Scala 2.12 version of this library
- Now cross-builds to Scala 2.12
Interface simplification and new features
BREAKING CHANGES
- Simplification of API - see docs for updates
ADDITIONAL CHANGES
- Additional constructors for writing filters with less boilerplate
- Add method to pretty print QC Results
- Make compilation for checks stricter so types have to align
- Allow storage of metrics even when no checks for them
- Comparator helper added for checking if values are within X percent of each other
- SumValues metric bug fixed where onColumn info was not stored or returned
- Made check descriptions a sealed trait so they display more information more clearly
Interface tweak
- Renamed DatasetComparisonCheck to DualDatasetCheck
- No other functional changes
First release
Testing CI...