v0.13.0 Release

Changelog

DefraDB v0.13 is a major pre-production release. Until the stable version 1.0 is reached, the SemVer minor patch number will denote notable releases, which will give the project freedom to experiment and explore potentially breaking changes.

To get a full outline of the changes, we invite you to review the official changelog below. This release does include a Breaking Change to existing v0.12.x databases. If you need help migrating an existing deployment, reach out at [email protected] or join our Discord at https://discord.source.network/.

Features

• feat: Doc encryption with symmetric key (#2731)
• feat: Doc field encryption (#2817)
• feat: Enable indexing for DateTime fields (#2933)
• feat: Handle P2P with SourceHub ACP (#2848)
• feat: Implement SourceHub ACP (#2657)
• feat: Remove IsObjectArray (#2859)

Fix

• fix: Add ns precision support to time values (#2940)
• fix: Allow querying of 9th, 19th, 29th, etc collections (#2819)
• fix: Create mutation introspection (#2881)
• fix: Enable filtering doc by fields of JSON and Blob types (#2841)
• fix: Filter with date and document with nil date value (#2946)
• fix: Handle index queries where child found without parent (#2942)
• fix: Handle multiple child index joins (#2867)
• fix: No panic if filter condition on indexed field is empty (#2929)
• fix: Panic with different composite-indexed child objects (#2947)
• fix: Support one-many self joins without primary directive (#2799)

Refactoring

• refactor: Decouple client.DB from net (#2768)
• refactor: GQL responses (#2872)
• refactor: Network test sync logic (#2748)

Testing

• test: Add assert on DocIndex for child documents (#2871)
• test: Fix refreshing of docs in change detector (#2832)
• test: Remove hardcoded test identities (#2822)

v0.12.0 Release

Changelog

DefraDB v0.12 is a major pre-production release. Until the stable version 1.0 is reached, the SemVer minor patch number will denote notable releases, which will give the project freedom to experiment and explore potentially breaking changes.

To get a full outline of the changes, we invite you to review the official changelog below. This release does include a Breaking Change to existing v0.11.x databases. If you need help migrating an existing deployment, reach out at [email protected] or join our Discord at https://discord.source.network/.

Features

• feat: Ability to generate a new identity (#2760)
• feat: Add async transaction callbacks (#2708)
• feat: Add authentication for ACP (#2649)
• feat: Allow lens runtime selection via config (#2684)
• feat: Enable sec. indexes with ACP (#2602)
• feat: Inject ACP instance into the DB instance (#2633)
• feat: Keyring (#2557)
• feat: Sec. indexes on relations (#2670)

Fix

• fix: Add version check in basicTxn.Query (#2742)
• fix: Allow primary field declarations on one-many (#2796)
• fix: Change new identity keys to hex format (#2773)
• fix: Incorporate schema root into docID (#2701)
• fix: Keyring output (#2784)
• fix: Make node options composable (#2648)
• fix: Merge retry logic (#2719)
• fix: Race condition when testing CLI (#2713)
• fix: Remove limit for fetching secondary docs (#2594)
• fix: Remove shared mutable state between database instances (#2777)
• fix: Resolve incorrect merge conflict (#2723)
• fix: Return slice of correct length from db.AddSchema (#2765)
• fix: Use node representation for Block (#2746)

Documentation

• docs: Add http/openapi documentation & ci workflow (#2678)
• docs: Document Event Update struct (#2598)
• docs: Remove reference to client ping from readme (#2793)
• docs: Streamline cli documentation (#2646)

Refactoring

• refactor: Change counters to support encryption (#2698)
• refactor: Change from protobuf to cbor for IPLD (#2604)
• refactor: Change local_acp implementation to use acp_core (#2691)
• refactor: DAG sync and move merge outside of net package (#2658)
• refactor: Extract Defra specific logic from ACPLocal type (#2656)
• refactor: Extract definition stuff from collection.go (#2706)
• refactor: Move internal packages to internal dir (#2599)
• refactor: Reorganize global CLI flags (#2615)
• refactor: Replace subscription events publisher (#2686)
• refactor: Rework definition validation (#2720)
• refactor: Use events to test network logic (#2700)

Testing

• test: Add relation substitute mechanic to tests (#2682)
• test: Allow assertion of AddSchema results (#2788)
• test: Allow test harness to execute benchmarks (#2740)
• test: Remove duplicate test (#2787)
• test: Support asserting on doc index in test results (#2786)
• test: Test node pkg constructor via integration test suite (#2641)

v0.11.0 Release

Changelog

DefraDB v0.11 is a major pre-production release. Until the stable version 1.0 is reached, the SemVer minor patch number will denote notable releases, which will give the project freedom to experiment and explore potentially breaking changes.

To get a full outline of the changes, we invite you to review the official changelog below. This release does include a Breaking Change to existing v0.10.x databases. If you need help migrating an existing deployment, reach out at [email protected] or join our Discord at https://discord.source.network/.

Features

• feat: Add Access Control Policy (#2338)
• feat: Add Defra-Lens support for branching schema (#2421)
• feat: Add P Counter CRDT (#2482)
• feat: Add PatchCollection (#2402)
• feat: Allow mutation of col sources via PatchCollection (#2424)
• feat: Force explicit primary decl. in SDL for one-ones (#2462)
• feat: Lens runtime config (#2497)
• feat: Move relation field properties onto collection (#2529)
• feat: Update corelog to 0.0.7 (#2547)

Fix

• fix: Add check to filter result for logical ops (#2573)
• fix: Allow update when updating non-indexed field (#2511)
• fix: Handle compound filters on related indexed fields (#2575)
• fix: Make all array kinds nillable (#2534)
• fix: Return correct results from one-many indexed filter (#2579)

Documentation

• docs: Add data definition document (#2544)

Refactoring

• refactor: Add NormalValue (#2404)
• refactor: Clean up client/request package (#2443)
• refactor: DB transactions context (#2513)
• refactor: Merge collection UpdateWith and DeleteWith (#2531)
• refactor: Replace logging package with corelog (#2406)
• refactor: Rewrite convertImmutable (#2445)
• refactor: Unify Field Kind and Schema properties (#2414)

Testing

• test: Add flag to skip network tests (#2495)

v0.10.0 Release

Changelog

DefraDB v0.10 is a major pre-production release. Until the stable version 1.0 is reached, the SemVer minor patch number will denote notable releases, which will give the project freedom to experiment and explore potentially breaking changes.

To get a full outline of the changes, we invite you to review the official changelog below. This release does include a Breaking Change to existing v0.9.x databases. If you need help migrating an existing deployment, reach out at [email protected] or join our Discord at https://discord.source.network/.

Features

• feat: Add JSON scalar (#2254)
• feat: Add case insensitive like operator (#2368)
• feat: Add composite indexes (#2226)
• feat: Add support for views with Lens transforms (#2311)
• feat: Allow setting null values on doc fields (#2273)
• feat: Generate OpenAPI command (#2235)
• feat: Model Col. SchemaVersions and migrations on Cols (#2286)
• feat: Multiple docs with nil value on unique-indexed field (#2276)
• feat: Replace FieldDescription.RelationType with IsPrimary (#2288)
• feat: Reverted order for indexed fields (#2335)
• feat: Rework GetCollection/SchemaByFoo funcs into single (#2319)

Fix

• fix: Add latest image tag for ghcr (#2340)
• fix: Add missing delta payload (#2306)
• fix: Add missing directive definitions (#2369)
• fix: Add validation to JSON fields (#2375)
• fix: Fix compound relational filters in aggregates (#2297)
• fix: Load root dir before loading config (#2266)
• fix: Make peers sync secondary index (#2390)
• fix: Make returned collections respect explicit transactions (#2385)
• fix: Mark docs as deleted when querying in delete mut (#2298)
• fix: Move field id off of schema (#2336)
• fix: Update GetCollections behaviour (#2378)

Refactoring

• refactor: Decouple net config (#2258)
• refactor: Generate field ids using a sequence (#2339)
• refactor: HTTP config (#2278)
• refactor: Make CollectionDescription.Name Option (#2223)
• refactor: Make config internal to CLI (#2310)
• refactor: Node config (#2296)
• refactor: Remove unused Delete field from client.Document (#2275)

v0.9.0 Release

DefraDB v0.9 is a major pre-production release. Until the stable version 1.0 is reached, the SemVer minor patch number will denote notable releases, which will give the project freedom to experiment and explore potentially breaking changes.

To get a full outline of the changes, we invite you to review the official changelog below. This release does include a Breaking Change to existing v0.8.x databases. If you need help migrating an existing deployment, reach out at [email protected] or join our Discord at https://discord.source.network/.

Changelog

Features

• feat: Add PN Counter CRDT type (#2119)
• feat: Add blob scalar type (#2091)
• feat: Add unique secondary index (#2131)
• feat: Allow users to add Views (#2114)
• feat: Mutation typed input (#2167)
• feat: New cmd for docs auto generation (#2096)

Fix

• fix: Add entropy to counter CRDT type updates (#2186)
• fix: Filtering on unique index if there is no match (#2177)
• fix: Handle multiple nil values on unique indexed fields (#2178)

Refactoring

• refactor: Add strong typing to document creation (#2161)
• refactor: Rename key,id,dockey to docID terminology (#1749)
• refactor: Simplify Merkle CRDT workflow (#2111)

Testing

• test: Add auto-doc generation (#2051)

v0.8.0 Release

DefraDB v0.8 is a major pre-production release. Until the stable version 1.0 is reached, the SemVer minor patch number will denote notable releases, which will give the project freedom to experiment and explore potentially breaking changes.

To get a full outline of the changes, we invite you to review the official changelog below. This release does include a Breaking Change to existing v0.7.x databases. If you need help migrating an existing deployment, reach out at [email protected] or join our Discord at https://discord.source.network/.

Changelog

Features

• feat: Add CCIP Support (#1896)
• feat: Add OpenAPI route (#1960)
• feat: Add Swagger UI to playground (#1979)
• feat: Add means to fetch schema (#2006)
• feat: Allow setting of default schema version (#1888)
• feat: Change GetCollectionBySchemaFoo funcs to return many (#1984)
• feat: Enable playground in Docker build (#1986)
• feat: Make queries utilise secondary indexes (#1925)
• feat: Remove CollectionDescription.Schema (#1965)
• feat: Remove collection from patch schema (#1957)
• feat: Rename Schema.SchemaID to Schema.Root (#2005)

Fix

• fix: Fix test module relying on closed memory leak (#2037)
• fix: Infinite loop when updating one-one relation (#1915)
• fix: Make return type for FieldKind_INT an int64 (#1982)
• fix: Node private key requires data directory (#1938)
• fix: Remove collection name from schema ID generation (#1920)

Refactoring

• refactor: Add peerstore to multistore (#1980)
• refactor: CLI client interface (#1839)
• refactor: CRDT merge direction (#2016)
• refactor: Deprecate CollectionDescription.Schema (#1939)
• refactor: P2P client interface (#1924)
• refactor: Remove net GRPC API (#1927)
• refactor: Reorganise collection description storage (#1988)

v0.7.0 Release

DefraDB v0.7 is a major pre-production release. Until the stable version 1.0 is reached, the SemVer minor patch number will denote notable releases, which will give the project freedom to experiment and explore potentially breaking changes.

This release has focused on robustness, testing, and schema management. Some highlight new features include notable expansions to the expressiveness of schema migrations.

To get a full outline of the changes, we invite you to review the official changelog below. This release does include a Breaking Change to existing v0.6.x databases. If you need help migrating an existing deployment, reach out at [email protected] or join our Discord at https://discord.source.network/.

Features

• Allow field indexing by name in PatchSchema (#1810)
• Auto-create relation id fields via PatchSchema (#1807)
• Support PatchSchema relational field kind substitution (#1777)
• Add support for adding of relational fields (#1766)
• Enable downgrading of documents via Lens inverses (#1721)

Fixes

• Correctly handle serialisation of nil field values (#1872)
• Compound filter operators with relations (#1855)
• Only update updated fields via update requests (#1817)
• Error when saving a deleted document (#1806)
• Prevent multiple docs from being linked in one one (#1790)
• Handle the querying of secondary relation id fields (#1768)
• Improve the way migrations handle transactions (#1737)

Tooling

• Add Akash deployment configuration (#1736)

Refactoring

• HTTP client interface (#1776)
• Simplify fetcher interface (#1746)

Testing

• Convert and move out of place explain tests (#1878)
• Update mutation tests to make use of mutation system (#1853)
• Test top level agg. with compound relational filter (#1870)
• Skip unsupported mutation types at test level (#1850)
• Extend mutation tests with col.Update and Create (#1838)
• Add tests for multiple one-one joins (#1793)

v0.6.0 Release

DefraDB v0.6 is a major pre-production release. Until the stable version 1.0 is reached, the SemVer minor patch number will denote notable releases, which will give the project freedom to experiment and explore potentially breaking changes.

There are several new and powerful features, important bug fixes, and notable refactors in this release. Some highlight features include: The initial release of our LensVM based schema migration engine powered by WebAssembly (#1650), newly embedded DefraDB Playround which includes a bundled GraphQL client and schema manager, and last but not least a relation field (_id) alias to improve the developer experience (#1609).

To get a full outline of the changes, we invite you to review the official changelog below. This release does include a Breaking Change to existing v0.5.x databases. If you need help migrating an existing deployment, reach out at [email protected] or join our Discord at https://discord.source.network/.

Features

• Add _not operator (#1631)
• Schema list API (#1625)
• Add simple data import and export (#1630)
• Playground (#1575)
• Add schema migration get and set cmds to CLI (#1650)
• Allow relation alias on create and update (#1609)
• Make fetcher calculate docFetches and fieldFetches (#1713)
• Add lens migration engine to defra (#1564)
• Add _keys attribute to selectNode simple explain (#1546)
• CLI commands for secondary indexes (#1595)
• Add alias to groupBy related object (#1579)
• Non-unique secondary index (no querying) (#1450)
• Add ability to explain-debug all nodes (#1563)
• Include dockey in doc exists err (#1558)

Fixes

• Better wait in CLI integration test (#1415)
• Return error when relation is not defined on both types (#1647)
• Change core.DocumentMapping to pointer (#1528)
• Fix invalid (badger) datastore state (#1685)
• Discard index and subscription implicit transactions (#1715)
• Remove duplicated peers in peerstore prefix (#1678)
• Return errors from typeJoinOne (#1716)
• Document change detector breaking change (#1531)
• Standardise schema migration CLI errors (#1682)
• Introspection OrderArg returns null inputFields (#1633)
• Avoid duplicated requestable fields (#1621)
• Normalize int field kind (#1619)
• Change the WriteSyncer to use lock when piping (#1608)
• Filter splitting and rendering for related types (#1541)

Documentation

• Improve CLI command documentation (#1505)

Refactoring

• Schema list output to include schemaVersionID (#1706)
• Reuse lens wasm modules (#1641)
• Remove redundant txn param from fetcher start (#1635)
• Remove first CRDT byte from field encoded values (#1622)
• Merge node into net and improve coverage (#1593)
• Fetcher filter and field optimization (#1500)

Testing

• Rework transaction test framework capabilities (#1603)
• Expand backup integration tests (#1699)
• Disable test (#1675)
• Add tests for 1-1 group by id (#1655)
• Remove CLI tests from make test (#1643)
• Bundle test state into single var (#1645)
• Convert explain group tests to new explain setup (#1537)
• Add tests for foo_id field name clashes (#1521)
• Resume wait correctly following test node restart (#1515)
• Require no errors when none expected (#1509)

Continuous integration

• Add workflows to push, pull, and validate docker images (#1676)
• Build mocks using make (#1612)
• Fix terraform plan and merge AMI build + deploy workflow (#1514)
• Reconfigure CodeCov action to ensure stability (#1414)

Chore

• Bump to GoLang v1.20 (#1689)
• Update to ipfs boxo 0.10.0 (#1573)

v0.5.1 Release

DefraDB v0.5.1 is a minor release, primarily focusing on additional/extended features and fixes of items added in the v0.5.0 release.

Features

• Add collection response information on creation (#1499)
• CLI client request from file (#1503)
• Add commits fieldName and fieldId fields (#1451)
• Add allowed origins config (#1408)
• Add descriptions to all system defined GQL stuff (#1387)
• Strongly type Request.Errors (#1364)

Fixes

• Skip new test packages in change detector (#1495)
• Make nested joins work correctly from primary direction (#1491)
• Add reconnection to known peers (#1482)
• Rename commit field input arg to fieldId (#1460)
• Reference collectionID in p2p readme (#1466)
• Handling SIGTERM in CLI start command (#1459)
• Update QL documentation link and replicator command (#1440)
• Fix typo in readme (#1419)
• Limit the size of http request bodies that we handle (#1405)
• Improve P2P event handling (#1388)
• Serialize DB errors to json in http package (#1401)
• Do not commit if errors have been returned (#1390)
• Unlock replicator lock before returning error (#1369)
• Improve NonNull error message (#1362)
• Use ring-buffer for WaitForFoo chans (#1359)
• Guarantee event processing order (#1352)
• Explain of _group with dockeys filter to be []string (#1348)

Refactoring

• Use int32 for proper gql scalar Int parsing (#1493)
• Improve rollback on peer P2P collection error (#1461)
• Improve CLI with test suite and builder pattern (#928)

Testing

• Add DB/Node Restart tests (#1504)
• Provide tests for client introspection query (#1492)
• Convert explain count tests to new explain setup (#1488)
• Convert explain sum tests to new explain setup (#1489)
• Convert explain average tests to new explain setup (#1487)
• Convert explain top-level tests to new explain setup (#1480)
• Convert explain order tests to new explain setup (#1478)
• Convert explain join tests to new explain setup (#1476)
• Convert explain dagscan tests to new explain setup (#1474)
• Add tests to assert schema id order independence (#1456)
• Capitalize all integration schema types (#1445)
• Convert explain limit tests to new explain setup (#1446)
• Improve change detector performance (#1433)
• Convert mutation explain tests to new explain setup (#1416)
• Convert filter explain tests to new explain setup (#1380)
• Retry test doc mutation on transaction conflict (#1366)

Continuous integration

• Remove secret ssh key stuff from change detector wf (#1438)
• Fix the SSH security issue from AMI scan report (#1426)
• Add a separate workflow to run the linter (#1434)
• Allow CI to work from forked repo (#1392)
• Bump go version within packer for AWS AMI (#1344)

Chore

• Enshrine defra logger names (#1410)
• Remove some dead code (#1470)
• Update graphql-go (#1422)
• Improve logging consistency (#1424)
• Makefile tests with shorter timeout and common flags (#1397)
• Move to gofrs/uuid (#1396)
• Move to ipfs boxo (#1393)
• Document collection.txn (#1363)

dev-unstable-12-05-2023

THIS IS A UNSTABLE DEV BUILD. DO NOT USE UNLESS SPECIFICALLY DIRECTED TO.

Built using commit: 8aa317a
Built on date: 12-05-2023.

Unstable pre-release of upcoming 0.5.1