Releases: clockworklabs/SpacetimeDB
v1.0.0 Release Candidate 1
🚀 It's an extremely exciting day! 🚀
We are releasing the first release candidate of 1.0 SpacetimeDB!
In this version of SpacetimeDB, we do not expect to introduce any breaking changes to the user facing API. You can rest assured that if you build on this version it will be fully compatible or very nearly fully compatible with the final 1.0 release API.
While we reserve the right to add new APIs and make small changes before 1.0, this is more or less what 1.0 will look like!
NOTE! This is still a pre-release version. It is not the final 1.0 version of SpacetimeDB. As such we still suggest that developer exersize caution about using SpacetimeDB in production and be aware of the risks of pre-release software. Including the potential need to wipe their database instances or to manually migrate their data to the final 1.0 version.
SpacetimeDB identities are now OIDC compliant! This means that you'll have access to a huge host of existing tooling to create identities for your users and allow them to log in to your module through your own login flows and infrastructure.
CLI Improvements
- Login Flow Updates: We've dramatically improved how
Identity
s work in SpacetimeDB, including allowing you to use the sameIdentity
across multiple SpacetimeDB clusters by logging into the website. This includes introducing new CLI login flow, includinglogin show
andlogout
commands. spacetime upgrade
Update: Now uses--yes
instead of the deprecated--force
flag.- Help Text Enhancements: Added help text for
--build-options
. - Server Command Stabilization: Stabilized
spacetime server
subcommands and resolved issues withspacetime server edit
.
Database Enhancements
- Renaming: "Database instance" renamed to "replica" for clearer terminology.
- Row-Level Security (RLS): Added a system table for RLS along with a new filter macro.
- Optimized Querying: Added non-unique index join iterator and a physical query plan with executors for performance improvements.
- Persistent Memory Metering: Enhanced energy metering for persistent memory and improved handling for large datasets.
Compression
- New Compression Options: Introduced gzip and none compression options; SDK now selects optimal compression.
Code and Dependency Updates
- C# Bindings: Bumped C# bindings to version 1.0.0, with improved codegen for managing
DbConnection
states. - Code Cleanup: Removed obsolete elements like
BytesWrapper
and improved commit log traversal efficiency. - Dependency Upgrades: Updated Wasmtime dependency; improved resilience of ARM64/AMD64 Docker builds.
Security and Identity Management
- JWT Handling: Enhanced JWT token handling with OIDC providers.
- Endpoint and Token Updates: Removed email recovery endpoints, deprecated SendGrid, and updated token formats for ID tokens.
- Short-Lived Token Fixes: Corrected the endpoint used for short-lived tokens.
What's Changed
- CLI -
spacetime upgrade
takes--yes
instead of deprecated--force
by @bfops in #1805 - Renamed database instance to replica by @cloutiertyler in #1806
- offset index read integration by @Shubham8287 in #1779
- commitlog: Yield
StoredCommit
in iterators by @kim in #1791 - Make smoketests not log expected teardown errors by @kazimuth in #1804
- Add gzip + none compression algos and let SDK pick compression by @Centril in #1802
- Correct
Schema::check_compatible
to resolve types in a ModuleDef by @kazimuth in #1813 - Add CODEOWNERS for CLI crate by @bfops in #1834
- Make ARM64 and AMD64 docker builds more resilient by @jdetter in #1844
- Adding the system table for row level security by @mamcx in #1746
- refactor: Add TableId to relvar type by @joshua-spacetime in #1803
- Remove email and recovery related identity endpoints by @jsdt in #1833
- Energy metering for persistent memory usage by @coolreader18 in #766
- No
Map
s for you! by @Centril in #1770 - CLI - Add helptext for
--build-options
by @bfops in #1853 - Upgrade wasmtime dependency by @coolreader18 in #1857
- Remove sendgrid by @kim in #1860
- Improve bindings diagnostics and add ui tests by @coolreader18 in #1216
- Fix Build: Use default() to init custom hash based datastructures by @Shubham8287 in #1871
- RLS: Adding a new filter! macro by @mamcx in #1849
- CLI - Fix
spacetime server edit
failing on renames by @bfops in #1877 - CLI - Stabilize
spacetime server
subcommands by @bfops in #1845 - Redefine Address as
U128
+Identity
and Hash asU256
by @Centril in #1616 - Have the scheduled_id/at columns be specified in the schema, not by the column name by @coolreader18 in #1861
- Use new token format when generating new id tokens by @jsdt in #1854
- Rust modules: Revise
BTreeIndexBounds
by @gefjon in #1815 - commitlog: Fix transactions iterator by @kim in #1884
- Remove CODEOWNERS from CLI codegen by @bfops in #1886
- The banishment of Address by @cloutiertyler in #1880
- CLI - Fix helptext after #1845 by @bfops in #1889
- Handle JWT tokens with oidc providers by @jsdt in #1882
- Reorganize macro crate by @coolreader18 in #1406
- Physical query plan + executors by @joshua-spacetime in #1881
- Small naming fixes. Fixes database identity creation. by @cloutiertyler in #1892
- feat: Add non-unique index join iterator by @joshua-spacetime in #1908
- Fix the endpoint for short-lived tokens by @jsdt in #1907
- core: Expose snapshot taking on
Locking
datastore by @kim in #1898 - core: Expose
next_tx_offset
of theReplay
decoder by @kim in #1899 - core: Conversion from
txdata::Inputs
toReducerContext
by @kim in #1901 - commitlog: Improve skipping behavior of traversals by @kim in #1902
- NFC: Remove obsolete BytesWrapper by @RReverser in #1896
- Bump C# bindings to 1.0.0 by @bfops in #1913
- Put
ExecutionContext
inside the Tx context by @mamcx in #1876 - Don't redownload WASI SDK by @RReverser in #1916
- perf: Use default buffer size for brotli compression by @joshua-spacetime in #1920
- CLI - Login flow by @bfops in #1878
- Fix Find() with struct-based rows by @RReverser in #1917
- Hide "internal" datetime representation types by @RReverser in #1921
- Companion to spacetime-web#470 by @PuruVJ in #1912
- C# codegen making state relative to a DbConnection by @lcodes in #1869
- Optimise C# in a ridiculous way by @RReverser in #1924
- CLI - Add
login show
andlogout
by @bfops in #1911 - Cached query execution plan by @joshua-spacetime in #1923
- chore: remove dead code by @joshua-spacetime in #1930
- Use previous reducer context when downgrading a tx by @joshua-spacetime in #1932
- Websocket API: Light transaction updates &
NoSuccessNotify
by @Centril in #1812
New Contributors
Full Changelog: v0.12.0-beta-hotfix3...v1.0.0-rc1-hotfix1
v0.12.0-beta
🌟 It's happening! 🌟
We've got a big one for you today! We've been burning the midnight oil to bring you the cream of the SpacetimeDB crop. We are updating and improving our APIs and interfaces across the board in an effort to stabilize and polish them up for a final release!
SpacetimeDB is getting closer and closer to a final 1.0 release and this is a huge milestone on that journey. These new APIs will allow us to implement some amazing new features and vastly improve the user experience of using SpacetimeDB across multiple languages and clients.
Module API
The module API has had it's most major overhaul since the announcement of SpacetimeDB. We're reducing the amount of globals and enforcing accessing the database via a ReducerContext
.
Rust
#[reducer]
pub fn add(ctx: &ReducerContext, name: String) {
println!("Inserting {}", name);
ctx.db.person().insert(Person { name });
}
CHECK IT OUT! The
#[spacetimedb(reducer)]
macro has been simplified to just#[reducer]
!
C#
[SpacetimeDB.Reducer]
public static void Add(ReducerContext ctx, string name)
{
Log.Info($"Inserting {name}");
ctx.Db.Person.Insert(new Person { Name = name });
}
SDK API
The SDKs now share almost exactly the same API as you use to access your data inside your module.
let ctx = DbConnection.builder()
.with_url("https://testnet.spacetimedb.com")
.with_module_name("bitcraft")
.build();
...
for person in ctx.db.person().iter() {
println!("Hey, {}", person.name);
}
C#
var conn = DbConnection.Builder()
.WithUri("https://testnet.spacetimedb.com")
.WithModuleName("bitcraft")
.Build();
...
foreach (var person in ctx.Db.Person.Iter())
{
Log.Info($"Hello, {person.Name}!");
}
Now you can write your client code with the same patterns as your server code!
Migration Guide
Check out our full migration guide to update your code for v0.12 available in the docs section of our website.
CLI updates 🖊️
We've updated a lot of the args to the CLI to be more clear and consistent. If something gives you an errors, check --help
to see if it's changed.
Some of the important changes:
spacetime local clear
(to delete all local server data) is now calledspacetime server clear
spacetime build --skip_clippy
/-S
is now more explicitly--skip-println-checks
spacetime logs database 10
is nowspacetime logs database -n 10
spacetime publish
andspacetime generate
now accept a--build-options
param, e.g.spacetime publish --build-options="--debug --skip-println-checks"
Small clarity changes:
- All
--anon-identity
parameters have been renamed to--anonymous
spacetime energy status
is nowspacetime energy balance
for clarityspacetime publish --clear-database
has been renamed tospacetime publish --delete-data
for clarityspacetime subscribe -n
now has a longform option--num-updates
spacetime publish|generate --wasm-file
->spacetime publish|generate --bin-path
What's Changed (It's... uhh a lot)
- Fix flaky subscription smoketests by @bfops in #1536
- core: Start system table ids at 1 by @kim in #1544
- Remove support for the
update
reducer by @kim in #1557 - Get rid of python SDK by @Centril in #1561
- client-api: Remove publisher address by @kim in #1564
- Minor NFC improvements to C# Roslyn codegen by @RReverser in #1562
- SATS: Flatten
AlgebraicType
, getting rid ofBuiltinType
by @Centril in #1559 - Updating the change date in anticipation of the anniversary by @cloutiertyler in #1574
- Update README.md by @cloutiertyler in #1575
- Make
schema
crate publishable by @bfops in #1569 Typespace::is_nominal_normal_form
by @kazimuth in #1549- Add lib::db::{column_ordering, raw_def::v9} by @kazimuth in #1542
- Change ResolveRefs to return a Result rather than an Option by @kazimuth in #1565
- Add support for
I256
andU256
by @Centril in #1477 - core: Use
Program
instead of(Hash, Box<[u8]>)
by @kim in #1558 - Bump version to 0.12.0 by @bfops in #1578
- Fix iu BSATN implementations in C# by @RReverser in #1582
- C#: disable verbose test output in console by @RReverser in #1570
- Add helpers and change
CollectAllErrors
to be more composable by @kazimuth in #1586 - Preliminary Identifier validation by @kazimuth in #1584
- Allow empty
ColList
by @Centril in #1588 - [CI] Do not use branch names when publishing container images by @kurtismullins in #1583
- Verify Unity compatibility in the shared C# codegen by @RReverser in #1576
- Fix builds for MacOS amd64 platform by @kurtismullins in #1598
- [WASM ABI 1.0] Change
ColId
fromu32
tou16
by @Centril in #1597 - [WASM ABI 1.0]
__call_reducer__
receives Identity & Address by value by @Centril in #1607 - [WASM ABI 1.0] impl
__call_reducer__
usingbytes_source_read
by @Centril in #1609 - Placate some clippy errors in
snapshot
by @kazimuth in #1611 - fix: Always remember to release a read only tx by @joshua-spacetime in #1613
- NFC: enforce style for C# locally and on CI by @RReverser in #1567
- Make
ScheduleAt
special +Typespace::is_valid_for_client_code_generation
by @kazimuth in #1590 - Dedup schedueler queue by @Shubham8287 in #1587
- Updated docker internal port to 3000 by @jdetter in #1626
- Add volatile_nonatomic_schedule_immediate by @coolreader18 in #1612
- NFC: use record for BigInt equality and hash by @RReverser in #1633
- smoketests: Test unauthorized publish without -c by @kim in #1632
- [WASM ABI 1.0] impl
__call_reducer__
&__describe_module__
usingbytes_sink_write
by @Centril in #1615 - ABI v9 validation code by @kazimuth in #1572
- Fix ptr mutability of some new C# FFIs by @RReverser in #1645
- WASM ABI: implement
row_iter_bsatn_advance
&row_iter_bsatn_close
by @Centril in #1622 - WASM ABI:
get_table_id
->table_id_from_name
by @Centril in #1634 - WASM ABI: add
datastore_table_row_count
by @Centril in #1636 - [WASM ABI 1.0] Fix
bytes_source_read
by @kim in #1646 - Add backwards-compatible validation for RawModuleDefV8. by @kazimuth in #1606
- Allow converting new ModuleDef to old TableSchema by @kazimuth in #1630
- Fix flaky smoketest due to race condition in subscribe by @coolreader18 in #1656
- Update SQL AST in accordance with the SQL spec by @mamcx in #1623
- WASM ABI:
iter_start
->datastore_table_scan_bsatn
by @Centril in #1637 - WASM ABI:
delete_by_rel
->datastore_delete_all_by_eq_bsatn
by @Centril in #1638 - Add edition to .rustfmt.toml by @gefjon in #1663
- bindings-csharp: pass
address_1
by @Centril in #1621 - C#: split table codegen logic from type codegen logic by @RReverser in #1573
- WASM ABI:
insert
->datastore_insert_bsatn
& impl new semantics by @Centril in #1639 - WASM ABI: implement console ABIs by @Centril in #1664
- C# module benchmarks by @RReverser in #1679
- Auto-generate C# ModuleDef bindings from Rust by @RReverser in #1680
- Impl macro rework by @coolreader18 in #1314
- commitlog: Make commit module pub...
v0.11.1-beta
What's this? What's this? A release in the air!
This release just adds two small improvements to the timer table functionality from the last release:
Don't be fooled by this tiny release, though - our next release will be a big one! Keep an eye out for the big API improvements and upcoming stability soon! Before you know it, we'll have our big 1.0 release!
Changelog
Full Changelog: v0.11.0-beta...v0.11.1-beta
v0.11.0-beta
We have a big release for you today 🎉 🎉 ! We're including some big performance improvements ⚡ and long-awaited bugfixes 🐞! Remember to stay tuned for our 0.12 release soon.
This release includes some major breaking changes, so you won't be able to use your old databases with this version. Run spacetime local clear
to remove them.
Important changes
Major
- ⚡ We removed our usage of protobuf, in favor of our own internal format (#1077) - this is a big performance improvement on the client and the server!
- ⌛ We removed the server-side
schedule!
macro and its equivalents in favor of table-based timers (#1449). Please check our server module docs for more info on how to use the new system.
Here is some example code which uses the new timer tables:
// The `scheduled` attribute links this table to a reducer.
#[spacetimedb(table, scheduled(send_message))]
struct SendMessageTimer {
text: String,
}
// Reducers linked to the scheduler table should have their first argument as `ReducerContext`
// and the second as an instance of the table struct it is linked to.
#[spacetimedb(reducer)]
fn send_message(ctx: ReducerContext, arg: SendMessageTimer) -> Result<(), String> {
// ...
}
// Scheduling reducers inside `init` reducer
fn init() {
// Scheduling a reducer for a specific Timestamp
SendMessageTimer::insert(SendMessageTimer {
scheduled_id: 0,
text:"bot sending a message".to_string(),
//`spacetimedb::Timestamp` implements `From` trait to `ScheduleAt::Time`.
scheduled_at: ctx.timestamp.plus(Duration::from_secs(10)).into()
});
// Scheduling a reducer to be called at fixed interval of 100 milliseconds.
SendMessageTimer::insert(SendMessageTimer {
scheduled_id: 0,
text:"bot sending a message".to_string(),
//`std::time::Duration` implements `From` trait to `ScheduleAt::Duration`.
scheduled_at: duration!(100ms).into(),
});
}
CLI changes
- CLI subcommands consistently use
--server
and--identity
args consistently instead of anonymous args (#1482) - We have a new
subscribe
subcommand! (#1343) - Bugfix: Update help text suggesting
spacetime server fingerprint
to have the correct-s
param (#1457) - Bugfix:
spacetime server add
- Remove trailing/
s from server URIs (#1552)
Web API
- Recovery code APIs are now under /identity and using POST (#1492)
- /identity GET returns an empty success if the email address is not found (#1491)
C#
- Use latest C# in BSATN.Runtime (#1548)
- Fixed exceptions in C# SDK when someone disconnects or when a transaction originates from CLI (#1461)
- Include BSATN.Codegen in nuget pack (#1424)
- Generate tagged enums in C# client code (#1421)
- Restructure NuGet packaging (#1440)
- Restructure C# SpacetimeDB runtime (#1455)
- Implement the module rng proposal for C# (#1425)
- Roslyn cacheability testing and fixes (#1420)
Using SpacetimeDB as a library
- table: Make
with_mut_schema
clone-on-write (#1530) - Fix running 'cargo test' in crates/lib (#1478)
- Move
db
module fromspacetimedb_sats
tospacetimedb_lib
(#1479) - Move schemas to
schema
crate, renameDef
toRawDefV8
(#1498) - core: Simplify custom bootstrap (#1404)
TableDef
: clarifygenerated_*
methods (#1419)- Make some commitlog helpers public (#1390)
- perf(1351): Add a row count metric for subscriptions (#1435)
Bugfixes
- ST sequences: respect allocated amount on restart (#1532)
- Drain scheduler Actor channel before start (#1529)
Table::is_row_present
: don't panic (#1526)- Ignore some sequence allocation mismatches when checking for compatible updates (#1524)
- Fix inconsistent auth/identity creation (#735)
ColList
: preserve list order onlist.push(..)
(#1474)- HACK: Tweak
schema_updates
to allow adding/removing non-unique indices (#1434) - Fix index removal and additions + add smoketest (#1444)
- fix(1409): Counter metric names (#1411)
- core: Downgrade host log verbosity (#1446)
- Implement a temporary type check validation on sql compiling (#1456)
- core: Replace host scheduler on update (#1453)
Changelog
- Remove unused file table.rs by @Centril in #1417
- fix(1409): Counter metric names by @joshua-spacetime in #1411
- chore: Remove already tracked subscription counter by @joshua-spacetime in #1412
- refactor: Record txn-level metrics in a single utility by @joshua-spacetime in #1414
- Generate tagged enums in C# client code by @RReverser in #1421
- Include BSATN.Codegen in nuget pack by @kurtismullins in #1424
TableDef
: clarifygenerated_*
methods by @Centril in #1419- Add small 'realistic' workload benchmark integration test by @mamcx in #714
- Impl subscribe subcommand & subscription smoketests by @coolreader18 in #1343
- HACK: Tweak
schema_updates
to allow adding/removing non-unique indices by @Centril in #1434 - Update tools/publish-crates.sh by @bfops in #1426
- refactor: Reconfigure histogram buckets for certain metrics by @joshua-spacetime in #1436
- perf(1351): Add a row count metric for subscriptions by @joshua-spacetime in #1435
- Implement the module rng proposal for C# by @RReverser in #1425
- Bump version to 0.10.1 by @bfops in #1443
- Roslyn cacheability testing and fixes by @RReverser in #1420
- Restructure NuGet packaging by @RReverser in #1440
- Fix index removal and additions + add smoketest by @Centril in #1444
- core: Downgrade host log verbosity by @kim in #1446
- Smoketest auto-disconnect after restart by @kim in #1367
- core: Simplify custom bootstrap by @kim in #1404
- core: Replace host scheduler on update by @kim in #1453
- CLI - Update help text suggesting
spacetime server fingerprint
to have the correct-s
param by @bfops in #1457 - Fix inconsistent auth/identity creation by @coolreader18 in #735
- Fixed exceptions in C# SDK when someone disconnects or when a transaction originates from CLI by @SteveBoytsun in #1461
- Ensure MacOS builds from CI are executable by @kurtismullins in #1462
- Automated Discord post when a PR merges by @bfops in #1470
- Fix C# module smoketests by @bfops in #1475
- smoketests: Add test for module hotswapping by @kim in #1403
ColList
: preserve list order onlist.push(..)
by @Centril in #1474- CLI: Put some common params in a central place by @bfops in #1484
- Implement a temporary type check validation on sql compiling by @mamcx in #1456
- Make some commitlog helpers public by @lcodes in #1390
- Adding benchmark for deserialize json & product value by @mamcx in #1035
- CLI: Use
--server
and--identity
args consistently instead of anonymous args by @bfops in #1482 - Fix running 'cargo test' in crates/lib by @kazimuth in #1478
- Restructure C# SpacetimeDB runtime by @RReverser in #1455
- Protobufectomy: server by @coolreader18 in #1077
- refactor(1494): Remove SQL DDL per 1.0 SQL spec by @joshua-spacetime in #1499
- C# smoketests use
nuget.config
by @bfops in #1500 - CI - SpacetimeDB PRs run the C# SDK tests by @bfops in #1503
- Timer Table Implementation by @Shubham8287 in #1449
- CI - Post-to-discord workflow only fires if the PR merged to
master
by @bfops in #1520 - Return empty success from /identity GET by @lcodes in #1491
- Moving recovery code APIs under /identity and using POST by @lcodes in #1492
Table::is_row_present
: don't panic by @gefjon in #1526- Move
db
module fromspacetimedb_sats
tospacetimedb_lib
by @kazimuth in #1479 - Ignore some sequence allocation mismatches whe...
v0.10.1-beta
This is just a tiny fix in our C# SDK. 🔧
Stay tuned for our upcoming 0.11 release soon! 👀
What's changed
- Fixed exceptions in C# SDK when someone disconnects or when a transaction originates from CLI by @SteveBoytsun in #1461
Full Changelog: v0.10.0-beta...v0.10.1-beta
v0.10.0-beta
Yes, the rumors are true - it's release day! 🎉 🎉
We're gearing up for our big 1.0 release later this year, so this release has some great foundational API improvements, improves consistency across ours APIs, and adds some other fantastic foundational features!
Highlights
- Bumped to Rust 1.78
- We now regularly capture a snapshot of the database state, and restore from it when restarting the Database
- Tables are now private by default, and must be explicitly marked as
public
via#[spacetimedb(table(public))]
or[SpacetimeDB.Table(Public = true)]
- Our SDKs now have consistent
filter_by_*
functions that return a collection of results, andfind_by_*
functions that return a single result - Implement a new rand api which fixes bugs in our previous random number generator
- Bugfix: When the database is restarted, we properly treat all clients as disconnected
Big ones!
- Module hotswapping - You can now update your server logic with a new module without disconnecting existing clients! Players will keep on playing and clients will keep on clienting without being any the wiser!
- Recv-style ABI - We've improved the performance of passing data into your module from the host substantially by eliminating several allocations and copies.
CLI changes
spacetime build
uses the--project-path
param instead of an anonymous paramspacetime publish -c
requires confirmation (can be overridden with--force
)spacetime logs -f
does not start from the very beginning
What's Changed
impl PartialEq<ProductValue> for RowRef
by @Centril in #1164- Use a custom
FixedBitSet
+ optimizePage::iter_fixed_len
by @Centril in #1160 - Implement
RelValue: Eq + Hash
by @Centril in #1107 - Fix soundness hole in
Table::delete
+ don't make & immedately drop PVs in the method by @Centril in #1162 - Add PR status check that fails unless based on
master
by @bfops in #1180 - Some clarifications to dropping the updates in eval incr by @Centril in #1192
- perf(
JoinInner
): use AVs for keys instead of PVs by @Centril in #1194 - Make
Page
always fully init by @gefjon in #1193 - Remove C# GetArgsAsObjectArray by @RReverser in #1188
- feat(1168): Track subscription query duration for each reducer by @joshua-spacetime in #1195
- Fix unsoundness in
AlgebraicValue::type_of
by making it partial by @Centril in #1189 - From.{join => joins} + remove Option layer + Remove RelOps::try_fold by @Centril in #1208
- subscription benches: reorder footprint fields by @Centril in #1209
- C#: rename DbEventArgs to ReducerContext by @RReverser in #1191
- Bump to Rust 1.78 by @coolreader18 in #1205
spacetime build
uses--project-path
param by @jdetter in #1212- Remove some impls that were dead code by @Centril in #1210
- CrudExpr::{Delete, Insert, Update}: only DbTables are possible, so cleanup InMem path by @Centril in #1211
- Remove RowUpdate event from C# by @RReverser in #1190
- Switch to a better API for tagged enums for C# by @RReverser in #1177
- Remove a few dead impls by @Centril in #1214
- commitlog: Allow folds to not allocate
Mutations
values by @kim in #1215 - Fix #1180: Add PR status check that fails unless based on master by @bfops in #1213
- Split up #735:
auth.rs
refactors by @bfops in #871 - Create a lockfile when opening config files by @gefjon in #1196
- Bump version to 0.9.0 by @bfops in #1055
- Add CODEOWNERS for
rust-toolchain.toml
by @bfops in #1221 - Fix select * from * by @coolreader18 in #1218
- CLI - Small refactors to identity code by @bfops in #1169
- Add CODEOWNERS for
LICENSE.txt
by @bfops in #1225 - Add
CODEOWNERS
forCODEOWNERS
by @bfops in #1226 spacetime publish
: Add confirmation for-c
by @bfops in #1038- Make identity naming more consistent + fix bugs by @jdetter in #1140
- feat(1229): Double client channel capacity by @joshua-spacetime in #1230
- to_bsatn_extend/vec: use uninit instead of zeroed buffer by @Centril in #1204
- refactor compile_read_only_query: dedup SideEffect errors by @Centril in #1235
- Add test for
select * from B join A on B.y = A.x
by @Centril in #1238 - Always send TransactionUpdate to the sender by @drogus in #1111
- row_count field to table by @Shubham8287 in #1242
- Simplify
MutTxId::table_exists
by @Centril in #1246 - Remove unreachable branches from C# generate
by @RReverser in #1247 Table::get_fixed_row
->RowRef::get_row_data
+ Some docs by @Centril in #1250- Move
StaticBsatnLayout
code to its module + harden test by @Centril in #1254 - Remove
BTreeIndex::name
again by @Centril in #1251 - Atomically downgrade lock when committing tx to prevent deadlock by @coolreader18 in #1252
- Tiny change: slightly better logging of sled errors by @kazimuth in #1248
- Add
PageHeader::unmodified_hash
, a BLAKE3 hash for snapshotting by @Centril in #1249 - Privatize
Table::row_layout
+ relatedBTreeIndex
refactoring by @Centril in #1262 - core: Collapse DBIC into HostController by @kim in #1186
- Reserve sequence range for system tables by @kim in #1265
- Add
fn ColList::last(&self) -> ColId
by @Centril in #1267 - Misc datastore cleanups & less cloning by @Centril in #1263
- Module hotswapping by @coolreader18 in #1147
- Consistent filtering rules for Rust bindings by @RReverser in #1280
- recv-style abi by @coolreader18 in #1002
- feat(1231): Basic query cardinality estimation by @joshua-spacetime in #1273
- Simplify
btree_index
module with more idiomatic Rust by @Centril in #1285 - Disconnect dangling clients by @kim in #1132
- CI workflows support
workflow_dispatch
event by @bfops in #1289 - Remove
incremental-joins.md
by @bfops in #1296 - Add table smoketest + more smoketest logging by @kazimuth in #1272
- Fix
workflow_dispatch
for ci.yml by @bfops in #1295 - Shub/st connected clients by @Shubham8287 in #1288
- Let ProgramStorage::external be async by @coolreader18 in #1291
- due cleanup in datastore test by @Shubham8287 in #1301
- fix(1297):
spacetime logs -f
should not start at the very beginning by @bfops in #1298 merge_apply_inserts/deletes
: do metrics work once per table, not per row by @Centril in #1286- Consistent filtering in Rust client + minor fixes by @RReverser in #1275
- Implement consistent filtering rules for C# modules by @RReverser in #1282
- Update C# codegen to consistent filtering rules by @RReverser in #1277
- Implement consistent filtering rules for TypeScript by @RReverser in https://github.com/clockworklabs/Sp...
v0.9.2-beta
Yes, we just released 0.9.1 🎉, but we just love releasing the latest and greatest things to the community!
And in this case, the latest and greatest thing is an 🔒 important security patchfix! Private tables are now properly protected.
Thank you to 📣 @Chippy
in our Discord server for pointing this out so we were able to fix it quickly!
Changelog
22dd786 Apply open PR #1224: Bump version to 0.9.2
e4f1021 Apply open PR #1274: execution_set.check_auth(...)
on initial subscription
Full Changelog: v0.9.1-beta...v0.9.2-beta
v0.9.1-beta
v0.9.1
Today's a good day - we have a new release for you! It's not the fanciest, shiniest release - but it includes some nice cleanups and important fixes.
As a reminder you can upgrade from the previous release with the
spacetime upgrade
command.
If you are on macOS and you've installed through brew we recommend usingbrew install clockworklabs/tap/spacetime
to upgrade.
Highlights
🗑️ Removed repeating reducers
#[spacetimedb(reducer, repeat = _)]
) in favor of just using schedule!
.
There are a lot of nuances to what exactly "repeating" means. It's easier for everyone to understand code that schedules what it wants, when it wants.
💻 CLI updates
testnet
instead of local
!
The meaning of -s
and -S
are more consistent between CLI commands (-s
is for --server
, -S
is for --skip_clippy
). spacetime build
, spacetime identity
, and spacetime server
have been tweaked.
🛠️ Bugfixes
- Fix
select * from *
- Atomically downgrade lock when committing tx to prevent deadlock
- fix(1170): Use scope guard to decrement reducer queue length
- fix(1173): Record wait time for all reducers
Changelog
- Table: skip alignment checks in eq_row_in_page and hash_row_in_page by @kazimuth in #1085
- core: Don't persist empty transactions by @kim in #1086
- Shrink AV by 8 bytes and AT by 16 by @Centril in #1047
- Fix iai-callgrind rustc version mismatch by @kazimuth in #1025
- Nuke
DataKey
🚀 by @Centril in #1093 - Simplify
ExecutionContext
constructors by @Centril in #1098 - perf: optimize
Table::clone_structure
by avoiding clones by @Centril in #1090 - Slow query log by @mamcx in #1052
- core: Store inputs (reducer info + args) in commitlog by @kim in #1091
- slow query log: only use
Instant::now
when needed + refactor by @Centril in #1110 - fix(1115): Identify subscribers by Identity AND Address by @joshua-spacetime in #1121
- Re-instantiate replay progress reporting by @kim in #1096
- cli: Suggest to make new identity the default for server by @kim in #1109
- commitlog: Support traversal without opening the log by @kim in #1103
- Default commitlog output to info instead of debug by @jdetter in #1123
- Nuke
#[spacetimedb(reducer, repeat = _)]
by @Centril in #1095 MultiMap
: don't sort values; use.push(..)
and.swap_remove(..)
instead by @Centril in #1029- core: AnyBytes from boxed slice by @kim in #1133
- Document RelValue variants by @Centril in #1134
- Build uses -S for skip clippy by @jdetter in #1129
- Use -s to specify server for spacetime identity list -s by @jdetter in #1130
- Fix an inconsistency with spacetime server fingerprint by @jdetter in #1131
- Remove
Column.col_id
field by @Centril in #1136 - Ignore discriminants in
Hash for AV + ArrayValue
+ nix len-prefix for PV by @Centril in #1057 - Reduce unwrap noise in CLI generate code by @RReverser in #1142
- Fix codegen tests by @RReverser in #1146
- Define
Hash + Eq
for BSATN by @Centril in #1112 - NFC: format C# module SDK code by @RReverser in #1149
impl Eq for RowRef
by @Centril in #1135- Tests use explicit
--server local
by @bfops in #1150 - CLI - Small refactor to default server configs by @bfops in #1151
- CLI - Set default server to
testnet
by @dylanh724 in #1078 - Fail CI on smoketest failures by @coolreader18 in #1155
- NFC: inline format args in
generate
code by @RReverser in #1153 - Dylan/doc: Link directly to cargo install on missing by @dylanh724 in #1102
- Simplify C# source generator csproj by @RReverser in #1143
- Ignore const members in C# types by @RReverser in #1175
- Disallow calling random lifecycle reducers by @coolreader18 in #1114
- chore: Remove unused server metrics by @joshua-spacetime in #1171
- fix(1170): Use scope guard to decrement reducer queue length by @joshua-spacetime in #1172
- fix(1173): Record wait time for all reducers by @joshua-spacetime in #1174
- NFC: few more C# shorthand syntax conversions by @RReverser in #1176
- Add
Hash for RowRef
+ make it consistent withHash for PV
by @Centril in #1163 - NFC: reuse common parts of C# generation by @RReverser in #1187
- Redefine FieldName as
(TableId, ColId)
by @Centril in #1165
New Contributors
- @dylanh724 made their first contribution in #1078
Full Changelog: v0.9.0-beta...v0.9.1-beta
v0.9.0-beta
v0.9 🚀🚀🚀
We have got a big one for you! Today we are releasing version 0.9 of SpacetimeDB. It's been a while since our last official release and we have got a huge number changes and performance improvements.
Most notably performance has improved by 10x - 100x or more for certain parts of the database! Subscription evaluation in particular has gotten almost 100 times faster! This makes an enormous difference for people shipping production games. 🏃🏃🏃
Version 0.9 is the same version that we ran the Closed Alpha for the MMOPRG BitCraft, which had 700 players playing concurrently in a single database! We are battle testing SpacetimeDB with our production game!
We're so excited to see what people build with this new version!
As a reminder you can upgrade from the previous release with the
spacetime upgrade
command. If you are on macOS and you've installed through brew we recommend usingbrew install clockworklabs/tap/spacetime
to upgrade.
Highlights
- Vastly improved metrics gathering performance overhead
- Stabilized new transaction log format for a 2-3x reduction in transaction log size and much faster reply times
- Reduced unnecessary copying and cloning in core database paths
- Improved threading model for optimal performance
- Fixed a critical issue for misbehaving clients or clients with very poor latency
- Multi-column index selection for the
query!
macro - Huge number of critical performance and bug fixes!
What's Changed
- Bump Rust to 1.76.0 by @coolreader18 in #811
- test: Add performance benchmark for incremental join query by @joshua-spacetime in #795
- perf(subscriptions): Add metric counter for subscribe calls by @joshua-spacetime in #817
- ResultInspectExt is obsolete by @coolreader18 in #818
- chore(datastore): Remove debug instrumentation by @joshua-spacetime in #812
- Fix benchmarks by @coolreader18 in #819
- This fixes replaying of the transaction log to no longer check constraints by @cloutiertyler in #806
ScanIterByColRange
: Avoid unnecessaryProductValue
allocations by @gefjon in #804- Update upgrade-version tool by @coolreader18 in #826
- chore(836): Remove tracing instrumentation from database iterators by @joshua-spacetime in #837
- test(829): Benchmark incremental update for table subscription by @joshua-spacetime in #830
- Verify that the upgrade-version tool still works for every PR by @jdetter in #825
- To make eval subscription concurrent by @Shubham8287 in #807
- Remove unused pre-publish.duck file by @bfops in #841
- Fix: Publish fails if rustup not present by @bfops in #814
- test: Deduplicate utilities for creating tables by @joshua-spacetime in #842
- Add workflow to tag latest docker image on release by @drogus in #756
- test(835): Benchmark full subscription eval for table scans and join by @joshua-spacetime in #839
- Core: depend on "serde" from lib by @gefjon in #845
- Reduce allocations for prometheus with_label_values by @coolreader18 in #810
- fix: Wrong algebraic type in query benchmarks by @joshua-spacetime in #849
- fix: Record durations for aborted and read transactions by @joshua-spacetime in #850
- Fix publish issues 0.8.2 by @jdetter in #853
- Fix identity subcommand ordering by @jdetter in #857
- v0.8.2 Version Upgrade by @jdetter in #858
- core: Ignore
indexed
-only constraints when updating by @kim in #734 - Use a version label for tag builds by @drogus in #751
- Fix issues caused by disabling metrics by @jdetter in #856
- Add
delete_by_{nonunique_column}
method to#[spacetimedb(table)]
by @gefjon in #859 - Fix deadlock by @coolreader18 in #854
- Fix bench errors and include in CI by @coolreader18 in #855
- Fix consistency issue by @coolreader18 in #861
- Refactor some ReadColumn stuff + relational_db tests by @Centril in #847
- Added extra check to create_table transactionality test by @cloutiertyler in #866
- system_tables, mut_tx, and friends: bye bye
to_product_value
by @Centril in #851 - Fix bootstrapping ids for relating objects like indexes by @mamcx in #777
- dedup some
with_label_values
+ reduce cloning in bootstrapping by @Centril in #867 - Set name for Rayon threads by @gefjon in #870
- Adds an isolation_level argument to begin_mut_tx calls by @cloutiertyler in #864
- chore: Remove instrumentation from DataKey deserializers by @joshua-spacetime in #852
- Replace DbProgram with a leaner version that compile directly to queries without environment by @mamcx in #860
- Use a copy-on-write structure for
RelValue
by @Centril in #869 - Fix module import warnings by @coolreader18 in #820
- Add GitHub workflow to check for PRs with merge-blocking labels by @bfops in #862
- Script to run perf against SpacetimeDB by @gefjon in #882
- HostThreadPool: don't run Rayon threads in Tokio blocking threads by @gefjon in #881
- Rename types in benchmarks by @kazimuth in #803
- num_threads = available_concurrency by @coolreader18 in #887
- Metric for request RTT by @Shubham8287 in #872
- serialise disconnect logic by @Shubham8287 in #886
- Run initial subscriptions evals on rayon by @coolreader18 in #888
- Revert "Run initial subscriptions evals on rayon" by @cloutiertyler in #894
- C#: reuse Consume helper by @RReverser in #704
- refactor(890): Improve magic constant for index join selection by @joshua-spacetime in #895
- VM: Wrap
Header
inArc
to avoid cloning by @gefjon in #897 - Parallelize QuerySet::eval by @coolreader18 in #891
- Remove
row_pk
from client API; hash rows not row_ids on client by @gefjon in #840 - perf(832): Remove redundant row deduplication in subscriptions by @joshua-spacetime in #863
- acquire remove subscription lock in spawn_blocking by @Shubham8287 in #903
- Adjust benchmarks yaml to run callgrind + with better comments by @kazimuth in #907
- perf(831): Remove row_pk computation from query by @joshua-spacetime in #908
- core: Fix host controller to not replace module if lifecyle hooks failed by @kim in #904
- Correctly show the error for AmbiguousField and simplify the code by @mamcx in #910
- Adding an index selector that take in account multi-column indexes (and improve the
query!
macro) by @mamcx in #694 - Revert 694 by @jdetter in #914
- Fix the RowCount estimation for select operator by @mamcx in #900
- Remove Tables from query plans by @gefjon in #912
iter_by_col_eq
: take AV by ref + impl RangeBounds<..> for &AV by @Centril in #925- perf(747): Single query execution for multiple subscriptions by @joshua-spacetime in #917
- Re-land mult-col index selection for queries by @cen...
v0.8.2-beta
Our release 0.8.2 is available now! It includes tons of performance improvements and some great bugfixes!
As a reminder you can upgrade from the previous release with the spacetime upgrade
command. If you are on macOS and you've installed through brew we recommend using brew install clockworklabs/tap/spacetime
to upgrade.
Highlights
- Memory architecture refactor: we're working on rearchitecting the way we store and retrieve data from the database. This is our first batch of improvements which reduces the amount of overhead when reading values from the database. This affects both subscription query performance and module performance.
- Tons of performance improvements
- Incremental subscription queries are now evaluated in parallel
- Join evaluation performance improvements
- Lookups on an indexed column are faster
- Fixed
spacetime generate --delete-files
works properly without using--force
- Bug fixes
- Fixed correctness issues with subscription queries
- Fixed some issues related to unwrapping on None
What's Changed
- core: Sort constraint definitions before comparing by @kim in #727
- Allows to convert into/from micros_since_epoch:u64 for TimeStamp by @mamcx in #731
- Fix callgrind action again by @kazimuth in #718
- Fix local callgrind runs by @kazimuth in #723
- Expand callgrind benches by @kazimuth in #728
- Version upgrade to 0.8.1 by @jdetter in #729
- Fix _delete_by_rel by @coolreader18 in #733
- Add
ColList
: a compact replacement ofNonEmpty<ColId>
, 8 vs 32 bytes by @Centril in #730 - feat: subscription to use read type tx by @Shubham8287 in #685
- Fix --delete-files command without using --force by @jdetter in #739
- Use
ColList
more and improve it by @Centril in #738 - Fix for reducer queue metric by @jdetter in #737
- Fix row count for the 'sql' command by @mamcx in #745
- Cleanup SpacetimeDB name usages by @jdetter in #744
- SATS: add support needed for BFLATN => BSATN directly by @Centril in #750
- Check benchmarks on CI + disable callgrind benchmarks on non-Linux by @kazimuth in #752
- Fix some bugs in
concat_byte_chunks
,serialize_bsatn
by @Centril in #755 - made subscription broadcast multithreaded by @Shubham8287 in #741
- Split the old
locking_tx_datastore
by @Centril in #757 - Fixed broken quickstart links in readme by @jdetter in #761
- Fix more broken links by @jdetter in #762
- Add the
spacetimedb_table
crate by @Centril in #763 - perf(759): Track the number of active subscription queries by @joshua-spacetime in #760
- spacetimedb_table: nix spacetimedb-core dep + build_unique_error fixes by @Centril in #767
- feat(748): Add compile time feature flag for table metrics by @joshua-spacetime in #749
- Fixed issue with perfcnt not working on certain platforms and architectures by @cloutiertyler in #768
- Split old datastore more by @Centril in #770
- pointers_yielded -> num_pointers_yielded by @Centril in #773
- core: Lock
unwritten_commit
across I/O by @kim in #775 - Use rayon for broadcast_commit_event by @coolreader18 in #779
- Integrate mem arch into
locking_tx_datastore
by @Centril in #769 - reducer to wait until subsciption broadcast finishes by @Shubham8287 in #776
- Handle returned status codes instead of Rust values by @bfops in #783
- fix: Put query metrics behind feature flag by @joshua-spacetime in #784
- feat(query): Make current plan type hashable by @joshua-spacetime in #782
- Fix assorted typos by @bfops in #785
trait ReadColumn
, to read a single column from aRowRef
by @gefjon in #789- feat(query): Add a utility for checking table refs by @joshua-spacetime in #790
- BTreeIndex: Specialize on primitive key types for great performance by @gefjon in #793
- Unify benchmark runs (to some extent) by @kazimuth in #742
- Fix private not building with latest master by @bfops in #798
- BTreeIndex: use
ReadColumn
instead ofProductValue
fields by @gefjon in #796 - Remove separate "callgrind please" script by @kazimuth in #800
- 1_000_000 insert + update benchmarks by @kazimuth in #764
- Add rust-version to Cargo.toml by @coolreader18 in #802
- Simplify broadcast_commit_event by @coolreader18 in #791
- Fix build errors in spacetimedb_table benches by @coolreader18 in #787
- perf(query): Moves Code::Block instead of cloning during eval by @joshua-spacetime in #805
Full Changelog: v0.8.1-beta...v0.8.2-beta-hotfix7