thread 'main' panicked at src/adapter/src/catalog/open.rs:551:17: unexpected update kinds: [StateUpdate ... #28055
Comments
|
This looks like a real issue, but I have a good idea of what's going on. I'll take a look today. |
I take that back, I found the issue and it is a test issue. I'll push a fix shortly. |
jkosh44
added a commit
to jkosh44/materialize
that referenced
this issue
Jul 8, 2024
This commit fixes when/where a read-only catalog attempts to perform writes. Read-only catalogs will return an error if they try to commit a non-empty transaction. Read-only catalogs are prevented from attempting to prune the storage usage events. Pruning happens based on a timer, which may cause read-only catalogs to mistakenly attempt and fail to write to the durable catalog. Additionally, read-only catalogs DO attempt to perform migrations. We do not support opening read-only catalogs at a higher version than the current catalog, so we want the migrations to cause an error if they are non-empty. Resolves MaterializeInc#28055
5 tasks
jkosh44
added a commit
to jkosh44/materialize
that referenced
this issue
Jul 8, 2024
This commit fixes when/where a read-only catalog attempts to perform writes. Read-only catalogs will return an error if they try to commit a non-empty transaction. Read-only catalogs are prevented from attempting to prune the storage usage events. Pruning happens based on a timer, which may cause read-only catalogs to mistakenly attempt and fail to write to the durable catalog. Additionally, read-only catalogs DO attempt to perform migrations. We do not support opening read-only catalogs at a higher version than the current catalog, so we want the migrations to cause an error if they are non-empty. Resolves MaterializeInc#28055
jkosh44
added a commit
to jkosh44/materialize
that referenced
this issue
Jul 8, 2024
Previously, storage usage pruning worked by taking the following steps: 1. Durably write retractions for old storage usage events. 2. Listen for all changes to the durable catalog. 3. Generate builtin table updates from the changes. This approach does not work for savepoint catalogs, because they only pretend to write things durably but don't actually commit the write. Therefore, listening for changes always returns an empty result. This commit fixes the issue by returning the updates from the durable transaction before commit. This matches the behavior of all other catalog writes. While we're modifying storage usage pruning, the read-only case is optimized to return immediately. This commit has the added benefit that read-only and savepoint catalogs will not see unrelated updates in step (2) from a writer catalog, which resolves MaterializeInc#28055
jkosh44
added a commit
to jkosh44/materialize
that referenced
this issue
Jul 8, 2024
Previously, storage usage pruning worked by taking the following steps: 1. Durably write retractions for old storage usage events. 2. Listen for all changes to the durable catalog. 3. Generate builtin table updates from the changes. This approach does not work for savepoint catalogs, because they only pretend to write things durably but don't actually commit the write. Therefore, listening for changes always returns an empty result. This commit fixes the issue by returning the updates from the durable transaction before commit. This matches the behavior of all other catalog writes. While we're modifying storage usage pruning, the read-only case is optimized to return immediately. This commit has the added benefit that read-only and savepoint catalogs will not see unrelated updates in step (2) from a writer catalog, which resolves MaterializeInc#28055
5 tasks
jkosh44
added a commit
to jkosh44/materialize
that referenced
this issue
Jul 8, 2024
Previously, storage usage pruning worked by taking the following steps: 1. Durably write retractions for old storage usage events. 2. Listen for all changes to the durable catalog. 3. Generate builtin table updates from the changes. This approach does not work for savepoint catalogs, because they only pretend to write things durably but don't actually commit the write. Therefore, listening for changes always returns an empty result. This commit fixes the issue by returning the updates from the durable transaction before commit. This matches the behavior of all other catalog writes. While we're modifying storage usage pruning, the read-only case is optimized to return immediately. This commit has the added benefit that read-only and savepoint catalogs will not see unrelated updates in step (2) from a writer catalog, which resolves MaterializeInc#28055
jkosh44
added a commit
to jkosh44/materialize
that referenced
this issue
Jul 8, 2024
Previously, storage usage pruning worked by taking the following steps: 1. Durably write retractions for old storage usage events. 2. Listen for all changes to the durable catalog. 3. Generate builtin table updates from the changes. This approach does not work for savepoint catalogs, because they only pretend to write things durably but don't actually commit the write. Therefore, listening for changes always returns an empty result. This commit fixes the issue by returning the updates from the durable transaction before commit. This matches the behavior of all other catalog writes. While we're modifying storage usage pruning, the read-only case is optimized to return immediately. This commit has the added benefit that read-only and savepoint catalogs will not see unrelated updates in step (2) from a writer catalog, which resolves MaterializeInc#28055
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What version of Materialize are you using?
54720ab
What is the issue?
Seen in https://buildkite.com/materialize/nightly/builds/8373#01907ecf-54e1-4898-89ed-12dcc10ed299
Maybe just a testdrive issue?
The code comes from #27872
ci-regexp: unexpected update kinds
The text was updated successfully, but these errors were encountered: