Skip to content

Conversation

@ajantha-bhat
Copy link
Contributor

Fixes #2501

@ajantha-bhat ajantha-bhat requested a review from snazy October 29, 2021 07:42
that must be consistent across all branches in Nessie, it is stored as so-called _global state_.
The value of the snapshot-ID is stored per Nessie named reference (branch or tag).
`IcebergTable` contains the _current_ pointer to Iceberg's table metadata plus the
global ID's called _idGenerators_ that are defined in the table metadata.
Copy link
Contributor

@nastra nastra Oct 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's wait with updating the Spec until the code is merged into Iceberg. It's likely that the naming will change

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I have mentioned same while creating this issue. I will keep it in draft till iceberg side PR is merged. apache/iceberg#3257

@codecov
Copy link

codecov bot commented Oct 29, 2021

Codecov Report

Merging #2544 (d0f14ee) into main (ad496c3) will increase coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##               main    #2544      +/-   ##
============================================
+ Coverage     86.28%   86.31%   +0.02%     
- Complexity     2609     2610       +1     
============================================
  Files           327      327              
  Lines         14602    14602              
  Branches       1105     1105              
============================================
+ Hits          12600    12603       +3     
+ Misses         1594     1588       -6     
- Partials        408      411       +3     
Flag Coverage Δ
java 86.88% <ø> (+0.02%) ⬆️
javascript 75.62% <ø> (ø)
python 83.67% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...nessie/versioned/persist/tx/TxDatabaseAdapter.java 78.38% <0.00%> (+0.42%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ad496c3...d0f14ee. Read the comment docs.

@ajantha-bhat ajantha-bhat marked this pull request as draft October 29, 2021 08:15
Copy link
Member

@snazy snazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM so far. But let's wait until everyhing's merged everywhere.

global ID's called _idGenerators_ that are defined in the table metadata.
Since Iceberg's table metadata manages information that must be consistent across all branches in Nessie,
required information is stored as so-called _global state_.
The current pointer to Iceberg's table metadata is stored per Nessie named reference(branch or tag)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The current pointer to Iceberg's table metadata is stored per Nessie named reference(branch or tag)
The current pointer to Iceberg's table metadata is stored per Nessie named reference (branch or tag)

@ajantha-bhat
Copy link
Contributor Author

@snazy : I looked up (Iceberg: snapshot-ID) text across the code. I think many places we need to change the comments as well. For this also I will wait for things to get merged and handle together.

@ajantha-bhat ajantha-bhat changed the title Update spec as per #2312 Update spec as per latest on ref state. Feb 1, 2022
@ajantha-bhat ajantha-bhat marked this pull request as ready for review February 1, 2022 12:04
@ajantha-bhat ajantha-bhat requested review from nastra and snazy February 1, 2022 12:04
@ajantha-bhat ajantha-bhat changed the title Update spec as per latest on ref state. Docs: Update spec as per latest on ref state. Feb 1, 2022
nastra
nastra previously approved these changes Feb 1, 2022

Nessie needs to track Iceberg's _table metadata_ as so called _Global State_ within Nessie to
ensure that schema evolution works as expected.
ensure that table evolution and other operations like delete works as expected.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: works -> work

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@ajantha-bhat ajantha-bhat merged commit 6385bcf into projectnessie:main Feb 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update Nessie spec asper #2312

4 participants