Skip to content

GitBlobstore: implement Put with CAS retries + configurable identity; add Put tests#10417

Merged
coffeegoddd merged 5 commits intomainfrom
db/gitblobstore-next
Feb 5, 2026
Merged

GitBlobstore: implement Put with CAS retries + configurable identity; add Put tests#10417
coffeegoddd merged 5 commits intomainfrom
db/gitblobstore-next

Conversation

@coffeegoddd
Copy link
Copy Markdown
Contributor

This PR adds the first GitBlobstore write path: GitBlobstore.Put, implemented on top of the existing internal/git.GitAPI plumbing. It also adds unit tests for Put, including a contention scenario to verify we don’t clobber concurrent writers.

@coffeegoddd coffeegoddd requested review from Copilot and reltuk February 4, 2026 21:48
Base automatically changed from db/gitblobstore to main February 4, 2026 21:53
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements the Put method for GitBlobstore, enabling write operations with compare-and-swap (CAS) retry logic to handle concurrent writers. The implementation uses exponential backoff and preserves other keys when conflicts occur, making it safe for multi-writer scenarios.

Changes:

  • Implemented GitBlobstore.Put with CAS retry logic and exponential backoff for handling concurrent updates
  • Added configurable identity support via NewGitBlobstoreWithIdentity for controlling git commit author/committer
  • Added comprehensive unit tests for Put including round-trip verification, overwrite behavior, and contention scenarios

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
go/store/blobstore/git_blobstore.go Implements Put method with CAS retries, adds identity configuration, and includes helper functions for building commits and managing temporary indexes
go/store/blobstore/git_blobstore_test.go Adds test infrastructure (requireGitOnPath, testIdentity) and comprehensive tests for Put including basic round-trip, overwrite, and concurrent writer scenarios

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

coffeegoddd and others added 2 commits February 4, 2026 14:02
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@coffeegoddd
Copy link
Copy Markdown
Contributor Author

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000
version result total
18dea9b ok 5937471
version total_tests
18dea9b 5937471
correctness_percentage
100.0

Copy link
Copy Markdown
Contributor

@reltuk reltuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. A couple comments.

}
}

if err := gbs.api.UpdateIndexCacheInfo(ctx, indexFile, "100644", blobOID, key); err != nil {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you do some testing and verify whether this is idempotent? If it isn't, can we make it so?

That is to say: If the file already exists, we should succeed here (AFAIK...)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe its idempotent for files (overwrites) but will error if the a directory prefix is already staged, then you try writing that. Added a test that demonstrates this, though I don't expect this to be a problem for us since we are writting files only: https://github.com/dolthub/dolt/pull/10417/changes#diff-7e67da3161a3fc914b18c9152a5ea340fbf582ca9cb2efc2dc5ef6ab7b0d5c20R335

@coffeegoddd
Copy link
Copy Markdown
Contributor Author

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000
version result total
f8fd7ee ok 5937471
version total_tests
f8fd7ee 5937471
correctness_percentage
100.0

@coffeegoddd coffeegoddd merged commit db7263e into main Feb 5, 2026
23 checks passed
@coffeegoddd coffeegoddd deleted the db/gitblobstore-next branch February 5, 2026 00:10
@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 5, 2026

@coffeegoddd DOLT

name add_cnt delete_cnt update_cnt latency
adds_only 60000 0 0 0.62
adds_updates_deletes 60000 60000 60000 3.09
deletes_only 0 60000 0 1.48
updates_only 0 0 60000 1.97

@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 5, 2026

@coffeegoddd DOLT

test_name detail row_cnt sorted mysql_time sql_mult cli_mult
batching LOAD DATA 10000 1 0.05 2
batching batch sql 10000 1 0.08 1.63
batching by line sql 10000 1 0.09 1.44
blob 1 blob 200000 1 0.89 4.3 4.51
blob 2 blobs 200000 1 0.89 4.75 4.56
blob no blob 200000 1 0.9 2.79 2.69
col type datetime 200000 1 0.83 2.63 2.57
col type varchar 200000 1 0.67 3.85 3.85
config width 2 cols 200000 1 0.81 3.02 2.41
config width 32 cols 200000 1 1.93 3.14 2.76
config width 8 cols 200000 1 0.95 2.92 2.72
pk type float 200000 1 0.95 2.29 2.19
pk type int 200000 1 0.79 2.67 2.54
pk type varchar 200000 1 1.56 1.7 1.59
row count 1.6mm 1600000 1 5.72 3.01 2.77
row count 400k 400000 1 1.45 2.93 2.79
row count 800k 800000 1 2.86 2.99 2.75
secondary index four index 200000 1 3.66 1.34 1.07
secondary index no secondary 200000 1 0.89 2.81 2.71
secondary index one index 200000 1 1.14 2.59 2.23
secondary index two index 200000 1 2 1.82 1.72
sorting shuffled 1mm 1000000 0 5.41 2.76 2.5
sorting sorted 1mm 1000000 1 5.39 2.77 2.61

@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 5, 2026

@coffeegoddd DOLT

name detail mean_mult
dolt_blame_basic system table 1.23
dolt_blame_commit_filter system table 1.15
dolt_commit_ancestors_commit_filter system table 0.63
dolt_commits_commit_filter system table 1.11
dolt_diff_log_join_from_commit system table 2.95
dolt_diff_log_join_to_commit system table 3
dolt_diff_table_from_commit_filter system table 1.24
dolt_diff_table_to_commit_filter system table 1.32
dolt_diffs_commit_filter system table 1.09
dolt_history_commit_filter system table 1.46
dolt_log_commit_filter system table 1.05

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants