Skip to content

NBS-on-Git: add GitBlobstore-backed NBS store and empty-remote bootstrap#10474

Merged
coffeegoddd merged 4 commits intomainfrom
db/nbs
Feb 11, 2026
Merged

NBS-on-Git: add GitBlobstore-backed NBS store and empty-remote bootstrap#10474
coffeegoddd merged 4 commits intomainfrom
db/nbs

Conversation

@coffeegoddd
Copy link
Copy Markdown
Contributor

Add nbs.NewGitStore wiring plus fetch/push semantics and tests to open against an empty Git remote and bootstrap refs/dolt/data on first write.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for opening NBS stores against empty Git remotes and bootstrapping the refs/dolt/data reference on the first write. It provides a new NewNoConjoinGitStore function and implements fetch/push semantics that gracefully handle missing remote references.

Changes:

  • Added NewNoConjoinGitStore function to create GitBlobstore-backed NBS stores without conjoin support
  • Implemented empty-remote bootstrap logic: fetch operations treat missing remote refs as empty stores, and push operations create refs that don't exist
  • Enhanced error handling to distinguish between missing remote refs and other fetch failures

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
go/store/nbs/store.go Added NewNoConjoinGitStore factory function following established patterns
go/store/nbs/git_blobstore_empty_remote_test.go Comprehensive tests for empty remote handling at the NBS layer
go/store/blobstore/internal/git/impl.go Enhanced FetchRef to detect missing remote refs and removed PushRefWithLease validation to allow bootstrap pushes
go/store/blobstore/internal/git/impl_test.go Tests for missing remote ref detection and bootstrap push semantics
go/store/blobstore/internal/git/api.go Updated PushRefWithLease documentation to clarify bootstrap semantics
go/store/blobstore/git_blobstore.go Implemented empty remote handling in syncForRead and remoteManagedWrite
go/store/blobstore/git_blobstore_test.go Added test verifying Put operation bootstraps empty remotes

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Base automatically changed from db/map to main February 11, 2026 00:40
@coffeegoddd
Copy link
Copy Markdown
Contributor Author

@coffeegoddd DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 0.54 0.55 1.85
groupby_scan 10.09 9.91 -1.78
index_join 1.82 1.82 0.0
index_join_scan 1.34 1.34 0.0
index_scan 21.89 22.28 1.78
oltp_point_select 0.27 0.27 0.0
oltp_read_only 5.28 5.28 0.0
select_random_points 0.54 0.54 0.0
select_random_ranges 0.55 0.55 0.0
table_scan 21.89 22.28 1.78
types_table_scan 66.84 66.84 0.0
write_tests from_latency to_latency percent_change
oltp_delete_insert 6.32 6.32 0.0
oltp_insert 3.07 3.07 0.0
oltp_read_write 11.45 11.45 0.0
oltp_update_index 3.19 3.19 0.0
oltp_update_non_index 3.13 3.13 0.0
oltp_write_only 6.09 6.09 0.0
types_delete_insert 6.91 6.79 -1.74

@coffeegoddd
Copy link
Copy Markdown
Contributor Author

@coffeegoddd DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 0.54 0.54 0.0
groupby_scan 9.91 10.09 1.82
index_join 1.82 1.82 0.0
index_join_scan 1.34 1.34 0.0
index_scan 21.89 21.89 0.0
oltp_point_select 0.27 0.27 0.0
oltp_read_only 5.28 5.28 0.0
select_random_points 0.54 0.54 0.0
select_random_ranges 0.55 0.55 0.0
table_scan 21.89 22.28 1.78
types_table_scan 65.65 66.84 1.81
write_tests from_latency to_latency percent_change
oltp_delete_insert 6.32 6.32 0.0
oltp_insert 3.02 3.07 1.66
oltp_read_write 11.45 11.45 0.0
oltp_update_index 3.25 3.19 -1.85
oltp_update_non_index 3.13 3.13 0.0
oltp_write_only 6.09 6.09 0.0
types_delete_insert 6.79 6.79 0.0

@coffeegoddd
Copy link
Copy Markdown
Contributor Author

@coffeegoddd DOLT

test_name from_latency_p95 to_latency_p95 percent_change
tpcc-scale-factor-1 61.08 61.08 0.0
test_name from_server_name from_server_version from_tps to_server_name to_server_version to_tps percent_change
tpcc-scale-factor-1 dolt 94fa063 37.97 dolt 5bee09c 37.55 -1.11

@coffeegoddd
Copy link
Copy Markdown
Contributor Author

@coffeegoddd DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 0.55 0.55 0.0
groupby_scan 10.09 9.91 -1.78
index_join 1.82 1.86 2.2
index_join_scan 1.34 1.34 0.0
index_scan 21.89 21.89 0.0
oltp_point_select 0.28 0.27 -3.57
oltp_read_only 5.37 5.37 0.0
select_random_points 0.54 0.54 0.0
select_random_ranges 0.56 0.55 -1.79
table_scan 22.28 22.28 0.0
types_table_scan 65.65 66.84 1.81
write_tests from_latency to_latency percent_change
oltp_delete_insert 6.32 6.43 1.74
oltp_insert 3.07 3.07 0.0
oltp_read_write 11.65 11.45 -1.72
oltp_update_index 3.25 3.25 0.0
oltp_update_non_index 3.13 3.13 0.0
oltp_write_only 6.09 6.09 0.0
types_delete_insert 6.91 6.91 0.0

@coffeegoddd
Copy link
Copy Markdown
Contributor Author

@coffeegoddd DOLT

test_name from_latency_p95 to_latency_p95 percent_change
tpcc-scale-factor-1 62.19 62.19 0.0
test_name from_server_name from_server_version from_tps to_server_name to_server_version to_tps percent_change
tpcc-scale-factor-1 dolt 94fa063 37.0 dolt 53fa46c 36.97 -0.08

@coffeegoddd
Copy link
Copy Markdown
Contributor Author

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000
version result total
964d718 ok 5937471
version total_tests
964d718 5937471
correctness_percentage
100.0

@coffeegoddd
Copy link
Copy Markdown
Contributor Author

@coffeegoddd DOLT

test_name from_latency_p95 to_latency_p95 percent_change
tpcc-scale-factor-1 61.08 61.08 0.0
test_name from_server_name from_server_version from_tps to_server_name to_server_version to_tps percent_change
tpcc-scale-factor-1 dolt 94fa063 37.63 dolt 964d718 37.97 0.9

@coffeegoddd coffeegoddd merged commit 86ca603 into main Feb 11, 2026
26 of 28 checks passed
@coffeegoddd coffeegoddd deleted the db/nbs branch February 11, 2026 02:47
@github-actions
Copy link
Copy Markdown

@coffeegoddd DOLT

name add_cnt delete_cnt update_cnt latency
adds_only 60000 0 0 0.6
adds_updates_deletes 60000 60000 60000 3.04
deletes_only 0 60000 0 1.43
updates_only 0 0 60000 1.93

@github-actions
Copy link
Copy Markdown

@coffeegoddd DOLT

test_name detail row_cnt sorted mysql_time sql_mult cli_mult
batching LOAD DATA 10000 1 0.05 2
batching batch sql 10000 1 0.07 1.86
batching by line sql 10000 1 0.07 2
blob 1 blob 200000 1 0.91 4.18 4.34
blob 2 blobs 200000 1 0.86 4.8 4.65
blob no blob 200000 1 0.87 2.86 2.76
col type datetime 200000 1 0.83 2.61 2.59
col type varchar 200000 1 0.68 3.74 3.59
config width 2 cols 200000 1 0.79 2.92 2.54
config width 32 cols 200000 1 1.93 3.08 2.76
config width 8 cols 200000 1 0.95 2.87 3.47
pk type float 200000 1 0.87 2.46 2.39
pk type int 200000 1 0.81 2.59 2.48
pk type varchar 200000 1 1.59 1.65 1.52
row count 1.6mm 1600000 1 5.88 2.9 2.81
row count 400k 400000 1 1.44 2.92 2.81
row count 800k 800000 1 2.81 3.02 2.88
secondary index four index 200000 1 3.81 1.26 1.08
secondary index no secondary 200000 1 0.88 2.81 2.74
secondary index one index 200000 1 1.13 2.58 2.32
secondary index two index 200000 1 2.06 1.74 1.67
sorting shuffled 1mm 1000000 0 5.83 2.53 2.34
sorting sorted 1mm 1000000 1 5.87 2.51 2.42

@github-actions
Copy link
Copy Markdown

@coffeegoddd DOLT

name detail mean_mult
dolt_blame_basic system table 1.25
dolt_blame_commit_filter system table 1.16
dolt_commit_ancestors_commit_filter system table 0.64
dolt_commits_commit_filter system table 1.1
dolt_diff_log_join_from_commit system table 3.07
dolt_diff_log_join_to_commit system table 3.11
dolt_diff_table_from_commit_filter system table 1.23
dolt_diff_table_to_commit_filter system table 1.24
dolt_diffs_commit_filter system table 1.06
dolt_history_commit_filter system table 1.52
dolt_log_commit_filter system table 1.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants