1.42.6

Merged PRs

dolt

• 8166: [kvexec] Fix panic in non-covering strict lookup
  I missed a test case in nonCovLaxSecondaryLookupGen. I tried to add more tests surrounding the specific panic query here: dolthub/go-mysql-server#2607.
  Also rename the rowexec to kvexec.
• 8139: Feature: Log binlog events to disk
  Major changes:
  • Moved binlogging initialization code from sqlengine to server, so that bin logging only happens when in sql-server mode.
  • binlogProducer now sends events to the new logManager type that writes binlog events to files on disk. binlogStreamer now reads events from those logs and streams them to replicas (instead of receiving events directly from binlogProducer).
  • DoltBinlogPrimaryController now validates that the missing GTIDs from a replica are available in the binlog files and sends an error if the primary doesn't have enough binlog data on disk to get a replica in sync.

go-mysql-server

• 2606: [memo] assume self-join stats cardinality continuity
  Self-join stats estimation is particularly expensive because all of the buckets exactly overlap. If the index is unique, the cardinality distribution will not change. If the index is non-unique, the cardinality will expand proportional to rowCount/distinctCount.

  before
  BenchmarkOltpJoinScan-12    	    1766	    694524 ns/op	  462834 B/op	    8240 allocs/op
  after
  BenchmarkOltpJoinScan-12    	    2460	    481166 ns/op	  193569 B/op	    7129 allocs/op
  

  sysbench perf here: #8159

vitess

• 359: fix detection of multi-statements in ComPrepare
  Currently, preparing multi-statements is not supported; so we can't prepare a query like select ?; select ?;.
  However, the check for this condition just looked for any characters after the first ;, which meant that queries like select ?; \n would incorrectly throw an error.
  This was made apparent using the Prisma ORM, which runs the query:

  SELECT TABLE_NAME AS view_name, VIEW_DEFINITION AS view_sql
  FROM INFORMATION_SCHEMA.VIEWS
  WHERE TABLE_SCHEMA = ?;

  The above query ends in a newline character.
  The fix is to use SplitStatementToPieces(), which trims these white space characters, and check if there's exactly one piece; this was taken from the vitessio repo: https://github.com/vitessio/vitess/blob/main/go/mysql/conn.go#L1204
  fixes #8157
• 358: Feature: parser support for PURGE BINARY LOGS syntax
  https://dev.mysql.com/doc/refman/8.4/en/purge-binary-logs.html
• 357: Bug fix: Send an error response when the server fails to handle COM_BINLOG_DUMP_GTID
  A MySQL primary needs to be able to send back an error response when handling the COM_BINLOG_DUMP_GTID command. Previously, when the integrator returned an error, it was logged in the primary server logs, but it was not being sent back to the replica who sent the command. This change causes an error packet to be sent to the replica, containing the details of the error the integrator returned.
  This change is difficult to test in isolation, but I have tests in dolt that will exercise this codepath.

Closed Issues

• 8157: Error when using Prisma ORM: "Error: unknown error: can not prepare multiple statements"
• 6816: subquery insert into not null column throws error

1.42.5

Merged PRs

dolt

• 8162: Better error message when attempting to push from a shallow clone
  Push from a shallow clone is possible, but we should message better when it's not possible. In order to do this we need to error better from the Generational Chunk Store with a custom error.
  Fixes: #8156
• 8158: fix no-op merge msg in cli
  Fixes: #8148
• 8154: When removing from a secondary index during merging, use the pre-merge ordinal mapping, not the merged mapping, to identify the secondary key to be removed.
  Basically, we have a bug where dropping a column on the remote side of a merge can interfere with updating secondary indexes.
  Example:
  Base Schema: (pk INT PRIMARY KEY, a TINYINT, b INT, UNIQUE KEY b_idx (b))
  "theirs" drops column a
  Merged Schema: (pk INT PRIMARY KEY, b INT, UNIQUE KEY b_idx (b))
  In effect, the merger would see that in the final table, b is the second column. Then, when updating b_idx for each resolved row, it would use the second column of "ours" to find the index entry to remove. But this is incorrect, because the second column of "ours" is a.
  If the user is lucky, these two columns will be different sizes and the merger will panic. But if the two columns are the same size, merge proceeds with an incorrect value. This will cause it to either fail to remove the old row from the secondary index, or remove a different row. Either way, the secondary index is now incorrect.
• 8150: fix warnings in dolt sql shell
  This PR fixes a bug where warnings were incorrectly being suppressed in the dolt sql shell.
  The bug is caused by the shell making queries to check if the working set is dirty, what database we're on, and what branch we're on. The fix is to set a special flag in the session to not clear the warnings for those specific queries.
  companion pr: dolthub/go-mysql-server#2605
  fixes: #8016
• 8124: [stats] limit stats boostrap to server start
  fixes: #8123

go-mysql-server

• 2606: [memo] assume self-join stats cardinality continuity
  Self-join stats estimation is particularly expensive because all of the buckets exactly overlap. If the index is unique, the cardinality distribution will not change. If the index is non-unique, the cardinality will expand proportional to rowCount/distinctCount.

  before
  BenchmarkOltpJoinScan-12    	    1766	    694524 ns/op	  462834 B/op	    8240 allocs/op
  after
  BenchmarkOltpJoinScan-12    	    2460	    481166 ns/op	  193569 B/op	    7129 allocs/op
  

  sysbench perf here: #8159
• 2605: add lock to prevent warnings from being cleared
  This PR adds two functions to BaseSession that toggle a boolean, so integrators can prevent warnings from being cleared.
  This is mostly useful for dolt sql shell.
  addresses #8016

Closed Issues

• 8123: stats refresh warning a little too zealous
• 8156: Unknown push Error When Pushing Large DB to Remote
• 8016: dolt sql suppresses warnings.
• 8148: Ghost Commit Error On Empty Merge
• 7638: Syntax Error Occurs When Using AS Clause with ON DUPLICATE KEY UPDATE

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	2.11	2.97	1.4
groupby_scan	13.46	17.32	1.3
index_join	1.37	2.81	2.1
index_join_scan	1.3	2.22	1.7
index_scan	34.33	53.85	1.6
oltp_point_select	0.18	0.46	2.6
oltp_read_only	3.49	7.7	2.2
select_random_points	0.34	0.77	2.3
select_random_ranges	0.39	0.89	2.3
table_scan	34.33	54.83	1.6
types_table_scan	74.46	142.39	1.9
reads_mean_multiplier			1.9

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.13	6.09	0.7
oltp_insert	3.82	3.02	0.8
oltp_read_write	8.58	13.95	1.6
oltp_update_index	3.82	3.07	0.8
oltp_update_non_index	3.89	3.02	0.8
oltp_write_only	5.37	6.43	1.2
types_delete_insert	7.7	6.67	0.9
writes_mean_multiplier			1.0

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	99.43	34.46	2.9
tpcc_tps_multiplier			2.9

Overall Mean Multiple	1.93

1.42.4

Merged PRs

dolt

• 8155: Bug fix: loading persisted @@server_id from config.json
  This fixes binlog replicas loading @@server_id values persisted in config.json so that they are properly converted from strings to the right type.
  Also added extra connection error logging
• 8153: make tags case insensitive
  This PR makes it so tags are case-insensitive.
  Additionally, fixes a display error in the sql shell.
  fixes #8147
• 8152: Revive ability to read really old dolt_schema tables
  We have several old databases which we don't touch but are public on DoltHub. These databases present an error to users with the removal of the migration ability in 1.42.0. The change will bring them back.
• 8135: Build Archives more robustly
  This change is long in coming. Several findings from building a large user database are incorporated here. Specifically:
  • No longer storing all chunks in memory
  • Cancellable process with ^C
  • Progress reporting for major stages of the build
  • No chunk grouping by default, --group-chunks to enable. Pathological cgo dictionary building problem needs to be fixed before we enable by default.
  • Writing a useful metadata json block with dolt version and origin table file ID.

go-mysql-server

• 2605: add lock to prevent warnings from being cleared
  This PR adds two functions to BaseSession that toggle a boolean, so integrators can prevent warnings from being cleared.
  This is mostly useful for dolt sql shell.
  addresses #8016
• 2603: [memo] reorder should add new plans to intermediate expr join child
  There was a bug where we'd add reordered join plans to project or distinct nodes, rather than their join children. Code comment explains more clearly how this works.
• 2602: disallow forward slash in database name
  fixes #8126
• 2600: Support information_schema.columns hook for doltgres
• 2593: custom row exec
  Additions for custom row operators on Dolt side: #8072

Closed Issues

• 8147: Tag releases: database not found

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	2.07	2.97	1.4
groupby_scan	13.46	17.01	1.3
index_join	1.34	2.81	2.1
index_join_scan	1.27	2.57	2.0
index_scan	34.33	54.83	1.6
oltp_point_select	0.18	0.45	2.5
oltp_read_only	3.49	7.56	2.2
select_random_points	0.34	0.75	2.2
select_random_ranges	0.39	0.89	2.3
table_scan	34.95	55.82	1.6
types_table_scan	75.82	142.39	1.9
reads_mean_multiplier			1.9

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.13	6.09	0.7
oltp_insert	3.82	3.02	0.8
oltp_read_write	8.58	13.95	1.6
oltp_update_index	3.89	3.07	0.8
oltp_update_non_index	3.89	3.02	0.8
oltp_write_only	5.47	6.32	1.2
types_delete_insert	7.7	6.67	0.9
writes_mean_multiplier			1.0

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	99.27	34.31	2.9
tpcc_tps_multiplier			2.9

Overall Mean Multiple	1.93

1.42.3

Merged PRs

dolt

• 8144: Don't panic when performing a GC on a shallow clone
• 8143: Add support for visualizing prolly tree and blob messages in dolt show
  This PR does a couple things:
  1. dolt show #address can now display the internals of a ProllyTreeMap message (typically used for storing indexes). Previously, only splunk/noms show could do this.
  2. Both dolt show and splunk can now display the contents of a Blob message.
  3. If a ProllyTreeMap leaf node contains a value that is itself an address (example: the value of text and json columns), that value is shown as a human readable address, which can be fed back into dolt show or splunk to explore the whole tree.
• 8141: support STAGED as commit hash
  This PR adds support for STAGED as a commit_hash when filtering dolt diff system tables.
  fixes for #7978
• 8140: Continue to support writes when archives are in play
  Error in hasMany effectively prevented writes after moving database to archives.
• 8084: enable key range iter
  There was an issue merging #8025 ontop of a revert. Enable the key iteration optimization.
• 8072: [rowexec] dolt-side lookup execution operator
  This PR adds custom Dolt execution operators for lookup joins. When building an execution plan, we try to replace joinIter with a Dolt equivalent that inlines the key building and map get. This is a lot faster than repeatedly building the secondary iterator and materializing sql.Rows in-between lookups.
  The main downside is that this PR hoists filters in join children to after materializing lookup join rows.
  This brings index_join from 5.18 ms/query to 2.64 ms/q, which will be about 2.0x MySQL's latency.
  This PR falls short of some aspiration goals:
  • We hoist table filters until after the final join row is built because we don't have a way to call scalar expressions on val.Tuple yet. There are edge case queries that might be dramatically slower because of this. To fix this, we could need to convert sql.Expression filters into a format that we could execute on val.Tuple KV pairs.
  • We do not yet try to to optimize consecutive lookup joins. I'm not sure if a materialization block would be better represented iteratively or recursively beyond a simple string of lookups. A lot of interfaces and indexing considerations to think about there.
    Safety comments:
  • we fallback to GMS when lookup source/dest keys are not prolly.Encoding compatible
  • the source iterators are the same as what we used before, but without projection mapping to sql.Rows. The keyless iterator required a change to return duplicate rows at the KV layer (vs the sql layer).
  • the secondary iterators are a generalization of what we currently use, but return KV pairs instead of rows
  • projection mapping is the same but generalized to merge an arbitrary list of KV pairs after the join
    There are extra tests here: dolthub/go-mysql-server#2593

go-mysql-server

• 2598: Adding the binlog_expire_logs_seconds system variable
  MySQL @@binlog_expire_logs_seconds reference docs
• 2596: Fix information_schema.columns for databases with schemas
  Missed this table in my original PR
• 2593: custom row exec
  Additions for custom row operators on Dolt side: #8072

Closed Issues

• 8126: CREATE DATABASE statements should not create databases hierarchicaly
• 7698: Incorrect return typefor UNIX_TIMESTAMP
• 7978: Should be able to use "STAGED" as commits in dolt_commit_diff_[tablename]
• 8131: Docker Mysql Server Resets When Connecting Immediately After Launch

1.42.2

Merged PRs

dolt

• 8132: Detect reading from non-indexed JSON documents and store it in a LazyJSONDocument instead of an IndexedJSONDocument.
  This adds tests that we properly read JSON documents written before the migration to the new indexed JSON storage format. When loading such documents from storage, we should represent them as a LazyJSONDocument instead of an IndexedJSONDocument. This ensures we don't accidentally use the optimized versions of the JSON functions when there aren't any keys in the prolly tree to guide the optimization.

go-mysql-server

• 2596: Fix information_schema.columns for databases with schemas
  Missed this table in my original PR
• 2595: Adding user name and host length validation to CREATE USER
  This change matches MySQL's behavior of limiting user names to 32 chars and host names to 255 chars. Attempting to create a user with a name or host longer than that limit now returns a validation error.
  Customer issue: #8120

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	2.07	3.02	1.5
groupby_scan	13.22	17.32	1.3
index_join	1.37	5.37	3.9
index_join_scan	1.27	2.57	2.0
index_scan	34.33	54.83	1.6
oltp_point_select	0.18	0.46	2.6
oltp_read_only	3.49	7.7	2.2
select_random_points	0.34	0.75	2.2
select_random_ranges	0.39	0.9	2.3
table_scan	34.33	56.84	1.7
types_table_scan	74.46	144.97	1.9
reads_mean_multiplier			2.1

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.13	6.09	0.7
oltp_insert	3.82	3.02	0.8
oltp_read_write	8.58	13.95	1.6
oltp_update_index	3.89	3.07	0.8
oltp_update_non_index	3.89	3.02	0.8
oltp_write_only	5.37	6.32	1.2
types_delete_insert	7.7	6.67	0.9
writes_mean_multiplier			1.0

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	99.02	35.06	2.8
tpcc_tps_multiplier			2.8

Overall Mean Multiple	1.97

1.42.1

Merged PRs

dolt

• 8128: Bug fix: dolt_reset() should commit the active transaction
  Fixes: #6272
• 8127: cache table list
  RootValue caches a hash of its table list, which is invalidated when we update the table address list. The hash is used to cache a map of table names in the root value and expedites table retrieval during planbuilder. The cache could be used elsewhere for performance in the future.
  dolthub/doltgresql#503

Closed Issues

• 6272: CALL DOLT_RESET('--hard') does not implicitly commit the transaction
• 8120: CREATE USER allows username over 32 chars, then SELECT throws error
• 8117: REGEXP isn't collation aware

1.42.0

Backwards incompatible changes in this release:

• The dolt_schemas and dolt_procedures tables now return empty results and disallow edits. In previous releases, select * from either table would result in an error if there was no data. These tables are internal tables to Dolt which are intended to be managed by Dolt itself, and not modified by users. If your application is currently writing to these tables, it will no longer be able to do so and changes will be required in order to upgrade your database.
• Username and host names are now restricted in length (32 and 255 respectively) to match the behavior of MySQL. Attempting to create a user with a name or host longer than that limit now returns a validation error. Existing users and hostnames which are longer than those limits are not affected.

Per Dolt’s versioning policy, this is a minor version bump because these changes may impact existing applications. Please reach out to us on GitHub or Discord if you have questions or need help with any of these changes.

Merged PRs

dolt

• 8116: Return empty dolt_procedures and dolt_schemas tables
  Previously dolt_procedures and dolt_schemas tables could not be selected from if there were not procedures or views. That resulted in errors in the when a dolt CLI client attempted to run dolt diff. This was previously swallowed by the client, but it made the server logs dirty for no reason.
  Now, select * from dolt_schemas will result in an empty result set.
  I did not add new tests for this as several existing tests were broken by this behavior change. I fixed those instead. Due to change in existinging behavior which people may depend on, this requires a minor version bump.

go-mysql-server

• 2595: Adding user name and host length validation to CREATE USER
  This change matches MySQL's behavior of limiting user names to 32 chars and host names to 255 chars. Attempting to create a user with a name or host longer than that limit now returns a validation error.
  Customer issue: #8120
• 2594: Fixed REGEXP
  This fixes the case-sensitivity issue found in: #8117
  Although we had moved REGEXP_LIKE to the ICU engine, we forgot to also move REGEXP, which is a synonym for REGEXP_LIKE according to the docs. This makes that change, and also completely removes all remnants of the old regex code.

Closed Issues

• 8110: ARM64 Docker Image does not contain arm binaries

1.41.4

Merged PRs

dolt

• 8107: Optimize JSON_SET and JSON_REPLACE on IndexedJsonDocument
  This PR includes a new implementation of the JSON_SET and JSON_REPLACE functions that leverage the new indexed JSON storage format.
  For JSON documents that span multiple chunks, only the affected chunks need to be loaded and modified, allowing operations to scale with the size of the removed value instead of the size of the entire document.
• 8103: Optimize JSON_REMOVE on IndexedJsonDocument
  This PR includes a new implementation of the JSON_REMOVE function that leverages the new indexed JSON storage format.
  For JSON documents that span multiple chunks, only the affected chunks need to be loaded and modified, allowing operations to scale with the size of the removed value instead of the size of the entire document.

go-mysql-server

• 2591: Bug fix: Fix @@binlog_row_metadata, add @@binlog_row_image
  Fixes an error in the definition for the @@binlog_row_metadata system variable that prevented it from being queried. Adds the @@binlog_row_image system variable that was missing.
• 2590: Fix databases iter for information_schema tables
  Doltgres information_schema tables include the schemas for the current database, not all databases
• 2588: Only register version function if it doesn't already exist
• 2587: Fix information_schema catalog and schema names
  We had already made this change for doltgres here, this just applies it to more information_schema tables
• 2580: Remove a duplicate column from information_schema
  Just what it says on the tin. This duplicate column causes problems for DuckDB when attempting to connect to doltdb databases.
• 2578: Catch panics in listener
  We had several goroutines in which panics would crash the server process. These were being triggered by panics due to errors in doltgres. Added a recover block to each goroutine.

vitess

• 357: Bug fix: Send an error response when the server fails to handle COM_BINLOG_DUMP_GTID
  A MySQL primary needs to be able to send back an error response when handling the COM_BINLOG_DUMP_GTID command. Previously, when the integrator returned an error, it was logged in the primary server logs, but it was not being sent back to the replica who sent the command. This change causes an error packet to be sent to the replica, containing the details of the error the integrator returned.
  This change is difficult to test in isolation, but I have tests in dolt that will exercise this codepath.
• 356: Bug fix: Off-by-one error when parsing multiple statements
  An off-by-one error in multistatement parsing prevents us from parsing multistatements without a space between the delimiter and the next statement. For example: "SELECT 1;SELECT 2;" would previously be parsed as "SELECT 1;S" and "ELECT 2;".
  Found while testing changes for dolthub/driver#28

Closed Issues

• 8109: ArmDocker

1.41.3

Merged PRs

dolt

• 8102: Bug fix: binlog heartbeat nextLogPosition field
  Heartbeat binlog events must have a correct NextLogPosition field that matches up with the previous events that have been sent in the stream. If not, the replica will shutdown the binlog stream. #8087 fixed this issue, but didn't account for when a heartbeat event is sent after the initial Format Description event, but before any user initiated requests. The way to trigger this is to start replica; on the replica, then don't run any commands on the primary and let the first heartbeat go out after the binlog stream has been up for 30s.
• 8101: go.mod: Migrate from gopkg.in/square/go-jose.v2 to gopkg.in/go-jose/go-jose.v2. Bump version. Picks up fix for CVE-2024-28180.
• 8088: [no-release-info] Add additional tests for manipulating large JSON documents and fix corner case bugs in JSON_LOOKUP and JSON_INSERT
  This PR adds additional tests for calling JSON_INSERT on large JSON documents. It also fixes three issues with IndexedJsonDocuments:
  1. Some operations are not supported by the new optimized implementation for JSON_LOOKUP, such as wildcards on array paths (eg $[*]). Instead of returning an error, we detect the error and fall back on the original implementation.
  2. Attempting to insert a value into a document could cause an infinite loop.
  3. We would fail to read some keys from an IndexedJsonDocument's StaticMap if the document contained arrays.

go-mysql-server

• 2583: [stats] Disable histogram bucket merging for now because it mutated shared memory
  Merging buckets in the current format is unsafe:
  • we collect statistics for an index where two buckets have overlapping values
  • we execute a join using the index with overlapping values, and use a merge algorithm to combine those buckets. The merged bucket is synthetic, but the statistics used for the join is also synthetic, so this all works as expected.
  • a future indexscan selects the compressed range from before, accessing one of the synthetic buckets created by the join
  • we error invalid bucket type: *stats.Bucket at the end of the indexscan when adding the filtered histogram with a synthetic back to the implementor-type statistic
    Edited mergeOverlappingBuckets to not share memory, but also I'm not sure if merging buckets is a common performance win in most cases, so disabling for now
• 2580: Remove a duplicate column from information_schema
  Just what it says on the tin. This duplicate column causes problems for DuckDB when attempting to connect to doltdb databases.

Closed Issues

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	2.07	2.97	1.4
groupby_scan	13.7	17.32	1.3
index_join	1.34	5.28	3.9
index_join_scan	1.27	2.57	2.0
index_scan	34.33	54.83	1.6
oltp_point_select	0.18	0.46	2.6
oltp_read_only	3.49	7.7	2.2
select_random_points	0.33	0.77	2.3
select_random_ranges	0.39	0.9	2.3
table_scan	34.95	56.84	1.6
types_table_scan	75.82	144.97	1.9
reads_mean_multiplier			2.1

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.13	6.09	0.7
oltp_insert	3.82	3.02	0.8
oltp_read_write	8.58	13.95	1.6
oltp_update_index	3.89	3.07	0.8
oltp_update_non_index	3.89	3.02	0.8
oltp_write_only	5.37	6.43	1.2
types_delete_insert	7.7	6.67	0.9
writes_mean_multiplier			1.0

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	98.43	32.02	3.1
tpcc_tps_multiplier			3.1

Overall Mean Multiple	2.07

1.41.2

Merged PRs

dolt

• 8093: go/libraries/doltcore/remotestorage: internal/reliable: Recv: Fix a race where a completed state machine run and a canceled parent context could return a nil response message with a nil error.
• 8090: dolt sql shell slash redux
• 8087: Bug fix: Use correct log position in Dolt to MySQL replication heartbeats
  Ensure heartbeat events sent from a Dolt primary to a MySQL replica have the latest nextLogPosition populated, otherwise the MySQL replica will close the binlog event stream.
• 8086: define schema for dolt_schemas table
  dolthub/doltgresql#454 depends on this PR.
• 8082: /docker/{docker-entrypoint.sh,serverDockerfile}: change image to pass all args to dolt sql-server command
  This PR fixes #8079. Now when running dolthub/dolt-sql-server if the args dolt sql-server are passed to the image, it will error. This will also prevent accidentally starting two Dolt servers in the container.
• 8081: Fixed keyless secondary indexing for Doltgres
  • Companion PR: dolthub/doltgresql#452
    This PR fixes two issues with creating secondary indexes for Doltgres types. The first deals with handlers, as we were not adding a nil handler for the additional hash type, which would cause a panic as the counts were not equal (all non-Extended types should have a matching nil handler).
    The second issue was due to the reuse of an ExtendedTupleComparator. When creating a new ExtendedTupleComparator, we pass in the previous TupleTypeHandler to handle all non-Extended types. If the previous TupleTypeHandler was ExtendedTupleComparator and the new one was also ExtendedTupleComparator, then we could end up with a misinterpretation of data that could lead to incorrect results, as the handler assumed a different type than the actual type. This has been changed so that ExtendedTupleComparator will always use the inner comparator of a previous ExtendedTupleComparator. For now this will always be the default comparator, but if we ever add another one, then this should properly handle that change.
• 8080: Re-enable doltgres sysbench scripts
• 8078: Archive index rework to make loading faster
  The initial impl of archive indexes over optimized for space. This resulted in being 10x slower to load the index of archives than noms table files. To address this:
  • Dropped the end to end compression of the index
  • Dropped the use of var ints for offset deltas and chunk refs
  • Altered the use of byte span offsets, and instead used a end-offset approach which requires no delta processing on load.
  • Used only slices of primitive types in the index memory. Constant time read path with a little more complexity, but allows us to read directly off disk into memory.
    Testing indicates that on a 41 Gb archive file, this returned load performance to match classic table files, and the size of the index increased by about 350Mb (total ~ 1Gb)
• 8077: /go/libraries/doltcore/remotestorage/chunk_fetcher.go: fix nil pointer
  We observe dolthubapi can crash with the following nil pointer error:

  panic: runtime error: invalid memory address or nil pointer dereference
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x29e14d1]
  goroutine 399548427 [running]:
  github.com/dolthub/dolt/go/libraries/doltcore/remotestorage.fetcherRPCDownloadLocsThread.func3()
  external/com_github_dolthub_dolt_go/libraries/doltcore/remotestorage/chunk_fetcher.go:266 +0xf1
  golang.org/x/sync/errgroup.(*Group).Go.func1()
  external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
  created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 399548420
  external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
  

  This pr aims to prevent this.
• 8076: Bump golang.org/x/image from 0.10.0 to 0.18.0 in /go
  Bumps golang.org/x/image from 0.10.0 to 0.18.0.
  Commits

  • 3bbf4a6 tiff: Validate palette indices when parsing palette-color images
  • 6c5fa46 go.mod: update golang.org/x dependencies
  • 55c4ab6 go.mod: update golang.org/x dependencies
  • 0057a93 tiff: fix function name in comment
  • 9e190ae webp: disallow multiple VP8X chunks
  • 445ab0e go.mod: update golang.org/x dependencies
  • 240a51a font/sfnt: support early version 0 OS/2 tables
  • c20bbc3 draw: simplify some calls to fmt.Fprintf
  • 491771c draw: merge draw_go117.go into draw.go
  • 4aa0222 go.mod: update go directive to 1.18
  • Additional commits viewable in compare view
  
  [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
  Dependabot commands and options

  
  You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/dolt/network/alerts).
• 8073: Added schema to index creation
  In Doltgres, whenever we would create an index, we would use the empty schema as the destination (the default value for the schema name). This meant that the updated table with an index was saved into the empty schema, which is incorrect since Doltgres always has a schema. This adds the schema to index creation, along with several other locations that it should be in.

go-mysql-server

• 2583: [stats] Disable histogram bucket merging for now because it mutated shared memory
  Merging buckets in the current format is unsafe:
  • we collect statistics for an index where two buckets have overlapping values
  • we execute a join using the index with overlapping values, and use a merge algorithm to combine those buckets. The merged bucket is synthetic, but the statistics used for the join is also synthetic, so this all works as expected.
  • a future indexscan selects the compressed range from before, accessing one of the synthetic buckets created by the join
  • we error invalid bucket type: *stats.Bucket at the end of the indexscan when adding the filtered histogram with a synthetic back to the implementor-type statistic
    Edited mergeOverlappingBuckets to not share memory, but also I'm not sure if merging buckets is a common performance win in most cases, so disabling for now
• 2581: [stats] populate types for nil zeroing
• [2577](https://github.com/d...