Add JSONL support to `dolt table {import,export}` by coffeegoddd · Pull Request #10577 · dolthub/dolt

coffeegoddd · 2026-02-25T23:00:31Z

Adds JSONL (.jsonl / --file-type jsonl) support for both dolt table export (newline-delimited rows reusing existing JSON serialization) and dolt table import (streaming JSONL reader), with updated CLI help plus unit and Bats integration tests.

Copilot

Pull request overview

Adds JSONL export support to dolt table export by introducing a new jsonl data format that reuses existing JSON row serialization, along with unit + integration test coverage and updated CLI help text.

Changes:

Add .jsonl / --file-type jsonl support to mvdata export writer selection.
Add Go unit tests verifying newline-delimited JSON output using the existing JSON writer with custom header/separator/footer.
Add Bats integration tests covering JSONL export (including JSON / binary / quoting edge cases) and update CLI help text.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
integration-tests/bats/export-tables.bats	Adds JSONL export assertions and edge-case coverage via Python JSON parsing.
go/libraries/doltcore/table/typed/json/writer_jsonl_test.go	New unit tests validating JSONL output behavior via `NewJSONWriterWithHeader`.
go/libraries/doltcore/mvdata/file_data_loc.go	Adds `jsonl` parsing + writer creation path for export.
go/libraries/doltcore/mvdata/data_loc_test.go	Extends data location tests to include `.jsonl` / `jsonl`.
go/libraries/doltcore/mvdata/data_loc.go	Introduces `JsonlFile` data format and wiring for extension inference.
go/cmd/dolt/commands/tblcmds/export.go	Updates `dolt table export` help text to mention JSONL and supported formats.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-25T23:06:08Z

go/libraries/doltcore/mvdata/file_data_loc.go

+	case "jsonl", ".jsonl":
+		return JsonlFile


DFFromString now treats jsonl as a valid DataFormat, but FileDataLocation.NewReader does not handle JsonlFile. This means dolt table import --file-type jsonl ... will pass the file-type validation (since DFFromString no longer returns InvalidDataFormat) and then fail later with a generic "unsupported format" error. Either add JSONL import support, or keep jsonl out of import validation (e.g., validate allowed file types per-command) / return a clear error when JsonlFile is used for reading.

Suggested change

case "jsonl", ".jsonl":

return JsonlFile

Copilot · 2026-02-25T23:06:08Z

go/cmd/dolt/commands/tblcmds/export.go

+
+{{.EmphasisLeft}}.json{{.EmphasisRight}} exports a single JSON object containing a {{.EmphasisLeft}}rows{{.EmphasisRight}} array; {{.EmphasisLeft}}.jsonl{{.EmphasisRight}} exports one JSON object per line.
+
 See the help for {{.EmphasisLeft}}dolt table import{{.EmphasisRight}} as the options are the same.


The help text says "See the help for dolt table import as the options are the same", but export now supports jsonl (and also lists sql / parquet) while table import help and validation do not list / support these formats. This is likely to confuse users; consider either updating the wording to avoid claiming parity with import, or explicitly calling out which --file-type values are valid for export vs import (and that stdout export is limited to csv/psv).

Suggested change

See the help for {{.EmphasisLeft}}dolt table import{{.EmphasisRight}} as the options are the same.

When writing to stdout (omitting {{.LessThan}}|file{{.GreaterThan}} or using {{.EmphasisLeft}}-{{.EmphasisRight}}), only {{.EmphasisLeft}}csv{{.EmphasisRight}} and {{.EmphasisLeft}}psv{{.EmphasisRight}} are supported.

For information on importing data, see the help for {{.EmphasisLeft}}dolt table import{{.EmphasisRight}}; note that its supported file types and options differ from export.

coffeegoddd · 2026-02-25T23:29:02Z

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
`4196a34`	ok	5937471

version	total_tests
`4196a34`	5937471

correctness_percentage
100.0

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-25T23:52:17Z

go/libraries/doltcore/table/typed/json/jsonl_reader.go

+		var err error
+		r.sampleRow, err = r.ReadSqlRow(context.Background())
+		return err == nil, nil


VerifySchema drops any error returned by ReadSqlRow (it returns nil error even when ReadSqlRow fails). This can mask malformed JSON / conversion errors during schema verification and cause later failures with less actionable context. Return the actual err (and only keep sampleRow when err == nil) so callers see the correct failure reason.

Suggested change

var err error

r.sampleRow, err = r.ReadSqlRow(context.Background())

return err == nil, nil

sampleRow, err := r.ReadSqlRow(context.Background())

if err != nil {

return false, err

}

r.sampleRow = sampleRow

Copilot · 2026-02-25T23:52:18Z

go/libraries/doltcore/table/typed/json/jsonl_reader.go

+}
+
+func (r *JSONLReader) ReadRow(ctx context.Context) (row.Row, error) {
+	panic("deprecated")


This method panics at runtime. Since JSONLReader is declared to implement table.SqlTableReader, any code path that still calls ReadRow (even indirectly) will crash the process. Prefer returning a non-panicking error (or implement the method properly) to avoid a hard failure.

Suggested change

panic("deprecated")

return nil, fmt.Errorf("ReadRow is deprecated for JSONLReader; use ReadSqlRow instead")

Copilot · 2026-02-25T23:52:18Z

go/libraries/doltcore/mvdata/file_data_loc.go


+func resolveJSONSchema(dEnv *env.DoltEnv, root doltdb.RootValue, opts interface{}) (schema.Schema, error) {
+	var sch schema.Schema
+	jsonOpts, _ := opts.(JSONOptions)


resolveJSONSchema silently ignores a failed type assertion (jsonOpts, _ := opts.(JSONOptions)). If a non-JSONOptions value is ever passed, this will fall through with zero-values (e.g., empty TableName) and produce misleading downstream errors. Check the ok result and return a clear error when opts is non-nil but not JSONOptions.

Suggested change

jsonOpts, _ := opts.(JSONOptions)

jsonOpts, ok := opts.(JSONOptions)

if opts != nil && !ok {

return nil, fmt.Errorf("invalid JSON options type: %T", opts)

}

Copilot · 2026-02-25T23:52:18Z

go/libraries/doltcore/table/typed/json/jsonl_reader.go

+	if r.closer != nil {
+		err := r.closer.Close()
+		r.closer = nil
+		return err
+	}
+	return errors.New("already closed")


Returning an error on a second Close() makes this reader non-idempotent, which can be surprising in Go (where Close is commonly safe to call multiple times, especially with layered defers). Consider returning nil when already closed to make cleanup more robust.

Suggested change

if r.closer != nil {

err := r.closer.Close()

r.closer = nil

return err

}

return errors.New("already closed")

if r.closer == nil {

return nil

}

err := r.closer.Close()

r.closer = nil

return err

coffeegoddd · 2026-02-26T00:41:05Z

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
`ca3a329`	ok	5937471

version	total_tests
`ca3a329`	5937471

correctness_percentage
100.0

coffeegoddd · 2026-02-26T00:59:58Z

fixes #10573

jycor

LGTM

go/libraries/doltcore/mvdata/file_data_loc.go

Co-authored-by: James Cor <jcor@ucsd.edu>

coffeegoddd · 2026-02-26T01:30:44Z

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
`06da432`	ok	5937471

version	total_tests
`06da432`	5937471

correctness_percentage
100.0

coffeegoddd · 2026-02-26T01:37:45Z

@coffeegoddd DOLT

test_name	from_latency_p95	to_latency_p95	percent_change
tpcc-scale-factor-1	61.08	61.08	0.0

test_name	from_server_name	from_server_version	from_tps	to_server_name	to_server_version	to_tps	percent_change
tpcc-scale-factor-1	dolt	`c31b3b9`	38.23	dolt	`ae32b5d`	37.66	-1.49

/{go,integration-tests}: add jsonl support to dolt table export

4196a34

coffeegoddd requested a review from Copilot February 25, 2026 23:00

Copilot started reviewing on behalf of coffeegoddd February 25, 2026 23:01 View session

Copilot AI reviewed Feb 25, 2026

View reviewed changes

dolthub deleted a comment from github-actions bot Feb 25, 2026

coffeegoddd added the performance_approved label Feb 25, 2026

coffeegoddd added the correctness_approved label Feb 25, 2026

coffeegoddd changed the title ~~Add JSONL support to dolt table export~~ Add JSONL support to dolt table {import,export} Feb 25, 2026

/{go,integration-tests}: support jsonl import

fe5d1cf

coffeegoddd requested a review from Copilot February 25, 2026 23:46

Copilot AI reviewed Feb 25, 2026

View reviewed changes

Copilot started reviewing on behalf of coffeegoddd February 25, 2026 23:54 View session

/go/libraries/doltcore/{mvdata,table}: pr feedback

ca3a329

coffeegoddd requested a review from jycor February 26, 2026 00:45

jycor approved these changes Feb 26, 2026

View reviewed changes

go/libraries/doltcore/mvdata/file_data_loc.go Outdated Show resolved Hide resolved

go/libraries/doltcore/mvdata/file_data_loc.go Outdated Show resolved Hide resolved

coffeegoddd and others added 2 commits February 25, 2026 17:02

Update go/libraries/doltcore/mvdata/file_data_loc.go

06da432

Co-authored-by: James Cor <jcor@ucsd.edu>

Update go/libraries/doltcore/mvdata/file_data_loc.go

ae32b5d

Co-authored-by: James Cor <jcor@ucsd.edu>

coffeegoddd merged commit 41d9b25 into main Feb 26, 2026
25 checks passed

coffeegoddd deleted the db/jsonl branch February 26, 2026 17:47

BrewTestBot mentioned this pull request Feb 27, 2026

dolt 1.83.0 Homebrew/homebrew-core#269832

Merged


		{{.EmphasisLeft}}.json{{.EmphasisRight}} exports a single JSON object containing a {{.EmphasisLeft}}rows{{.EmphasisRight}} array; {{.EmphasisLeft}}.jsonl{{.EmphasisRight}} exports one JSON object per line.

		See the help for {{.EmphasisLeft}}dolt table import{{.EmphasisRight}} as the options are the same.

-See the help for {{.EmphasisLeft}}dolt table import{{.EmphasisRight}} as the options are the same.
+When writing to stdout (omitting {{.LessThan}}|file{{.GreaterThan}} or using {{.EmphasisLeft}}-{{.EmphasisRight}}), only {{.EmphasisLeft}}csv{{.EmphasisRight}} and {{.EmphasisLeft}}psv{{.EmphasisRight}} are supported.
+For information on importing data, see the help for {{.EmphasisLeft}}dolt table import{{.EmphasisRight}}; note that its supported file types and options differ from export.

-		var err error
-		r.sampleRow, err = r.ReadSqlRow(context.Background())
-		return err == nil, nil
+		sampleRow, err := r.ReadSqlRow(context.Background())
+		if err != nil {
+			return false, err
+		}
+		r.sampleRow = sampleRow

	panic("deprecated")
	return nil, fmt.Errorf("ReadRow is deprecated for JSONLReader; use ReadSqlRow instead")

-	jsonOpts, _ := opts.(JSONOptions)
+	jsonOpts, ok := opts.(JSONOptions)
+	if opts != nil && !ok {
+		return nil, fmt.Errorf("invalid JSON options type: %T", opts)
+	}

Uh oh!

Conversation

coffeegoddd commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

coffeegoddd commented Feb 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

coffeegoddd commented Feb 26, 2026

Uh oh!

coffeegoddd commented Feb 26, 2026

Uh oh!

jycor left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

coffeegoddd commented Feb 26, 2026

Uh oh!

coffeegoddd commented Feb 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coffeegoddd commented Feb 25, 2026 •

edited

Loading