go/cmd/dolt: Make movable temp file configuration lazy.

[reltuk]

Previously, early in the process life-cycle after dolt was run, dolt would immediately check if a file created in os.TempDir() was able to be os.Renamed into a subdirectory of the dolt process's data directory, by default $PWD/.dolt. If the rename failed, it would configure Dolt to use .dolt/temptf as the movable temp file directory instead.

This meant that for many dolt invocations, Dolt would do some local filesystem writes, including potentially Mkdir(.dolt) before it it was fully loaded. With the advent of things like dolt accessing the running server through sql-server.info and dolt --host ... sql this behavior was not ideal. It creates races with concurrently run dolt processes that try to use the .dolt directory, and it creates odd behavior when running something like dolt sql in a non-Dolt directory but on a filesystem mount where this rename from os.TempDir() does not work.

This PR changes the check and the configuration of a potential .dolt/temptf directory to be delayed until the first temporary movable file is actually requested by Dolt. At that point we know that we have a need for a renamable temp file and it is OK to go ahead and create .dolt/temptf if we need it.

This PR changes the error handling around some edge cases like permissions errors on the calling process creating temptf within .dolt. In particular, some errors which were previously early and fatal are now delayed until use site and may end up being persistent but non-fatal to the process, depending on the operation.

[angelamayxie]

add a comment re: what this is for

[zachmu]

LGTM

[coffeegoddd]

@reltuk DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
5362a4c	ok	5937471

version	total_tests
5362a4c	5937471

correctness_percentage
100.0

[github-actions]

@coffeegoddd DOLT

name	add_cnt	delete_cnt	update_cnt	latency
adds_only	60000	0	0	0.64
adds_updates_deletes	60000	60000	60000	3.11
deletes_only	0	60000	0	1.47
updates_only	0	0	60000	1.98

[github-actions]

@coffeegoddd DOLT

test_name	detail	row_cnt	sorted	mysql_time	sql_mult	cli_mult
batching	LOAD DATA	10000	1	0.06	1.83
batching	batch sql	10000	1	0.07	1.86
batching	by line sql	10000	1	0.07	1.86
blob	1 blob	200000	1	0.87	4.94	5.05
blob	2 blobs	200000	1	0.83	5.55	5.55
blob	no blob	200000	1	0.85	3.13	2.89
col type	datetime	200000	1	0.8	2.78	2.55
col type	varchar	200000	1	0.65	4.72	4.43
config width	2 cols	200000	1	0.76	2.88	2.62
config width	32 cols	200000	1	1.74	3.75	3.36
config width	8 cols	200000	1	0.94	3.13	3.63
pk type	float	200000	1	0.85	2.74	2.33
pk type	int	200000	1	0.8	3.26	2.45
pk type	varchar	200000	1	1.43	1.87	1.96
row count	1.6mm	1600000	1	5.57	3.2	2.82
row count	400k	400000	1	1.41	3.11	2.74
row count	800k	800000	1	2.8	3.14	2.75
secondary index	four index	200000	1	3.39	1.48	1.14
secondary index	no secondary	200000	1	0.9	3.07	2.53
secondary index	one index	200000	1	1.13	2.84	2.27
secondary index	two index	200000	1	1.93	2.01	1.64
sorting	shuffled 1mm	1000000	0	5.51	3.24	2.97
sorting	sorted 1mm	1000000	1	5.58	3.25	3.02

[github-actions]

@coffeegoddd DOLT

name	detail	mean_mult
dolt_blame_basic	system table	1.2
dolt_blame_commit_filter	system table	2.69
dolt_commit_ancestors_commit_filter	system table	0.53
dolt_commits_commit_filter	system table	1
dolt_diff_log_join_from_commit	system table	2.58
dolt_diff_log_join_to_commit	system table	2.58
dolt_diff_table_from_commit_filter	system table	1.16
dolt_diff_table_to_commit_filter	system table	1.22
dolt_diffs_commit_filter	system table	1.07
dolt_history_commit_filter	system table	1.61
dolt_log_commit_filter	system table	1.05

[github-actions]

@coffeegoddd DOLT

name	add_cnt	delete_cnt	update_cnt	latency
adds_only	60000	0	0	0.66
adds_updates_deletes	60000	60000	60000	3.13
deletes_only	0	60000	0	1.54
updates_only	0	0	60000	2.05

[github-actions]

@coffeegoddd DOLT

test_name	detail	row_cnt	sorted	mysql_time	sql_mult	cli_mult
batching	LOAD DATA	10000	1	0.05	2
batching	batch sql	10000	1	0.07	1.86
batching	by line sql	10000	1	0.07	2.14
blob	1 blob	200000	1	0.89	4.29	4.49
blob	2 blobs	200000	1	0.91	4.64	4.43
blob	no blob	200000	1	0.89	2.82	2.73
col type	datetime	200000	1	0.89	2.44	2.49
col type	varchar	200000	1	0.67	3.84	3.76
config width	2 cols	200000	1	0.79	2.67	2.62
config width	32 cols	200000	1	1.96	3.06	2.58
config width	8 cols	200000	1	0.96	2.88	2.81
pk type	float	200000	1	0.99	2.22	2.15
pk type	int	200000	1	0.81	2.81	2.54
pk type	varchar	200000	1	1.53	1.77	1.75
row count	1.6mm	1600000	1	5.61	3.09	2.98
row count	400k	400000	1	1.44	2.94	2.88
row count	800k	800000	1	2.82	3.02	2.95
secondary index	four index	200000	1	3.65	1.36	1.18
secondary index	no secondary	200000	1	0.91	2.77	2.68
secondary index	one index	200000	1	1.14	2.65	2.39
secondary index	two index	200000	1	2.06	1.75	1.72
sorting	shuffled 1mm	1000000	0	5.57	2.69	2.57
sorting	sorted 1mm	1000000	1	5.56	2.7	2.7

[github-actions]

@coffeegoddd DOLT

name	detail	mean_mult
dolt_blame_basic	system table	1.22
dolt_blame_commit_filter	system table	2.59
dolt_commit_ancestors_commit_filter	system table	0.59
dolt_commits_commit_filter	system table	1.05
dolt_diff_log_join_from_commit	system table	2.86
dolt_diff_log_join_to_commit	system table	2.9
dolt_diff_table_from_commit_filter	system table	1.23
dolt_diff_table_to_commit_filter	system table	1.2
dolt_diffs_commit_filter	system table	1.06
dolt_history_commit_filter	system table	1.48
dolt_log_commit_filter	system table	1.16

[github-actions]

@coffeegoddd DOLT

name	add_cnt	delete_cnt	update_cnt	latency
adds_only	60000	0	0	0.72
adds_updates_deletes	60000	60000	60000	3.1
deletes_only	0	60000	0	1.57
updates_only	0	0	60000	2.01

[github-actions]

@coffeegoddd DOLT

test_name	detail	row_cnt	sorted	mysql_time	sql_mult	cli_mult
batching	LOAD DATA	10000	1	0.06	1.83
batching	batch sql	10000	1	0.08	1.63
batching	by line sql	10000	1	0.09	1.44
blob	1 blob	200000	1	0.87	4.32	4.55
blob	2 blobs	200000	1	0.85	4.85	4.67
blob	no blob	200000	1	0.88	2.83	2.72
col type	datetime	200000	1	0.83	2.64	2.58
col type	varchar	200000	1	0.69	3.83	3.62
config width	2 cols	200000	1	0.78	2.72	2.56
config width	32 cols	200000	1	1.93	3.16	2.64
config width	8 cols	200000	1	0.95	2.94	3.13
pk type	float	200000	1	0.86	2.52	2.85
pk type	int	200000	1	0.76	2.8	2.72
pk type	varchar	200000	1	1.68	1.61	1.54
row count	1.6mm	1600000	1	5.67	3.07	2.87
row count	400k	400000	1	1.42	2.99	2.82
row count	800k	800000	1	2.85	3.06	2.8
secondary index	four index	200000	1	3.5	1.4	1.16
secondary index	no secondary	200000	1	0.9	2.81	2.71
secondary index	one index	200000	1	1.14	2.63	2.3
secondary index	two index	200000	1	2.03	1.81	1.73
sorting	shuffled 1mm	1000000	0	5.19	2.97	2.77
sorting	sorted 1mm	1000000	1	5.43	2.84	2.6

[github-actions]

@coffeegoddd DOLT

name	detail	mean_mult
dolt_blame_basic	system table	1.21
dolt_blame_commit_filter	system table	2.59
dolt_commit_ancestors_commit_filter	system table	0.59
dolt_commits_commit_filter	system table	1
dolt_diff_log_join_from_commit	system table	2.87
dolt_diff_log_join_to_commit	system table	2.92
dolt_diff_table_from_commit_filter	system table	1.24
dolt_diff_table_to_commit_filter	system table	1.19
dolt_diffs_commit_filter	system table	1
dolt_history_commit_filter	system table	1.54
dolt_log_commit_filter	system table	1.1

[github-actions]

@coffeegoddd DOLT

name	add_cnt	delete_cnt	update_cnt	latency
adds_only	60000	0	0	0.62
adds_updates_deletes	60000	60000	60000	3
deletes_only	0	60000	0	1.49
updates_only	0	0	60000	1.97

[github-actions]

@coffeegoddd DOLT

test_name	detail	row_cnt	sorted	mysql_time	sql_mult	cli_mult
batching	LOAD DATA	10000	1	0.06	1.67
batching	batch sql	10000	1	0.09	1.44
batching	by line sql	10000	1	0.09	1.33
blob	1 blob	200000	1	0.92	4.2	4.34
blob	2 blobs	200000	1	0.91	4.64	4.57
blob	no blob	200000	1	0.94	2.67	2.63
col type	datetime	200000	1	0.81	2.74	2.79
col type	varchar	200000	1	0.66	3.92	3.85
config width	2 cols	200000	1	0.8	2.64	2.61
config width	32 cols	200000	1	1.92	3.34	2.68
config width	8 cols	200000	1	1.03	2.7	3.17
pk type	float	200000	1	0.85	2.56	2.55
pk type	int	200000	1	0.82	2.59	2.54
pk type	varchar	200000	1	1.58	1.68	1.59
row count	1.6mm	1600000	1	6	2.89	2.91
row count	400k	400000	1	1.45	2.96	3.01
row count	800k	800000	1	2.94	2.97	2.9
secondary index	four index	200000	1	3.58	1.39	1.17
secondary index	no secondary	200000	1	0.92	2.74	2.65
secondary index	one index	200000	1	1.21	2.46	2.26
secondary index	two index	200000	1	2.08	1.75	1.7
sorting	shuffled 1mm	1000000	0	5.48	2.73	2.55
sorting	sorted 1mm	1000000	1	5.37	2.86	2.65

[github-actions]

@coffeegoddd DOLT

name	detail	mean_mult
dolt_blame_basic	system table	1.2
dolt_blame_commit_filter	system table	2.56
dolt_commit_ancestors_commit_filter	system table	0.62
dolt_commits_commit_filter	system table	1.1
dolt_diff_log_join_from_commit	system table	2.88
dolt_diff_log_join_to_commit	system table	2.91
dolt_diff_table_from_commit_filter	system table	1.32
dolt_diff_table_to_commit_filter	system table	1.21
dolt_diffs_commit_filter	system table	1.09
dolt_history_commit_filter	system table	1.63
dolt_log_commit_filter	system table	1.1