Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend and fix overwrite command #272

Merged
merged 60 commits into from
Nov 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
a50176e
Add boilerplate for overwrite command
CobbCoding1 Oct 24, 2024
5770416
Add more boilerplate for overwrite and update help.c message
CobbCoding1 Oct 25, 2024
7aca16e
Prepare overwrite implementation
CobbCoding1 Oct 25, 2024
31e3b89
Setup sqlite in overwrite.c and add parsing for non-flag arguments
CobbCoding1 Oct 25, 2024
c2412a6
Add sqlite initialization functions to overwrite, and messages for ce…
CobbCoding1 Oct 26, 2024
fdc5315
Add simple test for overwrite
CobbCoding1 Oct 26, 2024
52fa9a5
clang-format
CobbCoding1 Oct 26, 2024
734cddb
Add `overwrite` command skeleton and initialization (#241)
CobbCoding1 Oct 26, 2024
4b6aa33
Add improved testing for overwrite command
CobbCoding1 Oct 26, 2024
34cbd79
Update usage and tests to reflect the usage
CobbCoding1 Oct 26, 2024
8d350f2
merge changes
CobbCoding1 Oct 26, 2024
2403e6e
Fix test makefile
CobbCoding1 Oct 26, 2024
416215c
Update `overwrite` tests and usage. (#243)
CobbCoding1 Oct 26, 2024
bf05639
Update sqlite3_open_v2 to be readonly unless necessary
CobbCoding1 Oct 27, 2024
93ec5ce
Update arg parsing for some of the new arguments and options
CobbCoding1 Oct 27, 2024
75303fb
Implement clear command (untested so far)
CobbCoding1 Oct 27, 2024
8b00c55
merge
CobbCoding1 Oct 27, 2024
8237961
Add insert command
CobbCoding1 Oct 27, 2024
fb6659f
Update row and column parsing to use new format
CobbCoding1 Oct 27, 2024
5e80735
Update tests and prepare overwrite file
CobbCoding1 Oct 28, 2024
321c06d
Update overwrite to integrate better with existing overwrite infastru…
CobbCoding1 Oct 28, 2024
b629df6
Update test and add length to output when listing overwrites
CobbCoding1 Oct 28, 2024
8db9195
clang-format
CobbCoding1 Oct 28, 2024
6fb667b
remove commented code
CobbCoding1 Oct 28, 2024
8e54a3c
Add error message to failed init
CobbCoding1 Oct 29, 2024
06d669e
add check for if an overwrite already exists at selected row and colu…
CobbCoding1 Oct 29, 2024
0ffddbd
clang format
CobbCoding1 Oct 29, 2024
f7140e3
fix build error
CobbCoding1 Oct 29, 2024
8ce138e
Fix test Makefile issue
CobbCoding1 Oct 29, 2024
8ee9bef
Update zsvext-test to reflect the usage changes
CobbCoding1 Oct 29, 2024
d949294
Add timestamp and author fields, and update the tests accordingly
CobbCoding1 Oct 30, 2024
7395edc
Clang format
CobbCoding1 Oct 30, 2024
1af3f75
Update overwrite help message, change position to overwrites_uix, add…
CobbCoding1 Oct 31, 2024
c9c3766
Run clang-format
CobbCoding1 Oct 31, 2024
ceed2a6
Add header to overwrite list output
CobbCoding1 Nov 1, 2024
3bc4fb5
Update to support Excel-style cells
CobbCoding1 Nov 5, 2024
3d1f9b5
clang format
CobbCoding1 Nov 5, 2024
77890b3
update parse_xl_address to be more universal
CobbCoding1 Nov 5, 2024
2aa20ad
Update Makefile
liquidaty Nov 5, 2024
7e69b08
Implement --A1 flag
CobbCoding1 Nov 5, 2024
accb758
Merge
CobbCoding1 Nov 5, 2024
010cfd2
Implement support for parsing no-timestamp (not implemented yet)
CobbCoding1 Nov 6, 2024
813b86c
Merge branch 'liquidaty:main' into main
CobbCoding1 Nov 7, 2024
0c72b36
Merge branch 'main' of github.com:CobbCoding1/zsv into overwrite-command
CobbCoding1 Nov 7, 2024
59b7718
Implement no-timestamp functionality
CobbCoding1 Nov 7, 2024
e672c5c
Change timestamps to UNIX time (tests not updated yet)
CobbCoding1 Nov 7, 2024
5b234d1
Update test to work with UNIX time
CobbCoding1 Nov 7, 2024
d3885c4
Add a few more tests
CobbCoding1 Nov 7, 2024
92822f1
Update tests to improve the timestamp accuracy. Not completely right …
CobbCoding1 Nov 7, 2024
858c33b
fix issues with timing with dates in the overwrite testing
CobbCoding1 Nov 8, 2024
8268bf7
Update tests to use stat command for timestamps
CobbCoding1 Nov 8, 2024
487fc74
Clang format
CobbCoding1 Nov 8, 2024
1fb3743
work more on tests
CobbCoding1 Nov 8, 2024
edaa70b
Fix timestamp test so that it does not fail if it falls a second behind
CobbCoding1 Nov 8, 2024
9bb161c
commit with another csv test file
CobbCoding1 Nov 8, 2024
c1ab973
add expected output
CobbCoding1 Nov 8, 2024
f610e68
Update overwrite.c
liquidaty Nov 9, 2024
41b8f59
fix test/overwrite/Makefile for bsd/mac
liquidaty Nov 9, 2024
2e01af1
Update Makefile
liquidaty Nov 9, 2024
6161496
Update Makefile
liquidaty Nov 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 84 additions & 27 deletions app/overwrite.c
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@

struct zsv_overwrite_args {
char *filepath;
// options
unsigned char a1 : 1;
unsigned char timestamp : 1;
// commands
unsigned char list : 1;
unsigned char clear : 1;
unsigned char add : 1;
Expand Down Expand Up @@ -65,6 +69,8 @@ const char *zsv_overwrite_usage_msg[] = {
" cell already exists",
" For `remove`, exit without error even if no overwrite for",
" the specified cell already exists",
" --no-timestamp : For `add`, don't save timestamp when adding an overwrite",
" --A1 : For `list`, Display addresses in A1-notation",
"",
"Description:",
" The `overwrite` utility allows you to manage a list of \"overwrites\" associated",
Expand Down Expand Up @@ -101,10 +107,12 @@ static int zsv_overwrites_init(struct zsv_overwrite_ctx *ctx, struct zsv_overwri
sqlite3_stmt *query = NULL;
int ret = 0;

/*
if ((ret = sqlite3_initialize()) != SQLITE_OK) {
fprintf(stderr, "Failed to initialize library: %d, %s\n", ret, sqlite3_errmsg(ctx->sqlite3.db));
return err;
}
*/

if ((ret = sqlite3_open_v2(overwrites_fn, &ctx->sqlite3.db, SQLITE_OPEN_READONLY, NULL)) != SQLITE_OK || args->add ||
args->clear) {
Expand Down Expand Up @@ -192,7 +200,8 @@ static int zsv_overwrite_check_for_value(struct zsv_overwrite_ctx *ctx, struct z
return err;
}

static int zsv_overwrites_insert(struct zsv_overwrite_ctx *ctx, struct zsv_overwrite_data *overwrite) {
static int zsv_overwrites_insert(struct zsv_overwrite_ctx *ctx, struct zsv_overwrite_data *overwrite,
struct zsv_overwrite_args *args) {
int err = 0;
sqlite3_stmt *query = NULL;
int ret;
Expand All @@ -203,10 +212,10 @@ static int zsv_overwrites_insert(struct zsv_overwrite_ctx *ctx, struct zsv_overw
sqlite3_bind_int64(query, 1, overwrite->row_ix);
sqlite3_bind_int64(query, 2, overwrite->col_ix);
sqlite3_bind_text(query, 3, (const char *)overwrite->val.str, -1, SQLITE_STATIC);
time_t my_time = time(NULL);
char *time_buf = ctime(&my_time);
time_buf[strlen(time_buf) - 1] = '\0'; // Remove newline
sqlite3_bind_text(query, 4, (const char *)time_buf, -1, SQLITE_STATIC);
if (args->timestamp)
sqlite3_bind_int64(query, 4, time(NULL));
else
sqlite3_bind_null(query, 4);
sqlite3_bind_text(query, 5, "", -1, SQLITE_STATIC); // author
if ((ret = sqlite3_step(query)) != SQLITE_DONE) {
err = 1;
Expand All @@ -225,16 +234,44 @@ static int zsv_overwrites_insert(struct zsv_overwrite_ctx *ctx, struct zsv_overw
return err;
}

static int zsv_overwrites_exit(struct zsv_overwrite_ctx *ctx, struct zsv_overwrite_data *overwrite,
static int zsv_overwrites_free(struct zsv_overwrite_ctx *ctx, struct zsv_overwrite_data *overwrite,
zsv_csv_writer writer) {
zsv_writer_delete(writer);
free(overwrite->val.str);
sqlite3_close(ctx->sqlite3.db);
sqlite3_shutdown();
if (writer)
zsv_writer_delete(writer);
if (ctx) {
free(ctx->src);
sqlite3_close(ctx->sqlite3.db);
}
if (overwrite)
free(overwrite->val.str);

// sqlite3_shutdown();
return 0;
}

static int show_all_overwrites(struct zsv_overwrite_ctx *ctx, zsv_csv_writer writer) {
static char *row_col_to_a1(size_t col, size_t row) {
char buffer[64];
int index = 63;
buffer[index] = '\0';

while (1) {
if (index == 0)
return NULL;
col--;
buffer[--index] = 'A' + (col % 26);
col /= 26;
if (col == 0)
break;
}
printf("%s\n", &buffer[index]);
// 20 extra bytes for row
char *result = malloc(strlen(&buffer[index]) + 20 + 1);
if (result)
sprintf(result, "%s%zu", &buffer[index], row + 1);
return result;
}

static int show_all_overwrites(struct zsv_overwrite_ctx *ctx, struct zsv_overwrite_args *args, zsv_csv_writer writer) {
int err = 0;
sqlite3_stmt *stmt;
int ret;
Expand All @@ -243,28 +280,42 @@ static int show_all_overwrites(struct zsv_overwrite_ctx *ctx, zsv_csv_writer wri
fprintf(stderr, "Failed to prepare statement: %s\n", sqlite3_errmsg(ctx->sqlite3.db));
return err;
}

// display header
zsv_writer_cell(writer, 0, "row", 3, 0);
zsv_writer_cell(writer, 0, "column", 6, 0);
zsv_writer_cell(writer, 0, "value", 5, 0);
zsv_writer_cell(writer, 0, "timestamp", 9, 0);
zsv_writer_cell(writer, 0, "author", 6, 0);
zsv_writer_cell(writer, 0, (const unsigned char *)"row", 3, 0);
zsv_writer_cell(writer, 0, (const unsigned char *)"column", 6, 0);
zsv_writer_cell(writer, 0, (const unsigned char *)"value", 5, 0);
zsv_writer_cell(writer, 0, (const unsigned char *)"timestamp", 9, 0);
zsv_writer_cell(writer, 0, (const unsigned char *)"author", 6, 0);

while ((ret = sqlite3_step(stmt)) == SQLITE_ROW) {
size_t row = sqlite3_column_int64(stmt, 0);
size_t col = sqlite3_column_int64(stmt, 1);
const unsigned char *val = sqlite3_column_text(stmt, 2);
size_t val_len = sqlite3_column_bytes(stmt, 2);
const unsigned char *timestamp = sqlite3_column_text(stmt, 3);
size_t timestamp_len = sqlite3_column_bytes(stmt, 3);
size_t timestamp = 0;
// If timestamp is null, that means --no-timestamp was passed on insertion
int timestamp_is_null = sqlite3_column_type(stmt, 3) == SQLITE_NULL;
if (!timestamp_is_null)
timestamp = sqlite3_column_int64(stmt, 3);
const unsigned char *author = sqlite3_column_text(stmt, 4);
size_t author_len = sqlite3_column_bytes(stmt, 4);

zsv_writer_cell_zu(writer, 1, row);
zsv_writer_cell_zu(writer, 0, col);
if (args->a1) {
char *col_a1 = row_col_to_a1(col, row);
if (!col_a1) {
err = 1;
fprintf(stderr, "Error converting column number to A1-notation\n");
return err;
}
zsv_writer_cell(writer, 1, (const unsigned char *)col_a1, strlen(col_a1), 0);
free(col_a1);
} else {
zsv_writer_cell_zu(writer, 1, row);
zsv_writer_cell_zu(writer, 0, col);
}
zsv_writer_cell(writer, 0, val, val_len, 0);
zsv_writer_cell(writer, 0, timestamp, timestamp_len, 0);
if (!timestamp_is_null)
zsv_writer_cell_zu(writer, 0, timestamp);
else
zsv_writer_cell(writer, 0, (const unsigned char *)"", 0, 0); // write an empty cell if null
zsv_writer_cell(writer, 0, author, author_len, 0);
}

Expand Down Expand Up @@ -335,6 +386,8 @@ int ZSV_MAIN_FUNC(ZSV_COMMAND)(int argc, const char *argv[], struct zsv_opts *op

struct zsv_overwrite_ctx ctx = {0};
struct zsv_overwrite_args args = {0};
// By default, save timestamps
args.timestamp = 1;
struct zsv_overwrite_data overwrite = {0};
struct zsv_csv_writer_options writer_opts = {0};

Expand All @@ -349,6 +402,10 @@ int ZSV_MAIN_FUNC(ZSV_COMMAND)(int argc, const char *argv[], struct zsv_opts *op
} else if (!strcmp(opt, "--old-value")) {
fprintf(stderr, "Error: %s is not implemented\n", opt);
err = 1;
} else if (!strcmp(opt, "--no-timestamp")) {
args.timestamp = 0;
} else if (!strcmp(opt, "--A1")) {
args.a1 = 1;
} else if (!strcmp(opt, "list")) {
args.list = 1;
} else if (!strcmp(opt, "clear")) {
Expand Down Expand Up @@ -385,13 +442,13 @@ int ZSV_MAIN_FUNC(ZSV_COMMAND)(int argc, const char *argv[], struct zsv_opts *op

zsv_csv_writer writer = zsv_writer_new(&writer_opts);
if (args.list)
show_all_overwrites(&ctx, writer);
show_all_overwrites(&ctx, &args, writer);
else if (args.clear)
zsv_overwrites_clear(&ctx);
else if (!err && args.add && ctx.sqlite3.db)
zsv_overwrites_insert(&ctx, &overwrite);
zsv_overwrites_insert(&ctx, &overwrite, &args);

zsv_overwrites_exit(&ctx, &overwrite, writer);
zsv_overwrites_free(&ctx, &overwrite, writer);

return err;
}
39 changes: 33 additions & 6 deletions app/test/overwrite/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,27 +40,54 @@ ifeq ($(EXE),)
$(error EXE is not defined)
endif

test: test-1 test-2 test-excel-cells test-echo-overwrite-auto
test: test-1 test-2 test-excel-cells test-add test-echo-overwrite-auto test-timestamp

test-1:
@${TEST_INIT}
@${PREFIX} ${EXE} dummy.csv clear
@${CHECK} [ "`${EXE} dummy.csv list`" = "row,column,value,timestamp,author" ] && ${TEST_PASS} || ${TEST_FAIL}

TIMESTAMP=$(shell date +"%a %b %-d %R:%S %Y")
TIMESTAMP=$(shell date +"%s")
HEADER=row,column,value,timestamp,author

test-2:
@${TEST_INIT}
@${PREFIX} ${EXE} dummy.csv clear
@${PREFIX} ${EXE} dummy.csv add 2-1 ABC
@${CHECK} [ "`${EXE} dummy.csv list`" = "`printf 'row,column,value,timestamp,author\n2,1,ABC,$(TIMESTAMP),\n'`" ] && ${TEST_PASS} || ${TEST_FAIL}
@${PREFIX} ${EXE} dummy.csv add 2-1 ABC --no-timestamp
@${CHECK} [ "`${EXE} dummy.csv list`" = "`printf \"$(HEADER)\n2,1,ABC,,\n\"`" ] && ${TEST_PASS} || ${TEST_FAIL}

test-excel-cells:
@${TEST_INIT}
@${PREFIX} ${EXE} dummy.csv clear
@${PREFIX} ${EXE} dummy.csv add C2 EXCEL
@${CHECK} [ "`${EXE} dummy.csv list`" = "`printf 'row,column,value,timestamp,author\n1,2,EXCEL,$(TIMESTAMP),\n'`" ] && ${TEST_PASS} || ${TEST_FAIL}
@${PREFIX} ${EXE} dummy.csv add C2 EXCEL --no-timestamp
@${CHECK} [ "`${EXE} dummy.csv list`" = "`printf \"$(HEADER)\n1,2,EXCEL,,\n\"`" ] && ${TEST_PASS} || ${TEST_FAIL}

test-add:
@${TEST_INIT}
@${PREFIX} ${EXE} dummy2.csv clear
@${PREFIX} ${EXE} dummy2.csv add 1-2 VAL1 --no-timestamp
@${PREFIX} ${EXE} dummy2.csv add 2-2 VAL2 --no-timestamp
@${PREFIX} ${EXE} dummy2.csv add 2-3 VAL3 --no-timestamp
@${PREFIX} ${EXE} dummy2.csv add 2-4 VAL4 --no-timestamp
@${CHECK} [ "`${EXE} dummy2.csv list`" = "`printf \"$(HEADER)\n1,2,VAL1,,\n2,2,VAL2,,\n2,3,VAL3,,\n2,4,VAL4,,\n\"`" ] \
&& ${TEST_PASS} || ${TEST_FAIL}
@${PREFIX} ${ECHO_EXE} --overwrite-auto dummy2.csv > ${TMP_DIR}/[email protected]
@${CMP} ${TMP_DIR}/[email protected] expected/[email protected] && ${TEST_PASS} || ${TEST_FAIL}

STAT_MOD_TS=$(shell if stat -c %Y Makefile >/dev/null 2>/dev/null; then echo 'stat -c %Y' ; else echo 'stat -f %m' ; fi)
test-timestamp:
@${TEST_INIT}
@${PREFIX} ${EXE} dummy2.csv clear
@(${PREFIX} ${EXE} dummy2.csv add 1-2 ABC && ${STAT_MOD_TS} .zsv/data/dummy2.csv/overwrite.sqlite3 > ${TMP_DIR}/timestamp1.txt)
@(${PREFIX} ${EXE} dummy2.csv add 1-3 ABC && ${STAT_MOD_TS} .zsv/data/dummy2.csv/overwrite.sqlite3 > ${TMP_DIR}/timestamp2.txt)
@{ \
EXPECTED_TS1=`cat ${TMP_DIR}/timestamp1.txt`; \
EXPECTED_TS2=`cat ${TMP_DIR}/timestamp2.txt`; \
LOWER_TS1=$$((EXPECTED_TS1 - 1)); \
LOWER_TS2=$$((EXPECTED_TS2 - 1)); \
OUTPUT="`${EXE} dummy2.csv list | tr -d '\n'`"; \
echo "$$OUTPUT" | grep -q "${HEADER}1,2,ABC,\($$LOWER_TS1\|$$EXPECTED_TS1\),1,3,ABC,\($$LOWER_TS2\|$$EXPECTED_TS2\)," && ${TEST_PASS} || ${TEST_FAIL}; \
}

test-echo-overwrite-auto:
@${TEST_INIT}
Expand Down
4 changes: 4 additions & 0 deletions app/test/overwrite/dummy2.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
A,B,C,D,E
1,2,3,4,5
6,7,8,9,10
11,12,13,14,15
4 changes: 4 additions & 0 deletions app/test/overwrite/expected/test-add.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
A,B,C,D,E
1,2,VAL1,4,5
6,7,VAL2,VAL3,VAL4
11,12,13,14,15