Nil values

[scossu]

I'm writing a CSV file from a 2D table with several nil values. Currently these are encoded as "nil".

E.g.:

> csv = require "ftcsv"
> t = {                
    {f1 = "a", f2 = "b", f3 = "c"},
    {f1 = "d", f2 = "e",},
    {f3 = "f"},
    -- etc.
} 
> csv.encode(t)
"f1","f2","f3"
"a","b","c"
"d","e","nil"
"nil","nil","f"

Is there a way to encode nil as the empty string? I see a transform function for the reader but not for the writer.

[FourierTransformer]

Unfortunately there isn't a way to do this in ftcsv. You'd have to pre-process your table. I do like the idea and would consider adding an encoding option encodeNilAs where people could specify, ex: encodeNilAs = 0 or encodeNilAs = "".

[FourierTransformer]

I have some working code in https://github.com/FourierTransformer/ftcsv/tree/encode-nil if you want to try it out and let me know if it works for your use case

[scossu]

By encodeNilAs do you mean "encode empty cell as..."?

In https://github.com/FourierTransformer/ftcsv/pull/47/files#diff-fe22ce3cdaac256b02d00a3a47da336c5328d8c7ffc07380e3061ccf7d61ea8eR741

if options and options.encodeNilAs ~= nil then

I'm not sure if this will allow me to encode empty cells as nil. How would I do that?

I was actually thinking that if you could easily wire the existing rename option in the writer, that would allow even more flexibility (as long as by "rename" I can also set something to nil).

[FourierTransformer]

Oh, perhaps I misunderstood what you meant. In the example it looks like you have a sparse table. Some of the values are missing, so t[2].f3 == nil. When this gets run through the encoder, it shows up as the string "nil" (since ftcsv does a tostring on the values). I assumed you wanted the nil values to get specified as the empty string "".

so from the example above:

csv = require "ftcsv"
t = {                
    {f1 = "a", f2 = "b", f3 = "c"},
    {f1 = "d", f2 = "e",},
    {f3 = "f"},
} 
csv.encode(t, {encodeNilAs=""}) -- adding the `encodeNilAs=""` option to make nil show up as the empty string in the csv

generates the following csv:

"f1","f2","f3"
"a","b","c"
"d","e",""
"","","f"

The rename option in the parser is for column headers, which I suppose could also be added to the encoder. Can you write out an example of what you're trying to achieve? That might help clarify a bit to see if I can help

[scossu]

Ugh. Disregard my earlier comment. Too many projects at hand and I'm getting wires crossed: indeed I did have the issue of empty cells coming through as "" when decoding, which I found a workaround for, but I had forgotten that this issue was about encoding. Ugh.

Your approach is right. I will review it soon and get back to you. Thanks a lot.

[scossu]

PR approved. Thanks.