Skip to content
This repository has been archived by the owner on May 24, 2022. It is now read-only.

scrub null byte into empty cell #17

Open
seamusabshere opened this issue Mar 4, 2020 · 0 comments
Open

scrub null byte into empty cell #17

seamusabshere opened this issue Mar 4, 2020 · 0 comments
Labels
enhancement faraday Requested by Faraday

Comments

@seamusabshere
Copy link
Member

null bytes are treated as valid data

feature request: by default or with an option, convert them into emptiness

i tried to use --null but (at least in ruby) you can't pass a null byte into a command

irb(main):007:0> NULL = ["\x0"]
=> ["\u0000"]

irb(main):008:0> NULL_REGEXP = "\\s*(\\s+|" + NULL.map { |x| Regexp.escape(x) }.join('|') + ")\\s*"                                                                                                       (irb):8: warning: already => "\\s*(\\s+|\u0000)\\s*"

irb(main):009:0> system({},"scrubcsv --clean-column-names --trim-whitespace --replace-newlines --null #{Shellwords.escape NULL_REGEXP}",in: 'null.csv',out: 'null.out.csv',)
Traceback (most recent call last):
        5: from /usr/bin/irb:23:in `<main>'
        4: from /usr/bin/irb:23:in `load'
        3: from /Library/Ruby/Gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
        2: from (irb):9
        1: from (irb):9:in `system'
ArgumentError (string contains null byte)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement faraday Requested by Faraday
Projects
None yet
Development

No branches or pull requests

1 participant