Unfinished quotes (also in skipped lines) result in an empty CSV with read_csv() #12440
Closed
2 tasks done
Labels
A-io-csv
Area: reading/writing CSV files
accepted
Ready for implementation
bug
Something isn't working
needs triage
Awaiting prioritization by a maintainer
python
Related to Python Polars
Checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of Polars.
Reproducible example
Log output
Issue description
Whenever there are unfinished quotes in a file, polars.read_csv() raises an exception and reports that the CSV is empty. This error is not very informative, since the file is not, actually, empty.
This can be particularly misleading when the unfinished quotes are in a commented/skipped line (see example above), since one would expect those to be completely ignored.
I managed to find a workaround by setting
quote_char
to another character that was not used anywhere else (%), but that's probably not the optimal way to deal with this. Settingquote_char
to None did not work.Expected behavior
The expected behaviour would be that:
a) Unfinished quotes are completely ignored in commented or skipped rows
b) The error message triggered by unfinished quotes elsewhere is informative
Installed versions
The text was updated successfully, but these errors were encountered: