Skip to content

Commit 6595297

Browse files
committed
docs(csvclean): Add more examples of checking/counting/viewing errors, closes #1262
1 parent d0d6cb5 commit 6595297

File tree

1 file changed

+50
-5
lines changed

1 file changed

+50
-5
lines changed

docs/scripts/csvclean.rst

+50-5
Original file line numberDiff line numberDiff line change
@@ -147,13 +147,16 @@ See also: :doc:`../common_arguments`.
147147
Examples
148148
========
149149

150-
Test a file with data rows that are shorter and longer than the header row:
150+
Process a file with data rows that are shorter and longer than the header row, and omit those rows:
151151

152152
.. code-block:: console
153153
154-
$ csvclean examples/bad.csv 2> errors.csv
154+
$ csvclean --length-mismatch --omit-error-rows examples/bad.csv 2> errors.csv
155155
column_a,column_b,column_c
156156
0,mixed types.... uh oh,17
157+
158+
.. code-block:: console
159+
157160
$ cat errors.csv
158161
line_number,msg,column_a,column_b,column_c
159162
1,"Expected 3 columns, found 4 columns",1,27,,I'm too long!
@@ -163,7 +166,7 @@ Test a file with data rows that are shorter and longer than the header row:
163166

164167
If any data rows are longer than the header row, you need to add columns manually: for example, by adding one or more delimiters (``,``) to the end of the header row. :code:`csvclean` can't do this, because it is designed to work with standard input, and correcting an error at the start of the CSV data based on an observation later in the CSV data would require holding all the CSV data in memory – which is not an option for large files.
165168

166-
Test a file with empty columns:
169+
Process a file with empty columns:
167170

168171
.. code-block:: console
169172
@@ -172,20 +175,62 @@ Test a file with empty columns:
172175
a,,,,
173176
,,c,,
174177
,,,,
178+
179+
.. code-block:: console
180+
:emphasize-lines: 3
181+
175182
$ cat errors.csv
176183
line_number,msg,a,b,c,,
177184
1,"Empty columns named 'b', '', ''! Try: csvcut -C 2,4,5",,,,,
178185
179-
Use :doc:`csvcut` to exclude the empty columns:
186+
Then, use :doc:`csvcut` to exclude the empty columns:
180187

181-
.. code-block:: bash
188+
.. code-block:: console
182189
183190
$ csvcut -C 2,4,5 examples/test_empty_columns.csv
184191
a,c
185192
a,
186193
,c
187194
,
188195
196+
Check whether any errors found:
197+
198+
.. code-block:: console
199+
200+
$ if [ csvclean -a examples/bad.csv ]; then echo "my message"; fi
201+
my message
202+
203+
Or:
204+
205+
.. code-block:: console
206+
207+
$ [ csvclean -a examples/bad.csv ] && echo "my message"
208+
my message
209+
210+
Or:
211+
212+
.. code-block:: console
213+
214+
$ csvclean -a examples/bad.csv >/dev/null 2>&1
215+
$ echo $?
216+
1
217+
218+
Count the number of errors found:
219+
220+
.. code-block:: console
221+
222+
$ csvclean -a examples/bad.csv 2>&1 >/dev/null | csvstat --count
223+
2
224+
225+
View only the errors found:
226+
227+
.. code-block:: console
228+
229+
$ csvclean -a examples/bad.csv 2>&1 >/dev/null
230+
line_number,msg,column_a,column_b,column_c
231+
1,"Expected 3 columns, found 4 columns",1,27,,I'm too long!
232+
2,"Expected 3 columns, found 2 columns",,I'm too short!
233+
189234
To change the line ending from line feed (LF or ``\n``) to carriage return and line feed (CRLF or ``\r\n``) use:
190235

191236
.. code-block:: bash

0 commit comments

Comments
 (0)