validate fails without telling me why

[alecristia]

I validated my project with this output:

$ child-project validate .

warning: ./metadata/children.csv: unknown column 'Unnamed: 0' in children, exepected columns are: experiment,child_id,child_dob,location_id,child_sex,language,languages,mat_ed,fat_ed,car_ed,monoling,monoling_criterion,normative,normative_criterion,mother_id,father_id,order_of_birth,n_of_siblings,household_size,dob_criterion,dob_accuracy
warning: ./metadata/recordings.csv: unknown columns 'Unnamed: 0,weird,repeats' in recordings, exepected columns are: experiment,child_id,date_iso,start_time,recording_device_type,recording_filename,duration,session_id,session_offset,recording_device_id,experimenter,location_id,its_filename,upl_filename,trs_filename,lena_id,might_feature_gaps,start_time_accuracy,noisy_setting,notes
warning: ./metadata/recordings.csv: 'NA' is not a proper date/time for column 'start_time' (expected %H:%M) on line 131

So I added times for as many lines as I could and then tried to validate again:

$ child-project validate .
Traceback (most recent call last):
File "/scratch1/home/acristia/ChildProjectVenv/bin/child-project", line 8, in
sys.exit(main())
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/ChildProject/cmdline.py", line 451, in main
args.func(args)
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/ChildProject/cmdline.py", line 101, in validate
errors, warnings = project.validate(args.ignore_recordings, args.profile)
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/ChildProject/projects.py", line 358, in validate
self.read(verbose=True)
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/ChildProject/projects.py", line 323, in read
self.recordings = self.rt.read()
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/ChildProject/tables.py", line 84, in read
self.df = pd.read_csv(self.path, **pd_flags)
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/pandas/io/parsers.py", line 688, in read_csv
return _read(filepath_or_buffer, kwds)
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/pandas/io/parsers.py", line 460, in _read
data = parser.read(nrows)
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/pandas/io/parsers.py", line 1198, in read
ret = self._engine.read(nrows)
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/pandas/io/parsers.py", line 2157, in read
data = self._reader.read(nrows)
File "pandas/_libs/parsers.pyx", line 847, in pandas._libs.parsers.TextReader.read
File "pandas/_libs/parsers.pyx", line 862, in pandas._libs.parsers.TextReader._read_low_memory
File "pandas/_libs/parsers.pyx", line 918, in pandas._libs.parsers.TextReader._read_rows
File "pandas/_libs/parsers.pyx", line 905, in pandas._libs.parsers.TextReader._tokenize_rows
File "pandas/_libs/parsers.pyx", line 2042, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: Expected 16 fields in line 18, saw 17

Since the only thing that changed was recordings.csv, that means the error is there (probably spaces within fields). I'll fix that, but I'd like to point out that we should have a more graceful way of dealing with this.

[alecristia]

yes, I confirm: I had spaces in some fields without a quote around the field. This may happen to others too!

[LoannPeurey]

=> added a task on asana Le mer. 18 mai 2022 à 11:55, Alex Cristia ***@***.***> a écrit :

…

yes, I confirm: I had spaces in some fields without a quote around the field. This may happen to others too! — Reply to this email directly, view it on GitHub <#374 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYCGCRAIKN2SFJFN33IEP2TVKS5BXANCNFSM5WH334RQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>