info on metadataimporter?

[alecristia]

I did a search in the docs for MetadataImporter, metadata, importer, and didn't come across an explanation of the behavior of:

• preserve_dates
• preserve_metadata

Thanks in advance!

[lucasgautheron]

Sorry about that; the EL1000 clearly lacks documentation. However, I thought it should not be confused with ChildProject's docs because it is too specific and that might cause more confusion than anything.

There's a little more einformation here:
https://gin.g-node.org/EL1000/tools/src/master/EL1000/metadata.py#L48

In other words:

• preserve_dates will preserve the dates originally present in the metadata/its instead of setting the date of birth to 1000-01-01 (and recordings dates accordingly)
• preserve_metadata currently only impacts languages spoken in the household. if enabled, this information will be preserved in the default metadata; otherwise it will be moved to metadata/confidential.

[alecristia]

thank you, I did confuse this with childproject! I now have all I need

[alecristia]

I just pushed 2 commits to Lyon:

• in the latest commit I did not recalculate ages, and that works
• in the previous one I was trying to recalculate age, but I was getting an error:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/EL1000/metadata.py", line 142, in recover_recording
recording['age'] = age(date, child_dob)
NameError: name 'age' is not defined

I thought this was due to the fact that my original metadata didn't have age, but it did have age (although it was not rounded?). I don't know enough python to be sure - perhaps it was complaining about the age function missing? Probably not, right?

[lucasgautheron]

It's a bug, which Marianne spotted and fixed :) Re-install the EL1000 package and try again? Le mer. 20 oct. 2021 à 19:15, Alex Cristia ***@***.***> a écrit :

…

I just pushed 2 commits to Lyon: - in the latest commit <https://gin.g-node.org/EL1000/lyon/commit/c53597de43f1aa3c217f9a710fda5e1da37aa6c3> I did not recalculate ages, and that works - in the previous one <https://gin.g-node.org/EL1000/lyon/commit/296b2a33dc5eea70ad7d0bf334136ac964615b2e> I was trying to recalculate age, but I was getting an error: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar return list(map(*args)) File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/EL1000/metadata.py", line 142, in recover_recording recording['age'] = age(date, child_dob) NameError: name 'age' is not defined I thought this was due to the fact that my original metadata didn't have age, but it did have age (although it was not rounded?). I don't know enough python to be sure - perhaps it was complaining about the age function missing? Probably not, right? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#308 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABI4QHXIGY4YSGP6XRPX2BTUH32J7ANCNFSM5GF6J27Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Lucas Gautheron 06 79 23 86 47 Laboratoire de Sciences Cognitives et Psycholinguistique Bureau 414 - 29 rue d'Ulm 75005 Paris

[alecristia]

oops right! all fixed

[alecristia]

Sorry, I did

pip install git+ssh://[email protected]:/EL1000/tools.git --upgrade

and that seems to have worked, but I'm still getting

Traceback (most recent call last):
File "scripts/metadata.py", line 34, in
children, recordings = importer.process(its)
File "/scratch1/home/acristia/ChildProjectVenv/lib/python3.7/site-packages/EL1000/metadata.py", line 161, in process
self.recordings = pool.map(self.recover_recording, annotations.to_dict(orient = 'records'))
File "/shared/apps/anaconda3/lib/python3.7/multiprocessing/pool.py", line 268, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/shared/apps/anaconda3/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
NameError: name 'age' is not defined

Note that recalculate ages is true & there is an age column in Lyon...

[lucasgautheron]

I did not try Marianne's fix, but I think she really got this fixed since she had to use recompute_ages.
My guess is this is the same issue as before, i.e. you need to pip uninstall EL1000 then pip install git+ssh://[email protected]:/EL1000/tools.git.

recompute_ages may be the safest option for Lyon since the its were fixe, right?

[alecristia]

yes, the bug was fixed (I stopped having it elsewhere), and you're right -- it must be the issue of not a real reinstall having happened. Trying that now!

[alecristia]

confirmed, bug went away! thanks for your patience- i'll remember this next time (i hope!)