Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

persons in memberships created through org.add_member won't resolve if member not in DB #254

Open
fgregg opened this issue Feb 20, 2017 · 1 comment

Comments

@fgregg
Copy link
Contributor

fgregg commented Feb 20, 2017

Members added through the add_member method of the Organization class will not resolve unless the member is already in the DB (from a previous scrape).

The correct behavior would be to also resolve against persons that have are only in the current import. r

relates to #253

I'm seeing this with the Illinois scraper. My current work around it to run the scraper twice. The first time. I disable the committee scraper and all the people are imported. The second time I can turn on the committee scraper and memberships resolve.

@jamesturk
Copy link
Member

I think this and some of the other cross-linking issues are probably going to best be resolved by a second linkage step within import. It'd be nice if we could have this all happen together instead of some being at import time and others being later.

rough outline:

  • possibly need to relax DB constraints (if there are any linkages that are currently enforced at that level) so that the objects can wind up in the DB prior to the links being created
  • then, instead of each importer having the option to be dependent on the importers that run before it, importers could keep track of a list of entities needing resolution
  • after all imports are done, their lists of entities needing resolution can be processed in any order, since the import phase will be done

exceptions:

  • Jurisdiction importer can be guaranteed to run first
  • PostImporter can probably remain dependent on Org
  • MembershipImporter can probably remain dependent on Org,Person,Post

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants