Committees: Old Committee members showing

[shrayshray]

Yesterday, I updated the Planning and Programming Committee members in Legistar. On boardagendas.metro.net, the new members are showing, but so are the old members. https://boardagendas.metro.net/committee/planning-and-programming-committee-a48178fe48d3/

https://boardagendas.metro.net/committees/ also shows two chairs and 11 members:

[hancush]

Hi, @shrayshray! Great to be back at it with you. 😊 I've gone in and cleaned up the errant memberships on the site: https://boardagendas.metro.net/committee/planning-and-programming-committee-a48178fe48d3/

Can you say a bit more about how you updated the memberships? It seems we got all of the incorrect ones in mid-July, and I no longer see them in the API. Were they deleted from Legistar?

Leaving myself a log, for posterity:

>>> Person.objects.get(family_name='Barger').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3').values_list('start_date', 'end_date', 'role', 'updated_at')
<QuerySet [('2021-07-01', '2022-06-30', 'Member', datetime.datetime(2021, 9, 2, 4, 7, 25, 579922, tzinfo=<UTC>)), ('2021-07-01', '2022-06-30', 'Chair', datetime.datetime(2021, 7, 15, 4, 10, 33, 873320, tzinfo=<UTC>)), ('2017-07-01', '2018-06-30', 'Member', datetime.datetime(2020, 3, 24, 15, 10, 48, 897245, tzinfo=<UTC>)), ('2017-01-01', '2017-06-30', 'Member', datetime.datetime(2020, 3, 24, 15, 10, 48, 856112, tzinfo=<UTC>))]>
>>> Person.objects.get(family_name='Barger').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3', end_date='2022-06-30', role='Chair').delete()

>>> Person.objects.get(family_name='Dutra').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3').values_list('start_date', 'end_date', 'role', 'updated_at')
<QuerySet [('2021-07-01', '2022-06-30', 'Member', datetime.datetime(2021, 7, 15, 4, 10, 34, 902676, tzinfo=<UTC>))]>
>>> Person.objects.get(family_name='Dutra').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3', end_date='2022-06-30').delete()

>>> Person.objects.get(family_name='Mitchell').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3').values_list('start_date', 'end_date', 'role', 'updated_at')
<QuerySet [('2021-07-01', '2022-06-30', 'Member', datetime.datetime(2021, 7, 15, 4, 10, 34, 438341, tzinfo=<UTC>))]>
>>> Person.objects.get(family_name='Mitchell').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3', end_date='2022-06-30').delete()

>>> Person.objects.get(family_name='Bonin').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3').values_list('start_date', 'end_date', 'role', 'updated_at')
<QuerySet [('2021-07-01', '2022-06-30', 'Member', datetime.datetime(2021, 7, 15, 4, 10, 33, 237468, tzinfo=<UTC>)), ('2020-07-01', '2021-06-30', 'Member', datetime.datetime(2020, 8, 2, 0, 40, 12, 103255, tzinfo=<UTC>)), ('2014-07-01', '2015-06-30', 'Chair', datetime.datetime(2020, 3, 24, 15, 10, 48, 690803, tzinfo=<UTC>)), ('2017-07-01', '2018-06-30', 'Member', datetime.datetime(2020, 3, 24, 15, 10, 48, 648163, tzinfo=<UTC>)), ('2018-07-01', '2019-06-30', 'Member', datetime.datetime(2020, 3, 24, 15, 10, 48, 565012, tzinfo=<UTC>)), ('2019-07-01', '2020-06-30', 'Member', datetime.datetime(2020, 3, 24, 15, 10, 48, 430846, tzinfo=<UTC>)), ('2015-07-01', '2016-06-30', 'Chair', datetime.datetime(2020, 3, 24, 15, 10, 48, 102101, tzinfo=<UTC>))]>
>>> Person.objects.get(family_name='Bonin').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3', end_date='2022-06-30').delete()

>>> Person.objects.get(family_name='Krekorian').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3').values_list('start_date', 'end_date', 'role', 'updated_at')
<QuerySet [('2021-07-01', '2022-06-30', 'Vice Chair', datetime.datetime(2021, 7, 15, 4, 10, 34, 836038, tzinfo=<UTC>)), ('2017-01-01', '2017-06-30', 'Vice Chair', datetime.datetime(2020, 3, 24, 15, 10, 48, 355968, tzinfo=<UTC>)), ('2016-07-01', '2016-12-31', 'Vice Chair', datetime.datetime(2020, 3, 24, 15, 10, 48, 209322, tzinfo=<UTC>))]>
>>> Person.objects.get(family_name='Krekorian').memberships.filter(organization__councilmatic_organization__slug='planning-and-programming-committee-a48178fe48d3', end_date='2022-06-30').delete()

[hancush]

What happened?

• Replicated 2020-1 memberships for 2021-2
• Created new memberships once updated info came in
• Replicated memberships that were not correct deleted

I'll open an issue to keep track of membership deletions so we get a better idea in a central place of how often this is happening, and open an issue on the scrapers about a revised mechanism for importing memebrships, where we delete and reimport everything on each scrape.