Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix Significant Date facet and add Fiscal Year facet #727

Merged
merged 11 commits into from
Sep 3, 2021
Merged

fix Significant Date facet and add Fiscal Year facet #727

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
0a8f3c5
make Date facet singular
Jun 1, 2021
b3a8222
add new fiscal_year facet based on plan_program_policy facet
Jun 1, 2021
9bd516c
add migration for fiscal_date facet
Jun 2, 2021
f95a613
debug missing facets & failing event scrape
Jun 8, 2021
c9f7b45
ensure legislative_session in search_indexes
Jun 8, 2021
2a45c6e
fix fiscal year facet logic
Jun 10, 2021
11b7d9e
remove legislative_session where it is redundant or unnecessary
Jun 15, 2021
1622456
refine logic for prepare_legislative_session
Jun 16, 2021
689d07f
working through modifying test_legislative_session
Jun 16, 2021
a5d02a0
continue fleshing out test_legislative_session
Jun 22, 2021
af7dbf9
Revise Solr prep test
hancush Jun 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,6 @@ debug.log
/configs/settings_deployment.staging.py

*.csv
.gnupg/
.gnupg/

lametro/secrets.py
2 changes: 2 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ services:
DATABASE_URL: 'postgis://postgres:@postgres/lametro'
SHARED_DB: "True"
command: sh -c 'pupa update --rpm=600 lametro people && pupa update --rpm=600 lametro bills window=30 && pupa update --rpm=600 lametro events'
volumes:
- ./lametro/secrets.py:/app/lametro/secrets.py

volumes:
lametro-solr-data:
Expand Down
5 changes: 4 additions & 1 deletion lametro/management/commands/refresh_guid.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ class ClassificationMixin:
'Subregion',
),
'significant_date_exact': (
'Dates',
'Date',
),
'motion_by_exact': (
'Board Member',
Expand All @@ -78,6 +78,9 @@ class ClassificationMixin:
'Program',
'Policy'
),
'legislative_session': (
'Legislative Session'
)
}

@property
Expand Down
18 changes: 18 additions & 0 deletions lametro/migrations/0007_auto_20210608_1353.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Generated by Django 2.2.23 on 2021-06-08 20:53

from django.db import migrations, models


class Migration(migrations.Migration):

dependencies = [
('lametro', '0006_add_plan_program_policy'),
]

operations = [
migrations.AlterField(
model_name='lametrosubject',
name='classification',
field=models.CharField(choices=[('bill_type_exact', 'Board Report Type'), ('lines_and_ways_exact', 'Lines / Ways'), ('phase_exact', 'Phase'), ('project_exact', 'Project'), ('metro_location_exact', 'Metro Location'), ('geo_admin_location_exact', 'Geographic / Administrative Location'), ('significant_date_exact', 'Significant Date'), ('motion_by_exact', 'Motion By'), ('topics_exact', 'Subject'), ('plan_program_policy_exact', 'Plan, Program, or Policy'), ('legislative_session', 'Legislative Session')], default='topics_exact', max_length=256),
),
]
3 changes: 2 additions & 1 deletion lametro/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -911,7 +911,8 @@ class LAMetroSubject(models.Model):
('significant_date_exact', 'Significant Date'),
('motion_by_exact', 'Motion By'),
('topics_exact', 'Subject'),
('plan_program_policy_exact', 'Plan, Program, or Policy')
('plan_program_policy_exact', 'Plan, Program, or Policy'),
('legislative_session', 'Legislative Session')
]

class Meta:
Expand Down
15 changes: 13 additions & 2 deletions lametro/search_indexes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ class LAMetroBillIndex(BillIndex, indexes.Indexable):
significant_date = indexes.MultiValueField(faceted=True)
motion_by = indexes.MultiValueField(faceted=True)
plan_program_policy = indexes.MultiValueField(faceted=True)
legislative_session = indexes.MultiValueField(faceted=True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already defined on the base index in django-councilmatic – why redefine here?


def get_model(self):
return LAMetroBill
Expand Down Expand Up @@ -50,8 +51,18 @@ def prepare_attachment_text(self, obj):
)

def prepare_legislative_session(self, obj):
start_year = obj.legislative_session.identifier
end_year = int(start_year) + 1
if len(obj.actions_and_agendas) > 0:
most_recent = sorted(obj.actions_and_agendas, key=lambda i: i['date'],reverse=True)[0]
action_date = most_recent['event'].start_time
Copy link
Collaborator

@hancush hancush Jun 11, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't quite right. The desired logic is to return the session associated with the most recent agenda date. If there are no agenda dates, return the session associated with the most recent action date. If there is neither an agenda date nor an action date, return None.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you propose how you'd go about getting the desired date, using the output of obj.actions_and_agendas?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I'm thinking about it, I would have to add additional logic to check if the most_recent event is an agenda (description being SCHEDULED) or an action. I would use the agenda if possible, otherwise using an action as the most_recent.

Either way, I can use most_recent['date'], unless a datetime instance is preferable to a date instance for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hancush am I on the right track with this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend something like:

actions_and_agendas = sorted(obj.actions_and_agendas, key=lambda i: i['date'], reverse=True)
agendas = [item for item in actions_and_agendas if it is an agenda]  # will be sorted bc list order is preserved in python
if agendas:
    # return the most recent date from your agendas list
else:
    # there are no agendas, i.e., everything in actions_and_agendas is an action
    # return the most recent date from your actions_and_agendas list

else:
action_date = obj.get_last_action_date()

if action_date.month <= 6:
start_year = action_date.year - 1
end_year = action_date.year
else:
start_year = action_date.year
end_year = action_date.year + 1

session = '7/1/{start_year} to 6/30/{end_year}'.format(start_year=start_year,
end_year=end_year)
Expand Down