Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Exclude replaced documents from IESG discusses #7712

Merged
merged 3 commits into from
Jul 21, 2024

Conversation

microamp
Copy link
Collaborator

Fixes #7179.

I'm retaining @pselkirk's commit from the PR, #7258.

I've updated the existing test, ietf.iesg.tests.IESGTests.test_feed, to ensure that replaced documents from IESG discusses are correctly filtered.

ietf/manage.py test ietf.iesg.tests.IESGTests.test_feed --settings=settings_test
...
Ran 7 tests in 18.513s

OK

@microamp
Copy link
Collaborator Author

Interesting observation:

possible_docs = possible_docs.exclude(states__type="draft", states__slug="repl")

adds a rather complex conditional to the existing query.

WHERE (
  -- the existing logic below
  (
    ("doc_state"."slug" IN ('lc', 'writeupw', 'goaheadw', 'iesg-eva', 'defer') AND "doc_state"."type_id" = 'draft-iesg')
    OR
    ("doc_state"."slug" IN ('intrev', 'iesgrev') AND "doc_state"."type_id" = 'charter')
    OR
    ("doc_state"."slug" IN ('iesgeval', 'defer') AND "doc_state"."type_id" IN ('statchg', 'conflrev'))
  )
  AND
  "name_ballotpositionname"."blocking"
  -- and with the new exclude filter below
  AND
  NOT (
    EXISTS(
      SELECT 1 AS "a" FROM "doc_document_states" U1 INNER JOIN "doc_state" U2 ON (U1."state_id" = U2."id")
      WHERE (U2."slug" = 'repl' AND U1."document_id" = ("doc_document"."id"))
      LIMIT 1
    )
    AND
    EXISTS(
      SELECT 1 AS "a" FROM "doc_document_states" U1 INNER JOIN "doc_state" U2 ON (U1."state_id" = U2."id")
      WHERE (U2."type_id" = 'draft' AND U1."document_id" = ("doc_document"."id"))
      LIMIT 1
    )
  )
)

I tried adding it to the main filter statement instead, but it generates a even more verbose one.

WHERE (
  -- the existing logic here
  AND
  NOT (
    EXISTS(
      SELECT 1 AS "a" FROM "doc_document_states" U1 INNER JOIN "doc_state" U2 ON (U1."state_id" = U2."id")
      WHERE (U2."slug" = 'repl' AND U1."id" = ("doc_document_states"."id") AND "doc_document_states"."document_id" = ("doc_document"."id"))
      LIMIT 1
    )
    AND
    EXISTS(
      SELECT 1 AS "a" FROM "doc_document_states" U1 INNER JOIN "doc_state" U2 ON (U1."state_id" = U2."id")
      WHERE (U2."type_id" = 'draft' AND U1."id" = ("doc_document_states"."id") AND "doc_document_states"."document_id" = ("doc_document"."id"))
      LIMIT 1
    )
  )
)

Let me know if I'm reading it wrong.

Copy link

codecov bot commented Jul 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.79%. Comparing base (b00dfd3) to head (57fdba4).
Report is 11 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7712      +/-   ##
==========================================
- Coverage   88.79%   88.79%   -0.01%     
==========================================
  Files         296      296              
  Lines       41319    41324       +5     
==========================================
+ Hits        36688    36692       +4     
- Misses       4631     4632       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@microamp microamp marked this pull request as ready for review July 20, 2024 19:35
@microamp microamp requested a review from rjsparks July 20, 2024 19:35
rjsparks
rjsparks previously approved these changes Jul 20, 2024
@rjsparks
Copy link
Member

You aren't reading it wrong, but it points out that we are getting lucky with a pattern we are using to filter things with.

@rjsparks
Copy link
Member

after discussion with Jennifer - we should change the exclude to work against a specific State (the form here works with .filter but not with .exclude.

@jennifer-richards
Copy link
Member

The exclude() query is due to behavior documented in the second "Note" box here: https://docs.djangoproject.com/en/5.0/topics/db/queries/#spanning-multi-valued-relationships

The upshot is that filter(states__type="draft", states__slug="repl") will do what we want by combining the type and slug conditions with an AND. For exclude(), you have to add a second query and it should look like exclude(states__in=State.objects.filter(type="draft", slug="repl")

@microamp
Copy link
Collaborator Author

microamp commented Jul 21, 2024

Replacing the existing exclude statement with

possible_docs = possible_docs.exclude(states__in=State.objects.filter(type="draft", slug="repl"))

generates a more compact query, but still has the NOT (EXISTS (... LIMIT 1)) pattern to it.

WHERE (
  -- the existing logic here
  AND
  NOT (
    EXISTS(
      SELECT 1 AS "a" FROM "doc_document_states" V1 WHERE (
        V1."state_id" IN (
          SELECT U0."id" FROM "doc_state" U0 WHERE (U0."slug" = 'replaced' AND U0."type_id" = 'draft')
        )
        AND
        V1."document_id" = ("doc_document"."id")
      )
      LIMIT 1
    )
  )
)

@microamp
Copy link
Collaborator Author

Here's the entire query after the change.

SELECT DISTINCT "doc_document"."id", "doc_document"."time", "doc_document"."type_id", "doc_document"."title", "doc_document"."stream_id", "doc_document"."group_id", "doc_document"."abstract", "doc_document"."rev", "doc_document"."pages", "doc_document"."words", "doc_document"."intended_std_level_id", "doc_document"."std_level_id", "doc_document"."ad_id", "doc_document"."shepherd_id", "doc_document"."expires", "doc_document"."notify", "doc_document"."external_url", "doc_document"."uploaded_filename", "doc_document"."note", "doc_document"."internal_comments", "doc_document"."rfc_number", "doc_document"."name", "name_streamname"."slug", "name_streamname"."name", "name_streamname"."desc", "name_streamname"."used", "name_streamname"."order", "group_group"."id", "group_group"."time", "group_group"."name", "group_group"."state_id", "group_group"."type_id", "group_group"."parent_id", "group_group"."description", "group_group"."list_email", "group_group"."list_subscribe", "group_group"."list_archive", "group_group"."comments", "group_group"."meeting_seen_as_area", "group_group"."used_roles", "group_group"."uses_milestone_dates", "group_group"."acronym", "group_group"."charter_id", "person_person"."id", "person_person"."user_id", "person_person"."time", "person_person"."name", "person_person"."plain", "person_person"."ascii", "person_person"."ascii_short", "person_person"."pronouns_selectable", "person_person"."pronouns_freetext", "person_person"."biography", "person_person"."photo", "person_person"."photo_thumb", "person_person"."name_from_draft"
FROM "doc_document"
INNER JOIN "doc_document_states" ON ("doc_document"."id" = "doc_document_states"."document_id")
INNER JOIN "doc_state" ON ("doc_document_states"."state_id" = "doc_state"."id")
INNER JOIN "doc_docevent" ON ("doc_document"."id" = "doc_docevent"."doc_id")
INNER JOIN "doc_ballotpositiondocevent" ON ("doc_docevent"."id" = "doc_ballotpositiondocevent"."docevent_ptr_id")
INNER JOIN "name_ballotpositionname" ON ("doc_ballotpositiondocevent"."pos_id" = "name_ballotpositionname"."slug")
LEFT OUTER JOIN "name_streamname" ON ("doc_document"."stream_id" = "name_streamname"."slug")
LEFT OUTER JOIN "group_group" ON ("doc_document"."group_id" = "group_group"."id")
LEFT OUTER JOIN "person_person" ON ("doc_document"."ad_id" = "person_person"."id")
WHERE (
  (
    ("doc_state"."slug" IN ('lc', 'writeupw', 'goaheadw', 'iesg-eva', 'defer') AND "doc_state"."type_id" = 'draft-iesg')
    OR
    ("doc_state"."slug" IN ('intrev', 'iesgrev') AND "doc_state"."type_id" = 'charter')
    OR
    ("doc_state"."slug" IN ('iesgeval', 'defer') AND "doc_state"."type_id" IN ('statchg', 'conflrev'))
  )
  AND
  "name_ballotpositionname"."blocking"
  AND
  NOT (
    EXISTS(
      SELECT 1 AS "a" FROM "doc_document_states" V1
      WHERE (
        V1."state_id" IN (
          SELECT U0."id" FROM "doc_state" U0 WHERE (U0."slug" = 'replaced' AND U0."type_id" = 'draft')
        )
        AND
        V1."document_id" = ("doc_document"."id")
      )
      LIMIT 1
    )
  )
)

@rjsparks rjsparks merged commit a3e4e63 into ietf-tools:main Jul 21, 2024
9 checks passed
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replaced documents show on https://datatracker.ietf.org/iesg/discusses/ - they shouldn't.
4 participants