Decision: Display of rules and NPRMs in sidebar

Thing	Info
Relevant features	Supplemental content sidebar
Date started	2021-10-18
Date finished	2022-01
Decision status	Done
Summary of outcome	We decided to implement grouped FR docs in sidebars, and we did!

Background/context

When you need to interpret or update a piece of regulation, you often need to look up the final rules and proposed rules (NPRMs) that contributed to that piece, because they include essential background information about CMCS regulatory decisions, particularly in the rule preambles. It can be very tedious to track down whole sequences of NPRMs and final rules.

We want to display relevant final rules in the sidebar subpart-by-subpart (and ideally also section-by-section), and we want those final rules to be connected to their relevant NPRMs. See our supplemental content sidebar prototype, under "Rules with Preambles". Our users have been very excited about this prototype.

Our users asked us to include RFIs in eRegs too, since they can also be helpful for understanding CMCS decisions.

Core questions

How can we do this?

How far can we get to that goal in an automated fashion? What elements might need to be done by hand?

What should we do as helpful intermediate steps in iterative development toward this goal?

What we know

Rules and NPRMs cite relevant CFR parts and sections

Each rule and NPRM includes unstructured information about which sections it amends. For example, for this proposed rule, it says:

• In the Preamble: "we are proposing to amend § 447.502"
• In the end part where it describes the specific proposed change: "Amend § 447.502 by revising the definitions of “States” and “United States” to read as follows"

Each rule and NPRM also includes a structured list of the parts it is related to. For example, for that same rule, the right sidebar includes: 42 CFR 433 42 CFR 438 42 CFR 447 42 CFR 456. The Federal Register API includes this structured data for documents (cfr_references).

RFIs may not have structured info about relevant CFR parts and sections

RFIs often mention some sections or parts of the regulations but may not suggest any specific amendments to a section or part. The FR structured data for an RFI may say "42 CFR chapter undef" - for example, see this January 2020 RFI and this May 2020 RFI. However, the second RFI mentions 431.52 in the unstructured (handwritten) text, and it could be helpful to readers researching 431.52.

Older FR documents have limited or no structured data available

The FR API only has data going back to 1994. Documents from the 1990s may have incomplete information in the Federal Register API, either plain text or scans without OCR - for example, see this 1994 rule available in plain text.

There are many older FR documents that are still important to our readers. For example, 42 CFR § 456.2 has the following footnote indicating that its sources are all rules from the 1980s: [46 FR 48566, Oct. 1, 1981, as amended at 50 FR 15327, Apr. 17, 1985; 51 FR 43198, Dec. 1, 1986]. The best that the Federal Register website can do in an automated way is provide a link to a not-very-readable yellowish scan. This system has some errors - it doesn't always fetch a relevant PDF.

The Library of Congress also has PDFs of old rules available, typically in better scans like this one. When we tested this aspect of cross-reference linking, our readers preferred the LoC PDFs that Rebecca had selected by hand for the older rules. From our study summary:

Both options for older rules had issues: page load timeouts with the Library of Congress (LOC) scans vs Federal Register’s less readable scans. Participants vastly preferred being linked to the Library of Congress PDFs for older rules instead of the Federal Register citation url, even though they ran the risk of page load timeouts.

CFR parts, subparts, and sections cite relevant FR documents

CFR text includes semi-structured data about its relationship to FR documents:

• Each part has a "Source" (FR document) statement at the beginning.

  • Example: Part 438 says "Source: 67 FR 41095, June 14, 2002, unless otherwise noted."

• Some subparts: have a "Source" statement at the beginning.

  • Example: Part 438 Subpart A says "Source: 81 FR 27853, May 6, 2016, unless otherwise noted."

• Most sections have footnotes that tell which FR documents established that section and updated it.

  • Example: Part 438, Subpart A, Section 438.4 says: "[81 FR 27853, May 6, 2016, as amended at 85 FR 72837, Nov. 13, 2020; 85 FR 72837, Nov. 13, 2020]"

Related documents have related docket numbers

Each FR document has a structured "Agency/Docket Number". We may be able to use this docket number to automatically group Final Rules and NPRMs.

• For example, for that same May 2021 proposed rule, the number is CMS-2482-P2. This number is available via the FR API (docket_ids).
• That proposed rule says that it is proposing to delay the implementation of a December 2020 final rule (85 FR 87000) that has the docket ID of CMS-2482-F. It has the same list of related parts: 42 CFR 433 42 CFR 438 42 CFR 447 42 CFR 456.
• That final rule references a June 2020 proposed rule with the number of 85 FR 37256, but the FR site links it to an unrelated rule because there appears to be a typo (?!?). The actual proposed rule it should reference is 85 FR 37286, which has a docket ID of CMS-2482-P. Also same list of related parts: 42 CFR 433 42 CFR 438 42 CFR 447 42 CFR 456.

We're not sure how consistent CMS has been over the last few decades about using these docket numbers in a structured way consistently.

Also, as that example shows, there is not a simple 1:1 relationship between a Final Rule and an NPRM. It's messy:

• A Final Rule can be related to a later followup NPRM
• A Final Rule can be related to multiple earlier NPRMs
• A NPRM can be related to multiple earlier Final Rules
• There can be a NPRM that never has a Final Rule
• There can be an Interim Final Rule that didn't have an NPRM (although rare)

What we've done so far

So far, we've used the FR API to create lists of documents relevant to specific parts (rules, NPRMs, and RFIs). We display them in our subpart sidebars and on our homepage.

What we don't know

• Can we collect up the semi-structured FR citations in the CFR text and use them to create section-by-section and subpart-by-subpart lists of related final rules? (Can we automate that reliably?)
• Can we use the docket numbers to group together related NPRMs and Final Rules? (Can we automate that reliably? If we can't automate that reliably, can we build a way for a program to do a rough pass that a person can then correct in our admin UI?)

Things we need to decide + options for them

How should we handle older NPRMs and Final Rules in the sidebar?

This is a question both from a UX perspective (how should it behave?) and a data perspective: should there be some way for a content author to input additional FR documents by hand that can be aggregated with the automated data?

How could we add section info to RFIs?

This is also a question about adding manual data to enrich the automated data.

Should we add the titles of Final Rules, NPRMs, and RFIs to the items people can search?

People have asked for a multi-faceted search that would cover reg text and the titles of supplemental content - should the FR documents be in there too?

The Federal Register API includes a way to search the FR. We would need to review the fields available for filtering and determine what fields we want to filter on.

What could be a good sequence of stories to get toward these goals?

Stories that exist:

• Spike: Can we correlate NPRMs and Final Rules automatically?
• Spike: Can we create lists of subpart by subpart rules?

Consequences