Decision: Empty states for supplemental content

Thing	Info
Relevant features	Supplemental content sidebar
Date started	2021-12-16
Date finished	2022-02
Decision status	Implemented
Summary of outcome	We chose the simplest option, "None"

Background/context

Currently supplemental resource categories and subcategories don't appear if they are empty. Empty can mean that corresponding publicly available content was not found by the eRegulations SMEs, or our team hasn’t compiled the content yet.

Based on our usability research, our users need to understand the comprehensiveness of eRegulations, including to help them evaluate whether they need to do more research for a particular question or topic. They would prefer to see a list of all potential categories and brief explanations of why some are empty.

Core questions

How should we present empty states that are clear and precise - that give readers a confident and correct understanding of the status of the content?

A couple of aspects of that question:

• How do we address both causes of missingness, without implying any one cause?
• What information should be inside the empty category, and what content should be in supplemental formats, such as an "i" icon for more information, or on the "about" page?

What we know

User research

We tested a prototype of empty states with a handful of users, and we wrote a research summary: When there are no resources to display, it's more informative and assuring to display all category labels along with category-specific empty state messages.

Key points:

• The majority of participants preferred the empty state design that displayed all categories, combined with category-specific empty state messages (Option B), over the design that only displayed categories with content (Option A).
• Although Option B was preferred, there was some concern over the ambiguity of the empty state copy.
  • One participant questioned why the copy stated, “No related statutes compiled for this subpart” as opposed to something more precise like, “No related statutes have been issued for this subpart.”
  • The empty state message reassured another participant that they weren't simply overlooking a resource. However, they mentioned wanting to verify that there wasn't a resource.

Technical details

Our admin panel has a "show if empty" option for each category and subcategory. If you enable "show if empty", the category shows up without any content or text inside it. We can write stories to change the implementation.

This "show if empty" option enables us to experiment with new categories without automatically listing them on every sidebar.

Things we need to decide + options for them

What should the site say?

Goals

1. Keep language short and simple
2. Ensure the language is informative for users
3. Cover both scenarios of missingness without implying the cause
4. Convey content will be available when either docs/resources are added or they’re released
5. Ensure the language is precise and does not imply more authority or comprehensiveness than it should

Options discussed: focus on "available"

To address the two scenarios resulting in missingness, we need language that conveys either corresponding resources have not been found by the eRegulations SMEs or the content may exist that hasn’t been added yet. Previously, we’ve tested terms like “displayed” or “compiled” and the concern was they implied a root cause. Using a more open-ended term, like “available”, seems preferable.

The term “available” is defined as being able to be used or obtained or at someone's disposal, which likely encapsulates both causes of missingness.

Further, since missingness is potentially a temporal concern (i.e., corresponding supplemental resources could always be released and the team will continue to add new content), the language should convey to users the dynamic nature of the displayed content. However, there will always be some categories for some regulation sections and subparts for which there is no supplemental content.

Potential variations:

• Content currently unavailable
• No content currently available
• SMDLs are currently unavailable
• No SMDLs are currently available for Subpart C
• SMDLs are currently unavailable for display for Subpart C
• SMDLs are currently unavailable. Check back as content is continuously added.
• No publicly available content was found by eRegulations SMEs

Points from discussion:

• "Currently unavailable" could imply that that there is a technical error preventing display of content.
• Choosing a single phrase for all categories and locations would simplify initial implementation.
• We do not want to imply that definitive policy analysis has been done, since it has not been reviewed by CMCS SMEs.
• We want to start as simple as possible with something we can put on the site and test.

Alternative approach based on discussion

For all categories and subcategories that we mark as "show if empty", potential variations:

• None available
• None to display
• None

This would be ambiguous about why, but that is ok. Avoiding implying a root cause is appropriate, because there could be multiple root causes. It's good for our readers to know the right kinds of questions to ask!

It should also be relatively straightforward to implement a single phrase.

We could also add explanatory language, like the "i" in the prototype that said "Resources are curated and updated by Medicaid subject experts." We could link to the about page and provide a more detailed explanation there. But since we're in the middle of redesigning how the supplemental content should be displayed in general, it makes sense to stick to one small incremental change for now.

Consequences

We'd need to implement the chosen empty-state phrase.