Decision: Content review workflow

Thing	Info
Relevant features	Content authoring
Date started	2021-09-01
Date finished
Decision status
Summary of outcome

Background/context

Our policy SME team members and our CMS partner policy SMEs need to be able to review our supplemental content in systematic ways, for both internal checking of work on entering content and for CMS review and approval of the content.

Core questions

How should we enable our CMS partner policy SMEs to do review work?

How would we incorporate their comments and actions (such as approvals, additions of sections, etc)?

What we know

Capabilities and tradeoffs

It is possible to modify our admin panel to provide various functionality, including providing views of the data or downloadable CSVs/spreadsheets. When people are logged in, we can record their actions to provide creation/audit/review trails.

Our options include data review directly within an admin panel (with information added) or in an editable spreadsheet.

User expertise and tool access

Some policy SMEs may be comfortable with complex spreadsheets of content, but would need to validate overall competency and comfort with format.

CMS policy SMEs usually don’t have access to Google Sheets for collaboration, but some of them use Box.net for a similar purpose - you can upload a spreadsheet and work on it in the browser together. Could also potentially use SharePoint but requires access permissions.

What do people need to do?

People may need a "regulation-first" view (view all pieces of supplemental content associated with a specific regulation subpart or section) or a "supplemental-content-first" view (view all regulation sections/subparts associated with a specific piece of supplemental content).

Both internal and partner SME review work may include:

• Make sure content is entered correctly (no typos, no accidental mixups, etc)
• See if anything is missing on accident (such as something not getting saved properly due to an issue with the content authoring workflow)
• Make sure the database already has all the relevant information that I know about (such as all the relevant content for a particular part or section, or all the relevant section tags for a piece of content)
• Check whether an obsolete or withdrawn piece of content is included
• Check whether a brand-new piece of content is included
• Review what’s been added or changed since the last time they checked
• Check whether supplemental content is tagged to the appropriate subpart/section, including if a subpart/section is missing or needs to be deleted
• Determine whether a document or resource link should not be included on the site
• Check whether supplemental content is tagged to the right category

Views that would be helpful for review work:

• Choose a regulation subpart or section and see a readable list of the documents attached to that subpart or section
• Choose a document and see a readable list of the regulation sections that are attached to it
• See a list of documents that don't have any regulation sections attached to them
• See a list of regulation subparts and sections that don't have any documents attached to them
• Choose a category and see what documents are are attached to that category

What specific workflow views and actions do we anticipate?

Information needed for each piece of subregulatory guidance: document info, what it's tagged to, who added it, who last updated it, who approved it.

Actions a reviewer could do (or recommend doing): delete, approve, add or delete section, delete document. We may need some form of permissions management, for example if people are allowed to directly delete items.

Things we need to decide + options for them

So far, we’ve discussed potential ways to support reviewing content:

• Within the admin panel, enable the policy SME to pull up various “report” views that give them insight into the data in the system
• Within the admin panel, enable exporting a CSV/spreadsheet that a policy SME could use for analysis, insight, and review

We’ve also discussed ways to enable managing/approving content:

• Within the admin panel, enable policy SMEs to approve, delete, hide, or comment on items
• A spreadsheet of content that has columns for approving, deleting, hiding, and commenting on items (team policy SME then inputs any necessary changes into the admin panel)

And ways to support oversight and transparency in general:

• Provide a detailed “recent actions” view in the admin panel
• Provide a log on the website of recent actions by user
• Enable exporting supplemental content information from the website as a CSV/spreadsheet

What do we need to know before we can decide?

How can we learn more, including prototypes people can try and questions we can ask?

We're already trying out a spreadsheet-based prototype for policy SME review! We should see how that goes.

Conduct usability testing on the admin tool and/or spreadsheets with policy SMEs. (We'd need to describe our usability testing goals.)

Consequences