Modification: Notary Rubric - Changes to "Engagement in Program" Levels

[dkkapur]

Note: this proposal is being posted on behalf of @Kevin-FF-USA.

Issue Description

Notary engagement expectations / SLAs currently use time as the primary metric of measurement. Here is what the rubric has today:

• L1: <30 minutes of engagement per week, averaged monthly. Engaging activites can be allocations (direct and/or Large Datasets), comments on discussion/issues, attendance on governance calls, messages in Slack, etc.
• L2: <1 hour of engagement per week
• L3: <4 hours of engagement per week
• L4: <8 hours of engagement per week
• L5: >8 hours of engagement per week, averaged monthly

Impact

This ends up being very difficult to track, and very difficult to enforce. Notaries are not held to the standard they commit to, and the application process just results in everyone claiming to commit max time.

Proposed Solution(s)

Change the leveling as follows:

• L1
  • Attend 1 Governance call a month
  • Respond to SLACK inquiries within 5 days
  • Vote/Comment on 1 proposal a month
• L2
  • Attend 1 Governance call a month
  • Join 1 working group
  • Respond to SLACK questions/comments within 2 days.
  • Vote/Comment on 1 proposal a month
• L3
  • Maintain a ranking in Leaderboard in the top 25% of Notaries.
  • Attend 2 Governance calls a month
  • Join 1 working group
  • Respond to SLACK inquiries within 2 days
  • Vote/Comment on 1 proposal a month
• L4
  • Maintain a ranking in Leaderboard in the top 10% of Notaries.
  • Attend 2 Governance calls a month.
  • Join 2 working groups
  • Respond to SLACK inquiries within 2 days
  • Vote/Comment on >2 proposals a month
• L5
  • Maintain a ranking in Leaderboard in the top 5% of Notaries.
  • Attend 2 Governance calls a month - with webcam turned on
  • Join 2 working groups
  • Respond to SLACK inquiries within 24 hours
  • Vote/Comment on all proposals submitted

Timeline

This should ideally be locked in before Notary Applications open - so before Nov 1.

Technical dependencies

Other than tracking Notary activity - nothing blocking.

End of POC checkpoint (if applicable)

N/A

Risks and mitigations

Changes can be reversed/modified in future notary election cycles.

Related Issues

[bmcnabb25]

For the following metrics, how will they be measured?

• Join 1-2 Working Group
• Respond to Slack inquiries within 24-48 hours
• Vote / Comment on 1+ proposals per month

Will someone be responsible for reviewing every notary's behavior on Slack/Github and scoring them on a monthly basis? Or do you have tooling planned to measure this automatically?

[bmcnabb25]

The L5 level requirements are as follows:

• Maintain a ranking in Leaderboard in the top 5% of Notaries.
• Attend 2 Governance calls a month - with webcam turned on
• Join 2 working groups
• Respond to SLACK inquiries within 24 hours
• Vote/Comment on all proposals submitted

Lets say the notary meets every requirement except Vote/Comment on all proposals submitted. Will they still be scored as a L5 notary? Or would they relegated to a L4 score?

i.e. if you meet the majority of the criteria outlined in a certain tier but not all of the criteria, would the notary still be scored for that level? Or would the notary be dropped to a lower level where they do meet all of the criteria?

[Carohere]

The concept of ‘Tiers’ for Notaries. There would be Tiers of participation, where
1 = Notary who votes/proposes
2 = Notary who approves Direct Allocations
3 = Notary who approves LDNs

Hey @Kevin-FF-USA,

• how will the SLA rubrics correspond to the tiering responsibilities proposed before?
• do the v3 notaries who have previously been involved in considerable controversy, remain in V4?

[jiessiee]

Is that means the new notary in the 4th round noly can't get more than 2 points for that they cannot rank in the Leaderboard?

[psh0691]

1. As I mentioned before, I would like you to check the activities of the notary and add them to the plus dashboard.
   ex.) plus.fil.org access/time, signatures, governance call participation…

2. V3 activities shall be evaluated. Because there will be many notaries who will apply for V4.

3. It can be much more effective if the notary's activities are compensated for notary activities rather than relying on voluntary commitments to increase the SLA hierarchy.

[Joss-Hua]

hi, I want to confirm that, what is the ranking standard of the above Leaderboard.

[Kevin-FF-USA]

For these scores, its all about time commitment. What are the Notaries committing to bring to the program.

In the 3rd round application we asked for time commitments, but that quickly became hard to quantify. Ex: If someone came to the Governance meetings, but didn't respond to questions in Github requests then it wasnt much value add for network. Intent of this proposal is to share the levels of which most Notaries are performing to allow newer Notaries a baseline for expectations.

These scores are self reflections and meant to be a forward leaning time commitment.

[Kevin-FF-USA]

Thank you for all the comments in the past Governance Meeting and Fil Lisbon live stream. We merged this feedback into #635 and have updated the final v4 Election Scoring template.

https://docs.google.com/spreadsheets/d/172-sbd5qzdbSofvL_C5FHRyXEsTUpMiKspItygAVJA4/edit?usp=sharing