Permissions rework

[Thingus]

Closes #5091

I have:

• Formatted any Python files with black
• Brought the branch up to date with master
• Added any relevant Github labels
• Added tests for any new additions
• Added or updated any relevant documentation
• Added an Architectural Decision Record (ADR), if appropriate
• Added an MPLv2 License Header if appropriate
• Updated the Changelog

Description

This PR reworks the way FlowAPI and FlowAuth handle query-scoped permissions.
At present, the permissions scopes are generated from a full walk of every set of query trees at every geographic level. This was causing memory issues with the growing number of queries, so this pull request implements a new system.

In this PR, FlowAuth consists of four entities; Servers, Users, Roles and Scopes.

erDiagram
    Servers ||--|{ Roles : contains
    Roles }|--|{ Scopes : "provides permissions for"
    Users }|--|{ Roles : "can work with"
    Servers ||--|{ Scopes : "provides"

Loading

Scopes are applied at the user level, giving permission to:

• read results
• run any queries at all
• use specified queries, aggregations and location metrics to build their queries
• (potentially) define the date range used to run their queries.
  Scopes are automatically populated from a Server when it is registered or updated via a new FlowAPI endpoint
  Roles are set by the FlowAuth admin, and consist of a set of Scopes. Roles are specific to Servers.
  Users are assigned Roles by the FlowAuth admins.

The new auth flow runs as below:

sequenceDiagram
    participant Flowauth
    actor User
    participant FlowAPI
    note over User: Selects roles for token
    User ->>+ Flowauth: roles
    note over Flowauth: Checks user is permitted roles
    Flowauth ->>+ User: signed token w/ roles + scopes
    note over User: Builds query
    User ->>+ FlowAPI: query, token
    note over FlowAPI: Checks query params are covered by a single role
    FlowAPI ->>+ User: query results

Loading

This PR will remove Groups and Capabilities from FlowAuth, being replaced with Roles and Scopes. Tokens will also be removed as a database entitiy; instead, FlowAuth will treat a Token as a view of a combination of a User, a set of Roles and an expiry date.

[cypress]

---

Test summary

4 • 0 • 0 • 0 • 0

---

Run details

Project	FlowAuth
Status	Passed
Commit	cf93b8d
Started	Dec 22, 2022 12:55 PM
Ended	Dec 22, 2022 12:56 PM
Duration	01:09 💡
OS	Linux Debian -
Browser	Electron 106

View run in Cypress Dashboard ➡️

---

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

[jc-harrison]

UserObject.can_get_available_dates() will need to be updated to check for the new 'get_available_dates' scope, rather than the old 'get_result:available_dates'.

-In branch now -John

[Thingus]

Permissions system pull request + changelist

(created by referring to
git diff master...permissions_rework -- 'flowauth/backend'
...subbing in for folder path as appropriate)j

Overarching changes

• Permissions are now granted on 'role' - a 'role' is a collection
  of 'scopes'

  • A role has an expiry date after which it cannot be used to grant scopes
  • A role has a maximum duration that tokens granted using it can last

• Each role is tied to one server.

• A 'user' has a set of 'roles'

• 'Groups' have been removed

• 'scopes' are now either simple or complex (still thinking on that)

• Simple scopes are one of the following strings: ('run',
  'get_available_dates', 'get_results') and represent a server-wide
  capacity granted by that role.

• Complex scopes are a string: '[agg_unit]:[top-level scope]:[inner scope]'
  granting access to a particular Flowmachine query

  • 'agg_unit'; An aggregation unit ('admin1 .. admin3, latlon, 'nonspatial')
    ('nonspatial' wwill be implemented on merge 'get_aggregation_unit' action and 'aggregation_unit' params for all spatial aggregates #5278#)
  • 'top_level query' is a flowmachine ExposedQuery
  • 'inner query' is a child query of ExposedQuery. Each ExposedQuery contains
    itself as an inner query, (ex. 'admin1:daily_users:daily_users')

FlowAPI changes

• 'scopes' now nested inside 'roles' in tokens

/flowapi/flowapi/api_spec.py

• Default scopes are added to role here (53-55)

flowapi/config.py

• Default JWT_IDENTITY_CLAIM changed to 'sub' in line with JWT recs

flowkit/jwt_auth_callbacks.py

• identitiy key replaced with 'sub'

flowkit/permissions.py

• schema_to_scopes now works with the new scope system
• Rest has been rewritten to accomodate the new scope format

flowkit/user_model.py

• UserObject now takes a scopes dictionary
• has_access now works with the new roles, checking that one role
  encompasses all requested scopes in the token

Flowauth backend changes

• Roles have been added (models.py, roles.py)
• Groups have been removed (models.py, groups.py)
• longest_token_life replaced with longest_token_life_minutes
  in all occurences

admin.py

• /tokens endpoint commented out (need to remove properly)

groups.py

• Removed

main.py

• Added an audit logger (check if I'm using this)
• Removed the groups blueprint
• Added '/roles' as the prefix for the interface to roles.py

models.py

• Added roles class + relationships (see diagram)
• Token renamed to TokenHistory (may not take this)
• Edited the User class to use the Role latest and longest in validity check
• Changed Server from having Groups and Capabilities to Roles and Scopes
• Removed all GroupServer classes
• Changed make_demodata to use a limited set of Scopes and Users (may expand back to full?)
• Replaced 'owner' with 'user' and 'expires' with 'expiry'
• (Double check relationship cascade rules a: work and b: are appropriate)

roles.py

• Created
• Implements endpoints for roles editing:
  • listing all roles
  • listing roles for a user
  • getting a specific role
  • creating a new role
  • editing a role (should this merge scopes or replace them with the new ones?)
  • deleting a role
  • getting the members of a role
  • getting the scopes from a role
  • listing all roles for a user on a server
  • listing roles on a server

server.py

• Replaced 'capabilites' endpoint with 'roles' (replication from roles.py - where live?)
• Added endpoint that lists scopes on a server
• Added endpoint that updates scopes on a server
• Added endpoint for activating and deactivating scopes on a server (need to check if I
  went through all the way with this or not)
• Added capabilites to edit_server to change the roles of a server
• (Does SQLAlchemy Cascade take care of deleting server roles?)

token_management.py

• rewrote list_my_servers()
• replaced 'expires' with 'expiry' in token args
• Implemented new token generation (Keep the currently commented-out API side checks?)

user_settings.py

• Added an endpoint that gets a set of roles for the logged-in user

users.py

• Stripped out references to groups
• Added 'roles' to edit_user endpoint

Flowauth frontend changes

AdminMenu.jsx
-Groups replaced with Roles

Dashboard.jsx

• serverName removed for UserServer
• group_admin removed
• role_admin added

GroupDetails.jsx

• Removed

Picker.jsx

• Sanctioned abuse of Javascript functional programming to return the right value

RoleDetails.jsx

• Created
• View for creating and editing roles
• Used to define the following:
  • Role name
  • Server role is associated with
  • Users with a role (capability shared with User view)
  • Expiry date of the role
  • Scopes granted by a role
• Implemented in the functional React style

GroupList.jsx/RoleList.jsx

• Renamed GroupList to RoleList
• Changed the Lister from GroupDetails to RoleDetails

GroupMembersPicker.jsx/RoleMembersPicker.jsx

• Renamed GroupMemebersPicker to RoleMembersPicker
• Replaced getGroupMembers with getRoleMembers

RoleScopePicker.jsx

• Created
• Component for selecting scopes from a tree into a role
• Based on and similar to the existing ServerScopePicker
• Takes as properties a role_id and a server_id and an updateScopes callback
• Presents every scope available to server_id as an item in a CascadePicker
• Presents any role already available to role_id as already checked
• On update, passes new values to updateScopes
• There are extensive comments on the details of this, would encourage reviewer to read

ScopeDetails.jsx

• (Don't think I actually used this in the end, so can maybe remove?)

ServerAdminDetails.jsx

• Removed references to capabilites
• Added references to Scopes
• Updated scopeGraph signature
• (This could do with some renaming of variables to make processes clearer)
• (Also lots of console.log() to remove)

TokenBuilder.jsx

• Created
• View used to make new tokens for a user by selecting their roles
• Only view reachable from a non-admin account
• Takes as properties a server and an onClick callback, usually used to return to prev. menu
• Used to define the following:
  • Token name (validated on update)
  • Roles to be included in token (why would a user not click checkAll?)
• On successful submit, returns user to the server token management view, where the
  new token is visible for copy, view or download.

TokenDetails

• Rewritten to use roles + scopes
• Now deirves expiry from role that is furthest in the future (this should prob. be
  other way around)

TokenList.jsx

• Added error log

TokensRolePicker

• Created
• Component for listing the roles available to build a token
• Takes as properties a list of roles, a checkAll function, a handleToggle function
  and a 'checked' list of checked items

UserAdminDetails.jsx

• Removed groups
• Refactored createUser + editUser to use new API endpoint

UserRoleList.jsx

• Created
• Component used in TokenBuilder to provide a list of roles from a user
• Takes as properties:
  • a list of role objects
  • a checkAll function
  • a handleToggle callback
  • a 'checked' list containing the indicies of checked items in the list of role objects
  • a setRoleState callback (I think this is unused and can be removed)

UserRolePicker.jsx

• Created
• Component used in UserAdminDetails to pick roles for a given user
• Properties:
  • user_id - the ID of the user in question
  • updateRoles - a callback to call when the selected roles are changed

UserServer.jsx

• Replaced TokenDetails with TokenBuilder view
• (Can prob delete TokenDetails view later, but want to use it to model tests on for now)

api.js

• Added 'roles' arg to editUser endpoint call
• Removed group-related functions
• Either removed capability-related functions or replaced 'capability' with 'scopes' or
  'roles' as appropriate
• Replaced 'longest_token_life' with 'longest_token_life_minutes'
• Added endpoint calls for:
  • getting server scopes
  • getting all roles
  • getting specific roles
  • Creating a new role (returns the new role)
  • Deleting a role
  • getting roles for a user
  • getting roles for a user on a server
  • editing a role in it's entireity (editRole)
  • renaming a role (I don't think this is used and it can be removed)
  • editing role members
  • editing role scopes
    • All role edits call the same backend endpoint (PATCH '.../roles/) with a varying json payload
    • (I'm not sure that any apart from editRole are actually used, though)
• Changed the createToken endpoint call to use roles instead of claims

util.js

• Almost full rewrite; most of the functions in here are scope-tree mangling of some form
• Rewrote jsonify to use the new scopes function
• Added docstrings to jsonify
• Added a scopesGraph function that turns a list of scopes into a nested tree
• Impleneted a function for finding the highest common roots of two trees,
  used in RoleScopePicker
• Implemented a function for rotating a list of scopes_with_roles to a list of scopes-from-
  role, again used in the picker functions.

Still to do:

• Integrate James' PR
• Take James' comment on the poll query into consideration
• Check simple permissions work
• Check inside add_role for duplicate roles
• Check expiry rules for tokens
• Unit tests for all new components
• Docstrings for new React components, paying special attention to component properties
• Check for and remove unneded bits from frontend
• Strip out many debug logs from frontend work