Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

integrate lmdb rust #125

Open
wants to merge 282 commits into
base: main
Choose a base branch
from
Open

integrate lmdb rust #125

wants to merge 282 commits into from

Conversation

Alexanderlacuna
Copy link
Contributor

integrate lmdb rust

fredmanglis and others added 30 commits November 14, 2022 11:37
* migrations/auth/20221110_05_BaNtL-create-roles-table.py: modify migration
* migrations/auth/20221114_03_PtWjc-create-group-roles-table.py: new migration
* tests/unit/auth/test_migrations_create_tables.py: test new migration
* tests/unit/auth/test_migrations_indexes.py: test new migration
* migrations/auth/20221114_04_tLUzB-initialise-basic-roles.py: new migration
* tests/unit/auth/test_migrations_insert_data_into_empty_table.py: test new
  migration
Some roles will not be user editable to prevent inconsistencies, and possible
privilege escalation.

* migrations/auth/20221110_05_BaNtL-create-roles-table.py: Add `user_editable`
  column to table
* migrations/auth/20221114_04_tLUzB-initialise-basic-roles.py: Set
  `group-leader` role as not user editable
* gn3/auth/authorisation/__init__.py: Add `authorised_p` decorator to be used
  for all function requiring authorisation.
* gn3/auth/authorisation/groups.py: Add `create_group` function stub
* tests/unit/auth/conftest.py: Add fixture for test users
* tests/unit/auth/test_groups.py: Add tests for `create_group`
* migrations/auth/20221114_05_hQun6-create-user-roles-table.py: new migration
* tests/unit/auth/test_migrations_create_tables.py: test new migration
* tests/unit/auth/test_migrations_indexes.py: test new migration
* The test app and the test client are both needed as fixtures in different
  places
* tests/unit/auth/test_groups.py: use Flask's application context directly
  rather than the request context to access `g` in order to get rid of the
  `DeprecationWarning`.
* tests/unit/auth/conftest.py: use the functions in `gn3.auth.db` to acquire
  the database connection and cursor since they handle some of the basic
  issues like rollback and commit, meaning we do not have to explicitly handle
  said issues in the fixtures.
* gn3/auth/authorisation/__init__.py: delete function
* gn3/auth/authorisation/checks.py: move function to `checks` module
Use specified types for privileges, roles and types rather than using strings
to help with limiting bugs.

* gn3/auth/authorisation/groups.py: Specify and use the `Group` type
* gn3/auth/authorisation/privileges.py: Specify and use the `Privilege` type
* gn3/auth/authorisation/roles.py: Specify the `Role` type. Add the
  `create_role` function.
* gn3/auth/authorisation/checks.py: Return results of calling the function
  rather than a dict of values that include the results.
* gn3/auth/authorisation/groups.py: Use the newer form of `authorised_p`
  decorator.
* tests/unit/auth/test_groups.py: Update tests
* gn3/db/rdf.py: Delete gn3.setting.SPARQL_ENDPOINT import.
(sparql_query): Inject SPARQLWrapper.
(get_dataset_metadata): Ditto.
* tests/unit/auth/test_roles.py: new tests.
* gn3/auth/authorisation/privileges.py: Set id to UUID type
* gn3/auth/authorisation/roles.py: fix parameters to types that sqlite3
  supports
* gn3/auth/db.py: add logging for errors and re-raise the exception
* tests/unit/auth/test_roles.py: fix test
* migrations/auth/20221116_01_nKUmX-add-privileges-to-group-leader-role.py:
  new migration to fix data errors.
* tests/unit/auth/test_privileges.py: test privileges
* migrations/auth/20221117_01_RDlfx-modify-group-roles-add-group-role-id.py:
  new migration
* tests/unit/auth/test_migrations_add_remove_columns.py: test new migration
* gn3/auth/authentication.py -> gn3/auth/authentication/__init__.py: Convert
  module to package
* gn3/auth/authentication/users.py: Define the `User` type
* tests/unit/auth/conftest.py: Add fixtures to help with testing
* tests/unit/auth/test_groups.py: Add incomplete and failing test
* migrations/auth/20221117_02_fmuZh-create-group-users-table.py: new migration
* tests/unit/auth/test_migrations_create_tables.py: test new migration
* tests/unit/auth/test_migrations_indexes.py: test new migration
* gn3/auth/authorisation/groups.py: Add `GroupRole` type. Fix typing
  annotations. Fix bugs.
* tests/unit/auth/conftest.py: Fix bugs.
* tests/unit/auth/test_groups.py: Fix test to run.
fredmanglis and others added 29 commits April 17, 2023 14:42
* gn3/api/metadata.py: Import Template, sparql_query and RDF_PREFIXES.
(get_genewiki_entries): New endpoint.
* gn3/db/rdf.py: Add new constant for storing rdf prefixes.

Signed-off-by: Munyoki Kilyungi <[email protected]>
Signed-off-by: Munyoki Kilyungi <[email protected]>
Consistently encode all values for the top-level keys stored in redis to avoid
issues with json encode/decode
Fix bugs with setting up of the selected traits for use while filtering the
search results.
During development, we need logging sometimes to help with troubleshooting
problems. This commit provides a module to help set up the logging in a
separate module from the app module.
Previously, the `oauth2/data/authorisation` endpoint was returning hard-coded
values for the privileges assigned to the user for each resource. In this
change, we rework to return the actual privileges for the user.
Some roles should not be user-editable, and as such, we need to check before
allowing any edits on such roles. This commit makes that possible.
While creating new group roles, enable the listing of non-resource privileges,
e.g. `system:group:*` and `system:user:*` that the user has to allow for them
to be used in role creation.
With the assignment of `system:*` privileges to roles, we need to check for
their existence when doing authorisation.

This commit provides a hack for that, seeing as user groups (and the system
itself) are not treated as resources, and therefore the way to fetch the
privileges is not entirely consistent.
Implement the "Authorization Code Flow" for the authentication of users.

* gn3/auth/authentication/oauth2/grants/authorisation_code_grant.py: query and
  save the authorisation code.
* gn3/auth/authentication/oauth2/models/authorization_code.py: Implement the
  `AuthorisationCode` model
* gn3/auth/authentication/oauth2/models/oauth2client.py: Fix typo
* gn3/auth/authentication/oauth2/server.py: Register the
  `AuthorisationCodeGrant` grant with the server.
* gn3/auth/authentication/oauth2/views.py: Implement `/authorise` endpoint
* gn3/templates/base.html: New HTML Templates of authorisation UI
* gn3/templates/common-macros.html: New HTML Templates of authorisation UI
* gn3/templates/oauth2/authorise-user.html: New HTML Templates of
  authorisation UI
* main.py: Allow both "code" and "token" response types.
Check only that the email format is correct, but don't bother with the
deliverability check during authentication. The deliverability check is done
at registration.
Add an endpoint to list a user's collections. This only works for logged in
users.
The code, as written previously had a subtle bug - if the user created a new
collection before they had tried accessing their list of collections, the
older code would not have migrated the older collections.

This commit fixes that by enabling the migration of older collections, whether
or not the user has created a collection with their new accounts.
Move the code to a new package.

Enable the creation of new collection by both authenticated and anonymous
users.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants