Changes to sample data editing #117
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* migrations/auth/20221110_08_23psB-add-privilege-category-and-privilege-description-columns-to-privileges-table.py: new migration * tests/unit/auth/test_migrations_add_remove_columns.py: test new migration
* .gitignore: ignore all yoyo configuration files * README.md: Update documentation * yoyo.auth.ini: stop tracking the yoyo configuration file.
* migrations/auth/20221113_01_7M0hv-enumerate-initial-privileges.py: new migration. * tests/unit/auth/test_migrations_insert_data_into_empty_table.py: test new migration.
Add table `generic_role_privileges` table to link the generic roles to the privileges they provide. * migrations/auth/20221114_01_n8gsF-create-generic-role-privileges-table.py: new migration * tests/unit/auth/test_create_table_migrations.py: test new migration * tests/unit/auth/test_migrations_indexes.py: test new migration
* Name all tests that test migrations to start with `test_migrations_`
Drop the `generic_role*` tables, since what they were envisioned for can be handled a different, (arguably) more simple way. * migrations/auth/20221114_02_DKKjn-drop-generic-role-tables.py: new migration * tests/unit/auth/test_migrations_drop_tables.py: test new migration
* migrations/auth/20221110_05_BaNtL-create-roles-table.py: modify migration * migrations/auth/20221114_03_PtWjc-create-group-roles-table.py: new migration * tests/unit/auth/test_migrations_create_tables.py: test new migration * tests/unit/auth/test_migrations_indexes.py: test new migration
* migrations/auth/20221114_04_tLUzB-initialise-basic-roles.py: new migration * tests/unit/auth/test_migrations_insert_data_into_empty_table.py: test new migration
Some roles will not be user editable to prevent inconsistencies, and possible privilege escalation. * migrations/auth/20221110_05_BaNtL-create-roles-table.py: Add `user_editable` column to table * migrations/auth/20221114_04_tLUzB-initialise-basic-roles.py: Set `group-leader` role as not user editable
* gn3/auth/authorisation/__init__.py: Add `authorised_p` decorator to be used for all function requiring authorisation. * gn3/auth/authorisation/groups.py: Add `create_group` function stub * tests/unit/auth/conftest.py: Add fixture for test users * tests/unit/auth/test_groups.py: Add tests for `create_group`
* migrations/auth/20221114_05_hQun6-create-user-roles-table.py: new migration * tests/unit/auth/test_migrations_create_tables.py: test new migration * tests/unit/auth/test_migrations_indexes.py: test new migration
* The test app and the test client are both needed as fixtures in different places
* tests/unit/auth/test_groups.py: use Flask's application context directly rather than the request context to access `g` in order to get rid of the `DeprecationWarning`.
* tests/unit/auth/conftest.py: use the functions in `gn3.auth.db` to acquire the database connection and cursor since they handle some of the basic issues like rollback and commit, meaning we do not have to explicitly handle said issues in the fixtures.
* gn3/auth/authorisation/__init__.py: delete function * gn3/auth/authorisation/checks.py: move function to `checks` module
Use specified types for privileges, roles and types rather than using strings to help with limiting bugs. * gn3/auth/authorisation/groups.py: Specify and use the `Group` type * gn3/auth/authorisation/privileges.py: Specify and use the `Privilege` type * gn3/auth/authorisation/roles.py: Specify the `Role` type. Add the `create_role` function.
* gn3/auth/authorisation/checks.py: Return results of calling the function rather than a dict of values that include the results. * gn3/auth/authorisation/groups.py: Use the newer form of `authorised_p` decorator. * tests/unit/auth/test_groups.py: Update tests
* gn3/db/rdf.py: Delete gn3.setting.SPARQL_ENDPOINT import. (sparql_query): Inject SPARQLWrapper. (get_dataset_metadata): Ditto.
* tests/unit/auth/test_roles.py: new tests.
* gn3/auth/authorisation/privileges.py: Set id to UUID type * gn3/auth/authorisation/roles.py: fix parameters to types that sqlite3 supports * gn3/auth/db.py: add logging for errors and re-raise the exception * tests/unit/auth/test_roles.py: fix test
* migrations/auth/20221116_01_nKUmX-add-privileges-to-group-leader-role.py: new migration to fix data errors. * tests/unit/auth/test_privileges.py: test privileges
Decouple the `gn3.db_utils` module from the global `flask.current_app` object, ensuring that the database uri value is passed in as a required argument to the `gn3.db_utils.database_connection` function.
* gn3/api/metadata.py: Import Template, sparql_query and RDF_PREFIXES. (get_genewiki_entries): New endpoint. * gn3/db/rdf.py: Add new constant for storing rdf prefixes. Signed-off-by: Munyoki Kilyungi <[email protected]>
Signed-off-by: Munyoki Kilyungi <[email protected]>
Signed-off-by: Munyoki Kilyungi <[email protected]>
Consistently encode all values for the top-level keys stored in redis to avoid issues with json encode/decode
Fix bugs with setting up of the selected traits for use while filtering the search results.
During development, we need logging sometimes to help with troubleshooting problems. This commit provides a module to help set up the logging in a separate module from the app module.
…herwise after the recent changes to the DB
There's probably a better way to fix this query (it was previously returning each sample twice), but DISTINCT was the easiest way I could come up with
This is necessary in order to allow for editing the values of samples that don't currently have values
BonfaceKilz
approved these changes
May 12, 2023
There was a problem hiding this comment.
@zsloan thanks for pinging me to review this. I've left some minor comments that you could look at. Also, generally, you could use tests to try and poke holes at weird edge cases.
| the "official" sample list is stored | ||
| """ | ||
|
|
||
| genofile_path = os.environ.get("GENENETWORK_FILES") + "/genotype/" + group + ".geno" |
There was a problem hiding this comment.
nitpick: genofile_path is a constant, and as such, upper-case it.
Comment on lines
+17
to
+33
| genofile = open(genofile_path) | ||
|
|
||
| for line in genofile: | ||
| line = line.strip() | ||
| if not line: | ||
| continue | ||
| if line.startswith(("#", "@")): | ||
| continue | ||
| break | ||
|
|
||
| headers = line.split("\t") | ||
|
|
||
| if headers[3] == "Mb": | ||
| samplelist = headers[4:] | ||
| else: | ||
| samplelist = headers[3:] | ||
| return samplelist |
There was a problem hiding this comment.
issue: When opening a file, use a context-manager, in this case, with open .... This way, you don't have to worry about closing any hanging file connections should something unforeseen happens.
| "SELECT Name " | ||
| "FROM InbredSet " | ||
| "WHERE " | ||
| "InbredSet.Id = %(group_id)s") |
There was a problem hiding this comment.
question: is this escaped? I thought just using simple "%s" like so:
"SELECT * FROM .... WHERE ... = %s", value
should escape things.
Comment on lines
+52
to
+54
| if res: | ||
| return res[0] | ||
| return None |
There was a problem hiding this comment.
nitpick: @arunisaac write some monadic constructs to deal with None values when doing SQL queries. You could look at those. If you get stuck, we could chat over on Matrix and I could show you how to use them.
BonfaceKilz
approved these changes
May 12, 2023
There was a problem hiding this comment.
@zsloan thanks for pinging me to review this. I've left some minor comments that you could look at. Also, generally, you could use tests to try and poke holes at weird edge cases.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR will make a couple changes to sample data editing:
The accompanying GN2 PR is here - genenetwork/genenetwork2#781