Skip to content

Conversation

@shapiromatron
Copy link
Owner

@shapiromatron shapiromatron commented Jun 25, 2024

I reviewed around 3000 visuals and found very minor differences in <1% of them. This PR addresses fixing the issues in that 1%, along with a few other bugs I found when investigating these differences. Here's a detailed list of changes I made:

  • The data pivot html view previously didn't appropriately handle a 500 error on a dataset and treated it like a valid response; this is now fixed and it throws an error message correctly
  • The animal treatment_period transformation method didn't appropriately transform text in a few cases; this was rewritten
  • In some cases, if a rare case where the study evaluation metrics have duplicate names, an export would fail because you can not assign multiple columns (df.loc[list_of_strings]) if columns are not unique; a get unique name method was added, and the study evaluation fetching code was updated to be reused across exports
  • If data is correct and upper and lower CI are not provided, epi can calculate upper and lower ci values. A new method was added to do so
  • Fixed invitro category addition to a datafarme where an endpoint has None as a category ID
  • Remove iv_endpoint-category_id from invitro data pivot exports

Unrelated change:

  • Fix deprecation warning when running pytest (in conftest.py):
    • Marks applied to fixtures have no effect
  • Add UDF API to the openapi yml specification

@shapiromatron shapiromatron marked this pull request as draft June 26, 2024 21:28
@shapiromatron shapiromatron requested a review from rabstejnek June 29, 2024 14:37
@shapiromatron shapiromatron marked this pull request as ready for review June 29, 2024 14:38
Copy link
Collaborator

@rabstejnek rabstejnek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good! I had just one comment about a TODO left in the code, after that is addressed I think its good to merge.

@shapiromatron shapiromatron merged commit 4a078d0 into exports-v2 Jul 2, 2024
@shapiromatron shapiromatron deleted the updates-from-image-review branch July 2, 2024 20:01
rabstejnek added a commit that referenced this pull request Jul 2, 2024
* Epi export rewrite (#911)

* preliminary epi rewrite

* changes

* changes

* changes

* fix

* added back something accidentally deleted

* moved code

* fix test

* remove old stuff

* cleanup:

* use vectorized timestamp conversion

* minor formatting

---------

Co-authored-by: Andy Shapiro <[email protected]>

* rewrite riskofbias exports (#921)

* preliminary epi rewrite

* changes

* changes

* changes

* fix

* added back something accidentally deleted

* moved code

* fix test

* remove old stuff

* cleanup:

* rewrite riskofbias exports

* update naming for domain and metric

---------

Co-authored-by: Daniel Rabstejnek <[email protected]>

* Epimeta export rewrite (#922)

* preliminary epi rewrite

* changes

* changes

* changes

* fix

* added back something accidentally deleted

* moved code

* fix test

* remove old stuff

* cleanup:

* epimeta export rewrite

* remove obsolete code

* changes

* merge fix

* update admin site to browse data pivot by evidence type

---------

Co-authored-by: Andy Shapiro <[email protected]>

* Invitro export rewrite (#958)

* Convert invitro exports

* Fix exporter datetime converter when datetime is None

* Fix exports where there's no data

* DTXSID in export should be None instead of useless dict if missing

* add endpoint categories

* Remove bloated dsstox dict from exports in favor of dtxsid

---------

Co-authored-by: Andy Shapiro <[email protected]>

* Animal export rewrite (#961)

* Made animal model exports, began configuring exporters

* first export largely done, second export mostly done

* Some cleanups, finished fourth animal exporter

* Changes after more testing

* Fix tests

* Remove remaining flat_complete_* methods

* Cleanup

* Update sql_display to accept dict

* Add TODO to comment to easily find it

* Rename exporters to match the flat file exporter class names

* Add safeguard for qs accessing and fix groupby side effects (#982)

* two updates

* move to correct place

* exports-v2 - Updates from image review (#1058)

* remove duplicate rename calls

* add ci calculation

* update treatment period calculation

* add back space; fix caused too many false positives in comparison

* properly handle a 500 response from the server in a data pivot

* fix epi logic from plot review

* fix invitro export when category id is null; remove category id from export

* add unique rob columns

* use same method for categories

* refactor into a reusable method

* remove pytest warning "Marks applied to fixtures have no effect"

* refactor; write tests

* add tests

* rewrite animal export test using same pattern

* add udf to openapi

* remove extra whitespace (will change visuals; known change)

---------

Co-authored-by: Daniel Rabstejnek <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants