Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CU-8694fk90t (almost) only primitive config #425

Merged
merged 16 commits into from
May 22, 2024
Merged

CU-8694fk90t (almost) only primitive config #425

merged 16 commits into from
May 22, 2024

Conversation

mart-r
Copy link
Collaborator

@mart-r mart-r commented May 2, 2024

This PR removes weighted_average_function from the config and ties it to the CDB instead.

The idea is to be able to more simply save and load configs in a simpler json format.

So the PR also makes new saves of the config saved in (mostly) regular json.
There are a few caveats:

  • Custom serialisation is used for sets and re.Patterns
  • And custom decoding as well
  • So that means that a simple json.load will not create a useful dict to merge into a config

Some further things to note:

  • The PR still allows loading old-style jsonpickled configs
  • Some things may have been moved around a little for simplicity
  • This PR may complicate some things
    • There's a backwards compatibility method that already fixes weighted_average_function issues
      • I.e makes sure they're not lambdas
    • There's already python 3.11-specific fixers to dill'ed weighted_average_functions
      • I.e when the method was loaded but couldn't be used
    • If someone sets a different value for cdb.weighted_average_function multiprocessing may be problematic
      • I.e if a lambda is used this won't be pickleable and multiprocessing methods will fail

Just to be clear, the previous json files were also human-readable. But they weren't as easy to follow as the (mostly) pure json that'd be used after this PR.

@tomolopolis
Copy link
Member

Task linked: CU-8694fk90t Remove non-primitive config entries

@mart-r mart-r changed the title CU-8694fk90t only primitive config CU-8694fk90t (almost) only primitive config May 7, 2024
tomolopolis
tomolopolis approved these changes May 14, 2024
View reviewed changes
Copy link
Member

@tomolopolis tomolopolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - in the unlikely event that someone has client code that calls:
config.linking['weighted_average_function'] can we raise a nicer exception here which currently will just raise an dict KeyError

@mart-r mart-r merged commit 2872d5e into master May 22, 2024
5 checks passed
@mart-r mart-r mentioned this pull request May 31, 2024
Merged
mart-r added a commit that referenced this pull request Jun 19, 2024
@mart-r @shubham-s-agarwal
@vladd-bit @tomolopolis
v1.12.0 release PR (#455)
ffd22a6 
* Pushing changes for bert-style models for MetaCAT

* Pushing fix for LSTM

* Pushing changes for flake8 and type fixes

* Pushing type fixes

* Fixing type issue

* Pushing changes

1) Added model.zero_grad to clear accumulated gradients
2) Fixed config save issue
3) Re-structured data preparation for oversampled data

* Pushing change and type fixes

Pushing ml_utils file which was missed in the last commit

* Fixing flake8 issues

* Pushing flake8 fixes

* Pushing fixes for flake8

* Pushing flake8 fix

* Adding peft to list of libraries

* Pushing changes with load and train workflow and type fixes

The workflow for inference is: load() and inference
For training: init() and train()
Train will always not load the model dict, except when the phase_number is set to 2 for 2 phase learning's second phase

* Pushing changes with type hints and new documentation

* Pushing type fix

* Fixing type issue

* Adding test case for BERT and reverting config changes

BERT test cases: Testing for BERT model along with 2 phase learning

* Merging changes from master to metacat_bert branch (#431)

* Small addition to contribution guidelines (#420)

* CU-8694cbcpu: Allow specifying an AU Snomed when preprocessing (#421)

* CU-8694dpy1c: Return empty generator upon empty stream (#423)

* CU-8694dpy1c: Return empty generator upon empty stream

* CU-8694dpy1c: Fix empty generator returns

* CU-8694dpy1c: Simplify empty generator returns

* Relation extraction (#173)

* Added files.

* More additions to rel extraction.

* Rel base.

* Update.

* Updates.

* Dependency parsing.

* Updates.

* Added pre-training steps.

* Added training & model utils.

* Cleanup & fixes.

* Update.

* Evaluation updates for pretraining.

* Removed duplicate relation storage.

* Moved RE model file location.

* Structure revisions.

* Added custom config for RE.

* Implemented custom dataset loader for RE.

* More changes.

* Small fix.

* Latest additions to RelCAT (pipe + predictions)

* Setup.py fix.

* RE utils update.

* rel model update.

* rel dataset + tokenizer improvements.

* RelCAT updates.

* RelCAT saving/loading improvements.

* RelCAT saving/loading improvements.

* RelCAT model fixes.

* Attempted gpu learning fix. Dataset label generation fixes.

* Minor train dataset gen fix.

* Minor train dataset gen fix No.2.

* Config updates.

* Gpu support fixes. Added label stats.

* Evaluation stat fixes.

* Cleaned stat output mode during training.

* Build fix.

* removed unused dependencies and fixed code formatting

* Mypy compliance.

* Fixed linting.

* More Gpu mode train fixes.

* Fixed model saving/loading issues when using other baes models.

* More fixes to stat evaluation. Added proper CAT integration of RelCAT.

* Setup.py typo fix.

* RelCAT loading fix.

* RelCAT Config changes.

* Type fix. Minor additions to RelCAT model.

* Type fixes.

* Type corrections.

* RelCAT update.

* Type fixes.

* Fixed type issue.

* RelCATConfig: added seed param.

* Adaptations to the new codebase + type fixes..

* Doc/type fixes.

* Fixed input size issue for model.

* Fixed issue(s) with model size and config.

* RelCAT: updated configs to new style.

* RelCAT: removed old refs to logging.

* Fixed GPU training + added extra stat print for train set.

* Type fixes.

* Updated dev requirements.

* Linting.

* Fixed pin_memory issue when training on CPU.

* Updated RelCAT dataset get + default config.

* Updated RelDS generator + default config

* Linting.

* Updated RelDatset + config.

* Pushing updates to model

Made changes to:
1) Extracting given number of context tokens left and right of the entities
2) Extracting hidden state from bert for all the tokens of the entities and performing max pooling on them

* Fixing formatting

* Update rel_dataset.py

* Update rel_dataset.py

* Update rel_dataset.py

* RelCAT: added test resource files.

* RelCAT: Fixed model load/checkpointing.

* RelCAT: updated to pipe spacy doc call.

* RelCAT: added tests.

* Fixed lint/type issues & added rel tag to test DS.

* Fixed ann id to token issue.

* RelCAT: updated test dataset + tests.

* RelCAT: updates to requested changes + dataset improvements.

* RelCAT: updated docs/logs according to commends.

* RelCAT: type fix.

* RelCAT: mct export dataset updates.

* RelCAT: test updates + requested changes p2.

* RelCAT: log for MCT export train.

* Updated docs + split train_test & dataset for benchmarks.

* type fixes.

---------

Co-authored-by: Shubham Agarwal <[email protected]>
Co-authored-by: mart-r <[email protected]>

* CU-8694fae3r: Avoid publishing PyPI release when doing GH pre-releases (#424)

* CU-8694fae3r: Avoid publishing PyPI release when doing GH pre-releases

* CU-8694fae3r: Fix pre-releases tagging

* CU-8694fae3r: Allow actions to run on release edit

---------

Co-authored-by: Mart Ratas <[email protected]>
Co-authored-by: Vlad Dinu <[email protected]>

* Pushing changed tests and removing empty change

* Pushing change for logging

* Revert "Pushing change for logging"

This reverts commit fbcdb70.

* CU-8694hukwm: Document the materialising of generator when multiproce… (#433)

* CU-8694hukwm: Document the materialising of generator when multiprocessing and batching for docs

* CU-8694hukwm: Add TODO note for where the generator is materialised

* CU-8694hukwm: Add warning from large amounts of generator data (10k items) is materialised by the docs size mp method

* CU-8694fk90t (almost) only primitive config (#425)

* CU-8694fk90r: Move backwards compatibility method from CDB to config utils

* CU-8694fk90r: Move weighted_average_function from config to CDB; create necessary backwards compatibility workarounds

* CU-8694fk90r: Move usage of weighted_average_function in tests

* CU-8694fk90r: Add JSON encode and decoder for re.Pattern

* CU-8694fk90r: Rebuild custom decoder if needed

* CU-8694fk90r: Add method to detect old style config

* CU-8694fk90r: Use regular json serialisation for config; Retain option to read old jsonpickled config

* CU-8694fk90r: Add test for config serialisation

* CU-8694fk90r: Make sure to fix weighted_average_function upon setting it

* CU-8694fk90t: Add missing tests for config utils

* CU-8694fk90t: Add tests for better raised exception upon old way of using weighted_average_function

* CU-8694fk90t: Fix exception type in an added test

* CU-8694fk90t: Add further tests for exception payload

* CU-8694fk90t: Add improved exceptions when using old/unsupported value of weighted_average_function in config

* CU-8694fk90t: Add typing fix exceptions

* CU-8694fk90t: Make custom exception derive from AttributeError to correctly handle hasattr calls

* CU-8694gza88: Create codeql.yml (#434)

Run CodeQL to identify vulnerabilities.
This will run on any push or pull request to `master`, but also runs once every day in case some new vulnerabilities are discovered (or something else changes).

* CU-8694mbn03: Remove the web app (#441)

* CU-8694n48uw better deprecation (#443)

* CU-8694n493m: Add deprecation and removal versions to deprecation decorator

* CU-8694n493m: Deprecation version to existing deprecated methods.

Made the removal version 2 minor versions from the minor version
in which the method was deprecated, or the next minor version if
the method had been deprecated for longer.

* CU-8694n4ff0: Raise exception upon deprecated method call at test time

* CU-8694n4ff0: Fix usage of deprecated methods call during test time

* CU-8694pey4u: extract cdb load to cls method, to be used in trainer for model pack loading

* CU-8694pey4u: extract meta cat loading also to a cls method

* CU-8694pey4u: docstrings

* CU-8694pey4u: typehints and mypy issues

* CU-8694pey4u: fix flake8

* CU-8694pey4u: fix flake8

* CU-8694pey4u: missing extra config if passed in

* CU-8694py1jr: Fix issue with reuse of opened file when loading old configs

* CU-8694py1jr: Make old config identifier more robust

* CU-8694py1jr: Add doc string to old config identifier

* CU-8694py1jr: Add test for old style MetaCAT config load

* CU-8694py1jr: Add test for old style main config load (functional)

* CU-8694py1jr: Refactor config utils load tests for more flexibility

* CU-8694py1jr: Add config utils load tests for NER and Rel CAT configs

* CU-8694vcvz7: Trust remote code when loading transfomers NER dataset (#453)

* CU-8694vcvz7: Trust remote code when loading transfomers NER dataset

* CU-8694vcvz7: Add support for older datasets without the remote code trusing kwarg

* CU-8694gzbn3 k fold metrics (#432)

* CU-8694gzbud: Add context manager that is able to snapshot CDB state

* CU-8694gzbud: Add tests to snapshotting CDB state

* CU-8694gzbud: Refactor tests for CDB state snapshotting

* CU-8694gzbud: Remove use of deprecated method in CDB utils and use non-deprecated one instead

* CU-8694gzbud: Add tests for training and CDB state capturing

* CU-8694gzbud: Small refactor in tests

* CU-8694gzbud: Add option to save state on disk

* CU-8694gzbud: Add debug logging output when saving state on disk

* CU-8694gzbud: Remove unused import

* CU-8694gzbud: Add tests for disk-based state save

* CU-8694gzbud: Move CDB state code to its own module

* CU-8694gzbud: Remove unused import

* CU-8694gzbud: Add doc strings to methods

* CU-8694gzbx4: Small optimisation for stats

* CU-8694gzbx4: Add MCTExport related module

* CU-8694gzbx4: Add MCTExport related tests

* CU-8694gzbx4: Add code for k-fold statistics

* CU-8694gzbx4: Add tests for k-fold statistics

* CU-8694gzbx4: Add test-MCT export with fake concepts

* CU-8694gzbx4: Fix a doc string

* CU-8694gzbx4: Fix types in MCT export module

* CU-8694gzbx4: Fix types in k-fold module

* CU-8694gzbx4: Remove accidentally committed test class

* CU-8694gzbn3: Add missing test helper file

* CU-8694gzbn3: Remove whitespace change from otherwise uncahnged file

* CU-8694gzbn3: Allow 5 minutes longer for tests

* CU-8694gzbn3: Move to python 3.8-compatible typed dict

* CU-8694gzbn3: Add more time for tests in worklow (now 30 minutes)

* CU-8694gzbn3: Add more time for tests in worklow (now 45 minutes)

* CU-8694gzbn3: Update test-pypi timeout to 45 minutes

* CU-8694gzbn3: Remove timeout from unit tests in main workflow

* CU-8694gzbn3: Make tests stop upon first failure

* CU-8694gzbn3: Fix test stop upon first failure (arg/option order)

* CU-8694gzbn3: Remove debug code and old comments

* CU-8694gzbn3: Remove all timeouts from main workflow

* CU-8694gzbn3: Remove more old / useless comments in tests

* CU-8694gzbn3: Add debug output when running k-fold tests to see where it may be stalling

* CU-8694gzbn3: Add debug output when ANY tests to see where it may be stalling

* CU-8694gzbn3: Remove explicit debug output from k-fold test cases

* CU-8694gzbn3: Remove timeouts from DEID tests in case they're the ones creating issues

* GHA/test fixes (#437)

* Revert "CU-8694gzbn3: Remove timeouts from DEID tests in case they're the ones creating issues"

This reverts commit faaf7fb.

* Revert "CU-8694gzbn3: Remove explicit debug output from k-fold test cases"

This reverts commit 9b02925.

* Revert "CU-8694gzbn3: Add debug output when ANY tests to see where it may be stalling"

This reverts commit 12c519a.

* Revert "CU-8694gzbn3: Add debug output when running k-fold tests to see where it may be stalling"

This reverts commit 03531da.

* Revert "CU-8694gzbn3: Remove all timeouts from main workflow"

This reverts commit e6debce.

* Revert "CU-8694gzbn3: Fix test stop upon first failure (arg/option order)"

This reverts commit 666c013.

* Revert "CU-8694gzbn3: Make tests stop upon first failure"

This reverts commit 94bce56.

* Revert "CU-8694gzbn3: Remove timeout from unit tests in main workflow"

This reverts commit 3618b9c.

* CU-8694gzbn3: Improve state copy code in CDB state tests

* CU-8694gzbn3: Fix a CDB state test issue

* CU-8694gzbn3: Split all tests into 2 halves

* CU-8694gzbn3: Remove legacy / archived / unused tests

* CU-8694gzbn3: Add doc strings for FoldCreator init

* CU-8694gzbn3: Move to a split-type enum

* CU-8694gzbn3: Add documentation to split-type enum

* CU-8694gzbn3: Create separate fold creators for different types of splitting strategies

* CU-8694gzbn3: Resort document order in test time nullification process

* CU-8694gzbn3: Add option to count number of annotations in doc for MCT export

* CU-8694gzbn3: Add weighted documents based split option along with relevant tests

* CU-8694gzbn3: Update default fold creation split type to weighted documents

* CU-8694gzbn3: Add test to ensure weighted documents split creates a reasonable number of annotations per split

* CU-8693n892x environment/dependency snapshots (#438)

* CU-8693n892x: Save environment/dependency snapshot upon model pack creation

* CU-8693n892x: Fix typing for env snapshot module

* CU-8693n892x: Add test for env file existance in .zip

* CU-8693n892x: Add doc strings

* CU-8693n892x: Centralise env snapshot file name

* CU-8693n892x: Add env snapshot file to exceptions in serialisation tests

* CU-8693n892x: Only list direct dependencies

* CU-8693n892x: Add test that verifies all direct dependencies are listed in environment

* CU-8693n892x: Move requirements to separate file and use that for environment snapshot

* CU-8693n892x: Remove unused constants

* CU-8693n892x: Allow URL based dependencies when using direct dependencies

* CU-8693n892x: Distribute install_requires.txt alongside the package; use correct path in distributed version

* CU-8694p8y0k deprecation GHA check (#445)

* CU-8694p8y0k: Add check for deprecations (code)

* CU-8694p8y0k: Add workflow check for deprecations

* CU-8694p8y0k: Fix (hopefully) workflow check for deprecations

* CU-8694p8y0k: Add option to remove version prefix when checking deprecation

* CU-8694p8y0k: Update deprecation checks with more detail (i.e current/next version).

* CU-8694p8y0k: Only run deprecation checking step when merging master into production

* CU-8694u3yd2 cleanup name removal (#450)

* CU-8694u3yd2: Add logged warning for when using full-unlink

* CU-8694u3yd2: Make CDB.remove_names simply expect an iterable of names

* CU-8694u3yd2: Improve CDB.remove_names doc string

* CU-8694u3yd2: Explicitly pass the keys to CDB.remove_names in CAT.unlink_concept_name

* CU-8694u3yd2: Add note regarding state (and order) dependent tests to some CDB maker tests

* CU-8694u3yd2: Rename/make protected CDB.remove_names method

* CU-8694u3yd2: Create deprecated CDB.remove_names method

* CU-8694vte2g 1.12 depr removal (#454)

* CU-8694vte2g: Remove CDB.add_concept method

* CU-8694vte2g: Remove unused import (deprecated decorator)

* CU-8694vte2g: Remove CAT.get_spacy_nlp method

* CU-8694vte2g: Remove CAT.train_supervised method

* CU-8694vte2g: Remove CAT multiprocessing methods

* CU-8694vte2g: Remove MetaCAT.train method

* CU-8694vte2g: Remove medcat.utils.ner.helper.deid_text method

* CU-8694vte2g: Remove use of deprecated method

* CU-8694vte2g: Add back removed deprecation import

---------

Co-authored-by: Shubham Agarwal <[email protected]>
Co-authored-by: Vlad Dinu <[email protected]>
Co-authored-by: Tom Searle <[email protected]>
@mart-r mart-r mentioned this pull request Jul 24, 2024
Merged
@mart-r mart-r deleted the CU-8694fk90t-only-primitive-config branch August 12, 2024 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tomolopolis tomolopolis tomolopolis approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mart-r @tomolopolis