v0.13.4

v0.13.4 (2024-04-08)

Documentation

• docs: minor updates to tables (9f7da4b)

Fix

• fix: updated ruff dependency (7e7a7ae)

• fix: Fixed broken HF links to scandeval (b99e300)

Unknown

• Merge pull request #174 from KennethEnevoldsen/minor-fixes

fix: Fixing links for prs (5dc4ca3)

v0.13.3

v0.13.3 (2024-02-23)

Documentation

• docs: Updated tables to include task subtypes (e119c58)

Fix

• fix: Added task subtypes to tasks

This follows the denotion in the paper. A task can have multiple task subtypes but only one task type. (7fc9ed5)

Unknown

• Merge pull request #162 from KennethEnevoldsen/add-task-subtypes

Added task subtypes (363ab09)

v0.13.2

v0.13.2 (2024-02-19)

Fix

• fix: Pass the task for encode_queries, and encode_corpus

This yield notable performance improvements for the instruct models for retrieval tasks (9992e80)

Unknown

• Merge pull request #156 from KennethEnevoldsen/fix_instruct_tuned_embed

fix: Pass the task for encode_queries, and encode_corpus (13786fe)

• Merge branch 'main' of https://github.com/KennethEnevoldsen/scandinavian-embedding-benchmark into fix_instruct_tuned_embed (69b2ae2)

v0.13.1

v0.13.1 (2024-02-19)

Chore

• chore: remove test file (14f9935)

Documentation

• docs: Updte docs script to handle new name format (aa171dd)

Fix

• fix: fix incorrect emb. size for e5 large instruct (7865ad7)

• fix: Added final for mult. e5 instruct, including speed test of ref. system (08e1779)

• fix: added multilingual-e5-large-instruct (56bfc16)

• fix: rename model_architecture to architecture to not take up protected attribute for pydantic (f845a49)

Unknown

• Merge pull request #155 from KennethEnevoldsen/add-multilingual-instruct

Add multilingual e5 instruct (c2cca49)

v0.13.0

v0.13.0 (2024-02-19)

Feature

• feat: Ensure that all model names are consistent

i.e. that they have the same name as they would have on the benchmark (c2299cd)

Fix

• fix: made the to method optional on the encoder (157a91c)

• fix: Add to method to lazyloadencoder (0b6d0be)

• fix: Ensure return type is always np.ndarray (e8d3994)

• fix: Ensure return type is always np.ndarray (06c5cd8)

Unknown

• Merge pull request #153 from KennethEnevoldsen/ensure-consistent-names

Ensure consistent names (83fd962)

• Merge branch 'ensure_return_type' of https://github.com/KennethEnevoldsen/scandinavian-embedding-benchmark into ensure_return_type (7d09487)

v0.12.2

v0.12.2 (2024-02-17)

Ci

• ci: Added not planned as valid no stale label (a2dd834)

Fix

• fix: Removed translate-embed integration test (adb9cd6)

• fix: removing smaller translate then embed models (fbb9e97)

• fix: removing smaller translate then embed models (91f6b79)

• fix: Add missing scores (3b92090)

• fix: Added e5 mistral scores (7515e79)

Style

• style: ran linting (f729288)

Unknown

• Merge pull request #143 from KennethEnevoldsen/run-e5

Updated e5-mistral model (0026c9c)

• Merge branch 'run-e5' of https://github.com/KennethEnevoldsen/scandinavian-embedding-benchmark into run-e5 (fdc19fb)

• Merge branch 'run-e5' of https://github.com/KennethEnevoldsen/Scandinavian-Embedding-Benchmark into run-e5 (e691448)

v0.12.1

v0.12.1 (2024-02-15)

Documentation

• docs: Added dataset disclaimer (#145) (6b3e71b)

Fix

• fix: Added get_documents to task interface (c4fb354)

Unknown

• Merge branch 'main' of https://github.com/KennethEnevoldsen/Scandinavian-Embedding-Benchmark (24644a7)

• Fix: Added performance metrics for translate and embed (91a2b8a)

v0.12.0

v0.12.0 (2024-02-12)

Unknown

• Merge pull request #137 from KennethEnevoldsen/update-sonar-models

Update and rerun sonar models (28ff5e5)

v0.11.4

v0.11.4 (2024-02-12)

Fix

• fix: Added model type and releae date to model meta

This is to allow the tracking of improvement on SEB over time (49c9f1a)

Unknown

• Merge pull request #140 from KennethEnevoldsen/adding-architecture

fix: Added model type and releae date to model meta (245e881)

• Merge pull request #131 from KennethEnevoldsen/mistral-instructions

Added instructions for all tasks in Mistral E5 (006c253)

• Added instructions for all tasks in Mistral E5 (07c4c95)

v0.11.3

v0.11.3 (2024-02-06)

Documentation

• docs: Updated table w. dataset descriptions (f955379)

• docs: Added across column for coverage

added swapped formalization and task columns (31d9d76)

Fix

• fix: Updated dataset description metadata and script (0e63eb4)

• fix: Update calc. descriptive stats (bdbd552)

Unknown

• Merge pull request #130 from KennethEnevoldsen/docs-update

Update task/dataset descriptions (144f025)