Add text embedder #996

abhijithneilabraham · 2021-11-24T15:38:54Z

Fixes #760

for more information, see https://pre-commit.ci

codecov · 2021-11-24T15:47:18Z

Codecov Report

Merging #996 (f69f207) into master (1ddd556) will increase coverage by 0.02%.
The diff coverage is 93.44%.

@@            Coverage Diff             @@
##           master     #996      +/-   ##
==========================================
+ Coverage   86.71%   86.74%   +0.02%     
==========================================
  Files         266      269       +3     
  Lines       13809    13870      +61     
==========================================
+ Hits        11974    12031      +57     
- Misses       1835     1839       +4

Flag	Coverage Δ
unittests	`86.74% <93.44%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
flash/text/embedding/model.py	`92.00% <92.00%> (ø)`
flash/core/utilities/imports.py	`91.36% <100.00%> (+0.06%)`	⬆️
flash/core/utilities/providers.py	`100.00% <100.00%> (ø)`
flash/text/__init__.py	`100.00% <100.00%> (ø)`
flash/text/embedding/__init__.py	`100.00% <100.00%> (ø)`
flash/text/embedding/backbones.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ddd556...f69f207. Read the comment docs.

for more information, see https://pre-commit.ci

…m/lightning-flash into ST_embeddings

for more information, see https://pre-commit.ci

tchaton

Great work ! Almost there !

flash/text/embedding/model.py

tchaton · 2021-12-08T17:33:14Z

flash_examples/text_embedder.py

+)
+
+# 2. Load a previously trained SentenceEmbedder
+model = SentenceEmbedder(backbone="sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2")


Would you mind adding some tests to increase the coverage ?

Yes. On it now.

@tchaton I just commited a test but I am not sure if it is the right way you want it to be. Please do a review and let me know how I can improve.

for more information, see https://pre-commit.ci

…m/lightning-flash into ST_embeddings

ethanwharris

Awesome, looking good 😃 A few comments suggestions. Also, could you update CHANGELOG.md?

flash/text/embedding/model.py

requirements.txt

tests/text/embedding/test_model.py

flash_examples/text_embedder.py

for more information, see https://pre-commit.ci

flash_examples/text_embedder.py

for more information, see https://pre-commit.ci

…m/lightning-flash into ST_embeddings

for more information, see https://pre-commit.ci

…m/lightning-flash into ST_embeddings

for more information, see https://pre-commit.ci

ethanwharris

Awesome, LGTM 😃

abhijithneilabraham · 2021-12-09T15:31:07Z

@tchaton

tchaton

LGTM ! Awesome work !

abhijithneilabraham added 2 commits November 24, 2021 21:06

Sentence Embedder API using sentence transformers

98ed529

remove train, test and pred step

83fbf1e

abhijithneilabraham requested review from ananyahjha93, Borda, carmocca, ethanwharris, justusschock, kaushikb11 and tchaton as code owners November 24, 2021 15:38

[pre-commit.ci] auto fixes from pre-commit.com hooks

4ad3abb

for more information, see https://pre-commit.ci

abhijithneilabraham and others added 8 commits December 5, 2021 17:06

sentence embedders with forward step and predict step

8cd9395

Merge branch 'master' into ST_embeddings

47bbce7

[pre-commit.ci] auto fixes from pre-commit.com hooks

34f39d1

for more information, see https://pre-commit.ci

Update __init__.py

219042a

Merge branch 'ST_embeddings' of https://github.com/abhijithneilabraha…

9f9a965

…m/lightning-flash into ST_embeddings

[pre-commit.ci] auto fixes from pre-commit.com hooks

4b3c772

for more information, see https://pre-commit.ci

Merge branch 'master' into ST_embeddings

5fecb22

Updates

06e35bd

ethanwharris changed the title ~~St embeddings~~ Sentence embedder Dec 8, 2021

tchaton reviewed Dec 8, 2021

View reviewed changes

abhijithneilabraham and others added 8 commits December 9, 2021 00:32

Create test_model.py

2071c9f

[pre-commit.ci] auto fixes from pre-commit.com hooks

5477415

for more information, see https://pre-commit.ci

__init__ for embedding

8db110d

Merge branch 'master' into ST_embeddings

2c09c71

remove download_data()

a6bfc9f

Merge branch 'ST_embeddings' of https://github.com/abhijithneilabraha…

602d9e1

…m/lightning-flash into ST_embeddings

Merge branch 'master' into ST_embeddings

b29b6e0

Merge branch 'master' into ST_embeddings

e771883

ethanwharris reviewed Dec 9, 2021

View reviewed changes

flash/text/embedding/model.py Outdated Show resolved Hide resolved

requirements.txt Outdated Show resolved Hide resolved

tests/text/embedding/test_model.py Outdated Show resolved Hide resolved

lower size model for text embededer examples and test

21305d6

abhijithneilabraham commented Dec 9, 2021

View reviewed changes

flash_examples/text_embedder.py Outdated Show resolved Hide resolved

[pre-commit.ci] auto fixes from pre-commit.com hooks

5d1b4c6

for more information, see https://pre-commit.ci

ethanwharris changed the title ~~Sentence embedder~~ Add text embedder Dec 9, 2021

abhijithneilabraham added 2 commits December 9, 2021 19:28

text embedder example entry to CI

9570522

Merge branch 'master' into ST_embeddings

8cfd877

ethanwharris reviewed Dec 9, 2021

View reviewed changes

flash_examples/text_embedder.py Outdated Show resolved Hide resolved

abhijithneilabraham and others added 10 commits December 9, 2021 19:35

change SentenceEmbedder to TextEmbedder

bb98d77

remove download_data import

923e6ec

[pre-commit.ci] auto fixes from pre-commit.com hooks

8c90286

for more information, see https://pre-commit.ci

fix bug - test_model.py

20233f2

Merge branch 'ST_embeddings' of https://github.com/abhijithneilabraha…

33f30e6

…m/lightning-flash into ST_embeddings

[pre-commit.ci] auto fixes from pre-commit.com hooks

57aa577

for more information, see https://pre-commit.ci

Update test_model.py

fdbb2de

Merge branch 'ST_embeddings' of https://github.com/abhijithneilabraha…

6f5b9e8

…m/lightning-flash into ST_embeddings

[pre-commit.ci] auto fixes from pre-commit.com hooks

14a5e27

for more information, see https://pre-commit.ci

Update CHANGELOG.md

3d14659

ethanwharris approved these changes Dec 9, 2021

View reviewed changes

tchaton approved these changes Dec 9, 2021

View reviewed changes

Merge branch 'master' into ST_embeddings

f69f207

ethanwharris merged commit eb09e26 into Lightning-Universe:master Dec 9, 2021

charleswm mentioned this pull request Feb 10, 2022

Text Embedder Docs are missing/incomplete #1159

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add text embedder #996

Add text embedder #996

abhijithneilabraham commented Nov 24, 2021 •

edited by ethanwharris

Loading

codecov bot commented Nov 24, 2021 •

edited

Loading

tchaton left a comment

tchaton Dec 8, 2021

abhijithneilabraham Dec 8, 2021

abhijithneilabraham Dec 8, 2021

ethanwharris left a comment

ethanwharris left a comment

abhijithneilabraham commented Dec 9, 2021

tchaton left a comment

Add text embedder #996

Add text embedder #996

Conversation

abhijithneilabraham commented Nov 24, 2021 • edited by ethanwharris Loading

codecov bot commented Nov 24, 2021 • edited Loading

Codecov Report

tchaton left a comment

Choose a reason for hiding this comment

tchaton Dec 8, 2021

Choose a reason for hiding this comment

abhijithneilabraham Dec 8, 2021

Choose a reason for hiding this comment

abhijithneilabraham Dec 8, 2021

Choose a reason for hiding this comment

ethanwharris left a comment

Choose a reason for hiding this comment

ethanwharris left a comment

Choose a reason for hiding this comment

abhijithneilabraham commented Dec 9, 2021

tchaton left a comment

Choose a reason for hiding this comment

abhijithneilabraham commented Nov 24, 2021 •

edited by ethanwharris

Loading

codecov bot commented Nov 24, 2021 •

edited

Loading