Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
1e50557
wip: start adding BLIP models
Jamie-Stirling Sep 6, 2024
8f8e05c
add other blip variants
Jamie-Stirling Sep 9, 2024
3ea4d3e
Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i…
Jamie-Stirling Sep 9, 2024
be8b4bb
wip: add blip2_models.py
Jamie-Stirling Sep 11, 2024
b57a395
make lint
Jamie-Stirling Sep 11, 2024
236a94f
wip: implement blip2 wrapper
Jamie-Stirling Sep 13, 2024
1f2f8c3
feat: add blip2 models, still mismatched names
Jamie-Stirling Sep 15, 2024
8c64860
fix: remove projections from image and text embeddings
Jamie-Stirling Sep 15, 2024
20839ca
make lint
Jamie-Stirling Sep 15, 2024
ec47c69
wip: add coco BLIP2
Jamie-Stirling Sep 15, 2024
e8f4ae1
fix: BLIP2 better zero-shot classification without text_proj and visi…
Jamie-Stirling Sep 16, 2024
57bc3b8
tidy blip2
Jamie-Stirling Sep 19, 2024
4cbec1b
add imagenet-dog-15 dataset
Jamie-Stirling Sep 19, 2024
6da26dc
Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i…
Jamie-Stirling Sep 19, 2024
35be38d
tidy and lint
Jamie-Stirling Sep 19, 2024
83d0f45
remove unused import
Jamie-Stirling Sep 19, 2024
a309de5
add cluster_accuracy, ari and nmi to Image.ClusteringEvaluator
Jamie-Stirling Sep 19, 2024
02c1f81
add imagenet-10 clustering task
Jamie-Stirling Sep 20, 2024
cede108
add SOPI2IRetrieval
Jamie-Stirling Sep 20, 2024
d226748
add results forclip on ImageNet10Clustering and ImageNetDog15Clustering
Jamie-Stirling Sep 20, 2024
2086904
Merge branch 'mieb-imagenet' of https://github.com/Jamie-Stirling/mte…
Jamie-Stirling Sep 20, 2024
c7c6893
add SOPI2IRetrieval results for clip 32
Jamie-Stirling Sep 20, 2024
5bf66fd
add results for clip vit 32/SOPI2IRetrieval
Jamie-Stirling Sep 23, 2024
3866647
merge incoming change from mieb
Jamie-Stirling Sep 23, 2024
b1759dc
resolve conflict
Jamie-Stirling Sep 23, 2024
a890171
add RP2kI2IRetrieval dataset
Jamie-Stirling Sep 23, 2024
d3396fa
add RP2kI2IRetrieval results with clip-vit-base-patch32
Jamie-Stirling Sep 23, 2024
aa58556
update image retrieval __init__.py
Jamie-Stirling Sep 23, 2024
fc4d1ee
add RP2kI2IRetrieval and METI2IRetrieval
Jamie-Stirling Sep 24, 2024
7d4f332
add METI2IRetreival
Jamie-Stirling Sep 24, 2024
57593eb
add SOP results
Jamie-Stirling Sep 24, 2024
62e6e9f
make lign
Jamie-Stirling Sep 24, 2024
5d84ace
merge upstream mieb
Jamie-Stirling Sep 24, 2024
094a654
new revision for METI2IRetrieval
Jamie-Stirling Sep 25, 2024
2cc7ffe
make lint
Jamie-Stirling Sep 25, 2024
d3f9db2
reset corpus chunk size
Jamie-Stirling Sep 25, 2024
b94e4e2
remove wrong classification import
Jamie-Stirling Sep 26, 2024
a037081
Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i…
Jamie-Stirling Sep 26, 2024
b54c227
add Flickr30k T2I and I2T
Jamie-Stirling Sep 27, 2024
08ce105
add Flickr30k T2I retriebal
Jamie-Stirling Sep 27, 2024
5a45bc4
reduced-size MET revision
Jamie-Stirling Sep 27, 2024
9b8c46d
fix: add Flickr30k T2I
Jamie-Stirling Sep 29, 2024
65c2bfb
make lint
Jamie-Stirling Sep 29, 2024
8d79c5b
add two landmark datasets and results
Jamie-Stirling Oct 1, 2024
fe0d053
add Sketchy i2i retrieval
Jamie-Stirling Oct 2, 2024
999d680
resolve conflicts
Jamie-Stirling Oct 8, 2024
72ace6e
add task metadata
Jamie-Stirling Oct 14, 2024
5afb2aa
add BLINKIT2IRetrieval dataset
Jamie-Stirling Oct 14, 2024
4ba56b1
add BLINKIT2TRetrieval
Jamie-Stirling Oct 14, 2024
dd99ddc
add ImageCoDeT2IRetrieval
Jamie-Stirling Oct 14, 2024
411e313
make lint
Jamie-Stirling Oct 14, 2024
74a2400
add vizwiz retrieval and results
Jamie-Stirling Oct 15, 2024
8067b17
fix vizwiz duplicate texts
Jamie-Stirling Oct 15, 2024
a80f3bb
add new vizwiz results
Jamie-Stirling Oct 15, 2024
4e1155c
add VQA2 results
Jamie-Stirling Oct 16, 2024
e91faf6
add GLD v2 I2T retrieval
Jamie-Stirling Oct 17, 2024
f2721c0
add gld v2 i2i retrieval
Jamie-Stirling Oct 17, 2024
c9df7ac
make lint
Jamie-Stirling Oct 18, 2024
a730ecc
add AbsTaskAny2AnyMultiChoice
Jamie-Stirling Oct 18, 2024
dab5913
make lint
Jamie-Stirling Oct 18, 2024
1123d53
remove GLDv2I2IRetrieval
Jamie-Stirling Oct 18, 2024
e053546
Merge branch 'mieb-datasets' into mieb-any2image-multi
Jamie-Stirling Oct 18, 2024
6d22297
Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i…
Jamie-Stirling Oct 20, 2024
d0cddd4
exclude AbsTaskAny2AnyMultiChoice from test_load_data
Jamie-Stirling Oct 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
450 changes: 450 additions & 0 deletions mteb/abstasks/Image/AbsTaskAny2AnyMultiChoice.py

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions mteb/abstasks/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from .AbsTaskSpeedTask import *
from .AbsTaskSTS import *
from .AbsTaskSummarization import *
from .Image.AbsTaskAny2AnyMultiChoice import *
from .Image.AbsTaskAny2AnyRetrieval import *
from .Image.AbsTaskAny2TextMultipleChoice import *
from .Image.AbsTaskImageClassification import *
Expand Down
Loading