Skip to content

Issues: embeddings-benchmark/mteb

MMTEB Overview
#360 opened Apr 13, 2024 by KennethEnevoldsen
Open 4
Language Overview
#366 opened Apr 14, 2024 by KennethEnevoldsen
Open
Finalizing MMTEB
#784 opened May 22, 2024 by KennethEnevoldsen
Open 6
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

94 Open 259 Closed
94 Open 259 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Solution for transforming retrieval datasets into parquet
#1090 opened Jul 15, 2024 by gowitheflow-1998
Issue with Language Classification Task for Swahili in MTEB
#1086 opened Jul 14, 2024 by msamwelmollel
1
FollowIR instruction format
#1083 opened Jul 13, 2024 by Muennighoff
@orionw
3
Distinguishing instructions for RAR-b
#1066 opened Jul 10, 2024 by Muennighoff
9
Integrate CoIR good first issue Good for newcomers help wanted Extra attention is needed
#1065 opened Jul 10, 2024 by Muennighoff
1
LLM2Vec multi-gpu error
#1052 opened Jul 6, 2024 by Muennighoff
1
Implements check on existing and new datasets enhancement New feature or request
#1049 opened Jul 5, 2024 by KennethEnevoldsen
1
Implements a custom for documents to make data format consistent enhancement New feature or request
#1047 opened Jul 5, 2024 by KennethEnevoldsen
Update prints to use a consistent format: good first issue Good for newcomers help wanted Extra attention is needed
#1046 opened Jul 5, 2024 by KennethEnevoldsen
2
Standardize scores structure for tasks
#1037 opened Jul 3, 2024 by Samoed
1
2
Leaks and duplications in the MTEB leaderboard bug Something isn't working
#1036 opened Jul 3, 2024 by lbourdois
10
Avoid try except when raise_errors = True
#1035 opened Jul 3, 2024 by KennethEnevoldsen
1
MSMARCOv2 is excessive large and takes too long to load
#1022 opened Jul 1, 2024 by Muennighoff
6
Confusion re: Retrieval w/Instructions
#1013 opened Jun 30, 2024 by Muennighoff
6
Paper writing: Authors + appendix author contributions [needs review]
#1009 opened Jun 28, 2024 by KennethEnevoldsen
@KennethEnevoldsen @imenelydiaker
3
Paper Writing: Speeding up the benchmark [waiting for retrieval]
#1007 opened Jun 28, 2024 by KennethEnevoldsen
@KennethEnevoldsen @orionw @isaac-chung
3
Paper Writing: Methods - The Open-source Effort [in need of review]
#1006 opened Jun 28, 2024 by KennethEnevoldsen
@KennethEnevoldsen @imenelydiaker
3
Paper Writing: Related works [probably finished]
#1005 opened Jun 28, 2024 by KennethEnevoldsen
@isaac-chung
9
extracting embedding from other than CLS token during eval mteb
#999 opened Jun 28, 2024 by riyajatar37003
Option to remove cached dataset files on large runs enhancement New feature or request
#984 opened Jun 25, 2024 by isaac-chung
2
Be able to cache embeddings and load them enhancement New feature or request good first issue Good for newcomers
#946 opened Jun 17, 2024 by orionw
5
Allow for model specific similarity in Bitext Mining and Retrieval enhancement New feature or request
#943 opened Jun 17, 2024 by KennethEnevoldsen
Avoid using global seeds enhancement New feature or request
#942 opened Jun 17, 2024 by KennethEnevoldsen
Standardising matrix arrays (adopt Array API spec) enhancement New feature or request
#941 opened Jun 17, 2024 by KennethEnevoldsen
End-to-end integration / testing with leaderboard
#932 opened Jun 15, 2024 by Muennighoff
33
ProTip! no:milestone will show everything without a milestone.