Ecosyste.ms: Timeline
Browse the timeline of events for every public repo on GitHub. Data updated hourly from GH Archive.
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
Sure, I've pushed an update to reflect this change.
sam-hey opened a pull request on embeddings-benchmark/mteb
doc: update pr template
Hello, I’d like to propose an update to the PR template to improve the stability of MTEB. The goal is to ensure that all PRs include appropriate tests and updated documentation where needed. ...
sam-hey pushed 52 commits to main sam-hey/mteb
- fix: update max tokens for OpenAI (#1772) update max tokens 0c5c3a5
- ci: skip AfriSentiLID for now (#1785) * skip AfriSentiLID for now * skip relevant test case instead --------- ... 71dbd61
- 1.28.7 Automatically generated by python-semantic-release bad27a6
- ci: fix model loading test (#1775) * pass base branch into the make command as an arg * test a file that has cust... 9b117a8
- feat: Update task filtering, fixing bug which included cross-lingual tasks in overly many benchmarks (#1787) * feat:... 4a70e5d
- 1.29.0 Automatically generated by python-semantic-release 15a6812
- fix: Added C-MTEB (#1786) Added C-MTEB 3ba7e22
- 1.29.1 Automatically generated by python-semantic-release 48370c7
- docs: Add contact to MMTEB benchmarks (#1796) * Add myself to MMTEB benchmarks * lint e9e9118
- fix: loading pre 11 (#1798) * fix loading pre 11 * add similarity * lint * run all task types 94103e6
- 1.29.2 Automatically generated by python-semantic-release b6fb5b8
- fix: allow to load no revision available (#1801) * fix allow to load no revision available * lint * add requir... a202884
- 1.29.3 Automatically generated by python-semantic-release bcb2cd9
- fix: Zero shot and aggregation on Leaderboard (#1810) * Made join_revision filter out no_revision_available when oth... 0acc166
- fix: Added `ModelMeta` for BGE, GTE Chinese and multilingual models (#1811) * Added BGE Chinese and multilingual-gem... 3f5ee82
- 1.29.4 Automatically generated by python-semantic-release 217dabe
- fix: Add additional contacts (#1817) add contacts from #1790 c4ee9fe
- Update points table e3a3df8
- 1.29.5 Automatically generated by python-semantic-release 186cc23
- fix: Added more Chinese models' `ModelMeta` (#1814) * Added Multilingual USE models * Added Moka models * Adde... 748955c
- and 32 more ...
sam-hey created a comment on a pull request on embeddings-benchmark/leaderboard
@Samoed, I believe we can merge now. The leaderboard looks good to me, and I’ve updated the README to point to the main MTEB project documentation for adding results. Centralizing all the documenta...
sam-hey created a comment on a pull request on embeddings-benchmark/results
@Samoed @x-tabdeveloping I believe that's all for now. When you have some time, please take a look. Thanks, and have a great weekend!
sam-hey pushed 1 commit to german sam-hey/mteb-leaderboard
- update External model results german 2c6eae4
sam-hey pushed 11 commits to german sam-hey/mteb-leaderboard
- Automated Leaderboard Update ad6aa4c
- Automated Leaderboard Update 42d1aaf
- Automated Leaderboard Update 1b1d5b4
- Automated Leaderboard Update da0f001
- update readme 47cf31a
- add new config for german retrieval 589bfc8
- add german to leaderboard 26e6a30
- match benchmarks config (de) 500e8b7
- Revert "add german to leaderboard" This reverts commit a6ea180b9181f1513e548ab5bb01587e81612d31. d909a3d
- update config " " d9c425d
- Update config.yaml Co-authored-by: Roman Solomatin <[email protected]> 38db99a
sam-hey opened a draft pull request on embeddings-benchmark/results
Add results for the German benchmark across all models
## Checklist <!-- Please do not delete this --> - [x] Run tests locally to make sure nothing is broken using `make test`. - [x] Run the results files checker `make pre-push`.
sam-hey created a comment on a pull request on embeddings-benchmark/leaderboard
Hello @x-tabdeveloping, Yes, as @Samoed already pointed out, I have submitted some missing tasks for German models. Additionally, I was able to secure more GPU resources, so I plan to add as man...
sam-hey opened a pull request on embeddings-benchmark/results
add german results for ger models
## Checklist <!-- Please do not delete this --> - [x] Run tests locally to make sure nothing is broken using `make test`. - [x] Run the results files checker `make pre-push`.
sam-hey closed a pull request on embeddings-benchmark/leaderboard
update readme to match results repo
Update to match: https://github.com/embeddings-benchmark/results > Previously it was possible to submit models results to MTEB by adding the results to the model metadata. This is no longer an...
sam-hey created a comment on a pull request on lightonai/pylate
You're absolutely right about that! At the moment, it's better suited for reranking tasks where recall is excellent, but performance can be a bottleneck due to its slower speeds. I'm also eager to ...