Ecosyste.ms: Timeline
Browse the timeline of events for every public repo on GitHub. Data updated hourly from GH Archive.
sam-hey created a comment on an issue on embeddings-benchmark/mteb
As mentioned in https://github.com/embeddings-benchmark/mteb/pull/1869#issuecomment-2614282333, I suggest developing a more general method that is not limited to Cross-Encoders but can also be appl...
sam-hey created a comment on a pull request on embeddings-benchmark/mteb
Suggestion: If I understand the intent of this PR correctly, the primary focus is to enable filtering on the leaderboard. Wouldn't it make more sense to add a list of model types as an Enum? This w...
sam-hey created a comment on a pull request on embeddings-benchmark/leaderboard
@orionw, since I know you definitely have the correct rights, I hope you can help out! Thanks a lot!
sam-hey pushed 3 commits to main sam-hey/mteb-results
sam-hey created a comment on an issue on embeddings-benchmark/mteb
It could be a great addition to include a script in the results repository that generates results with the correct settings, such as having CodeCarbon activated, as @Samoed suggested. Additionally...
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
@Samoed, I would definitely encourage measuring this in the pipeline to ensure everyone is aware of any changes not covered by tests. However, we need to keep in mind that coverage can sometimes be...
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
Sure, I've pushed an update to reflect this change.
sam-hey opened a pull request on embeddings-benchmark/mteb
doc: update pr template
Hello, I’d like to propose an update to the PR template to improve the stability of MTEB. The goal is to ensure that all PRs include appropriate tests and updated documentation where needed. ...
sam-hey pushed 52 commits to main sam-hey/mteb
- fix: update max tokens for OpenAI (#1772) update max tokens 0c5c3a5
- ci: skip AfriSentiLID for now (#1785) * skip AfriSentiLID for now * skip relevant test case instead --------- ... 71dbd61
- 1.28.7 Automatically generated by python-semantic-release bad27a6
- ci: fix model loading test (#1775) * pass base branch into the make command as an arg * test a file that has cust... 9b117a8
- feat: Update task filtering, fixing bug which included cross-lingual tasks in overly many benchmarks (#1787) * feat:... 4a70e5d
- 1.29.0 Automatically generated by python-semantic-release 15a6812
- fix: Added C-MTEB (#1786) Added C-MTEB 3ba7e22
- 1.29.1 Automatically generated by python-semantic-release 48370c7
- docs: Add contact to MMTEB benchmarks (#1796) * Add myself to MMTEB benchmarks * lint e9e9118
- fix: loading pre 11 (#1798) * fix loading pre 11 * add similarity * lint * run all task types 94103e6
- 1.29.2 Automatically generated by python-semantic-release b6fb5b8
- fix: allow to load no revision available (#1801) * fix allow to load no revision available * lint * add requir... a202884
- 1.29.3 Automatically generated by python-semantic-release bcb2cd9
- fix: Zero shot and aggregation on Leaderboard (#1810) * Made join_revision filter out no_revision_available when oth... 0acc166
- fix: Added `ModelMeta` for BGE, GTE Chinese and multilingual models (#1811) * Added BGE Chinese and multilingual-gem... 3f5ee82
- 1.29.4 Automatically generated by python-semantic-release 217dabe
- fix: Add additional contacts (#1817) add contacts from #1790 c4ee9fe
- Update points table e3a3df8
- 1.29.5 Automatically generated by python-semantic-release 186cc23
- fix: Added more Chinese models' `ModelMeta` (#1814) * Added Multilingual USE models * Added Moka models * Adde... 748955c
- and 32 more ...
sam-hey created a comment on a pull request on embeddings-benchmark/leaderboard
@Samoed, I believe we can merge now. The leaderboard looks good to me, and I’ve updated the README to point to the main MTEB project documentation for adding results. Centralizing all the documenta...
sam-hey created a comment on a pull request on embeddings-benchmark/results
@Samoed @x-tabdeveloping I believe that's all for now. When you have some time, please take a look. Thanks, and have a great weekend!
sam-hey pushed 1 commit to german sam-hey/mteb-leaderboard
- update External model results german 2c6eae4
sam-hey pushed 11 commits to german sam-hey/mteb-leaderboard
- Automated Leaderboard Update ad6aa4c
- Automated Leaderboard Update 42d1aaf
- Automated Leaderboard Update 1b1d5b4
- Automated Leaderboard Update da0f001
- update readme 47cf31a
- add new config for german retrieval 589bfc8
- add german to leaderboard 26e6a30
- match benchmarks config (de) 500e8b7
- Revert "add german to leaderboard" This reverts commit a6ea180b9181f1513e548ab5bb01587e81612d31. d909a3d
- update config " " d9c425d
- Update config.yaml Co-authored-by: Roman Solomatin <[email protected]> 38db99a
sam-hey opened a draft pull request on embeddings-benchmark/results
Add results for the German benchmark across all models
## Checklist <!-- Please do not delete this --> - [x] Run tests locally to make sure nothing is broken using `make test`. - [x] Run the results files checker `make pre-push`.