Ecosyste.ms: Timeline
Browse the timeline of events for every public repo on GitHub. Data updated hourly from GH Archive.
sam-hey created a comment on a pull request on embeddings-benchmark/leaderboard
Hello @x-tabdeveloping, Yes, as @Samoed already pointed out, I have submitted some missing tasks for German models. Additionally, I was able to secure more GPU resources, so I plan to add as man...
sam-hey opened a pull request on embeddings-benchmark/results
add german results for ger models
## Checklist <!-- Please do not delete this --> - [x] Run tests locally to make sure nothing is broken using `make test`. - [x] Run the results files checker `make pre-push`.
sam-hey closed a pull request on embeddings-benchmark/leaderboard
update readme to match results repo
Update to match: https://github.com/embeddings-benchmark/results > Previously it was possible to submit models results to MTEB by adding the results to the model metadata. This is no longer an...
sam-hey created a comment on a pull request on lightonai/pylate
You're absolutely right about that! At the moment, it's better suited for reranking tasks where recall is excellent, but performance can be a bottleneck due to its slower speeds. I'm also eager to ...
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
Great job! However, I'm not sure how long it will take. A new merge doesn't always correspond to a new version being released on PyPI. Since this PR includes a lot of changes, I would recommend pro...
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
BTW, the call is also within a method that is static, so `self` is not available.
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
I made the method `@staticmethod` because it does not require `self`, making it impossible to call it with `self`.
sam-hey created a comment on a pull request on embeddings-benchmark/mteb
> `max_sim` was used in main branch and I thought it was tested and worked with this name The `max_sim` function was updated to `MaxSim` to align with the upcoming pylate integration in a future...
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
I added a check to ensure that the revision is the same. The case mentioned above would have worked, but it would have been ignored if there was an error in ST
sam-hey pushed 1 commit to ci/test_colbert_bm25s sam-hey/mteb
- fix: colbert test py 3.9 & add revision c6bd6f8
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
I actually prefer this approach because: - No magic literals. - Less error-prone due to typos in the name.
sam-hey pushed 1 commit to ci/test_colbert_bm25s sam-hey/mteb
- fix: MaxSim add test fix ModelMeta 5420f20
sam-hey created a comment on a pull request on embeddings-benchmark/mteb
Why was `MaxSim` renamed to `max_sim`? This change breaks `ColBERT`. @Samoed, I’ll fix it and add a test to prevent similar issues in the future.
sam-hey pushed 45 commits to ci/test_colbert_bm25s sam-hey/mteb
- fix: Added C-MTEB (#1786) Added C-MTEB 3ba7e22
- 1.29.1 Automatically generated by python-semantic-release 48370c7
- docs: Add contact to MMTEB benchmarks (#1796) * Add myself to MMTEB benchmarks * lint e9e9118
- fix: loading pre 11 (#1798) * fix loading pre 11 * add similarity * lint * run all task types 94103e6
- 1.29.2 Automatically generated by python-semantic-release b6fb5b8
- fix: allow to load no revision available (#1801) * fix allow to load no revision available * lint * add requir... a202884
- 1.29.3 Automatically generated by python-semantic-release bcb2cd9
- fix: Zero shot and aggregation on Leaderboard (#1810) * Made join_revision filter out no_revision_available when oth... 0acc166
- fix: Added `ModelMeta` for BGE, GTE Chinese and multilingual models (#1811) * Added BGE Chinese and multilingual-gem... 3f5ee82
- 1.29.4 Automatically generated by python-semantic-release 217dabe
- fix: Add additional contacts (#1817) add contacts from #1790 c4ee9fe
- Update points table e3a3df8
- 1.29.5 Automatically generated by python-semantic-release 186cc23
- fix: Added more Chinese models' `ModelMeta` (#1814) * Added Multilingual USE models * Added Moka models * Adde... 748955c
- 1.29.6 Automatically generated by python-semantic-release 950f050
- Add model inf-retriever-v1 (#1744) * feat(models): add infly/inf-retriever-v1 model metadata- Add inf_models.py file... 60c4980
- ci: only return 1 model_name per file (#1818) * only return 1 model_name per file * fix args parse * revert test c... d7a7791
- fix: add bge-m3 `ModelMeta` (#1821) add bge 4ac59bc
- 1.29.7 Automatically generated by python-semantic-release 9733d85
- fix: Added Chinese Stella models (#1824) Added Chinese Stella models 74b495c
- and 25 more ...
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
I've added some more; please take a look
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
I believe .as_posix() is nicer than str(), so I would convert everything to use .as_posix() instead.
sam-hey pushed 1 commit to german sam-hey/mteb-leaderboard
- Update config.yaml Co-authored-by: Roman Solomatin <[email protected]> 1ddfc78
sam-hey created a comment on a pull request on embeddings-benchmark/mteb
@Samoed can you give me a review if you have some time?