Ecosyste.ms: Timeline
Browse the timeline of events for every public repo on GitHub. Data updated hourly from GH Archive.
sam-hey created a comment on a pull request on embeddings-benchmark/results
@KennethEnevoldsen A bit short on time this month, but I'll take a look next month
sam-hey created a comment on an issue on embeddings-benchmark/mteb
This commit can be used as ref: https://github.com/embeddings-benchmark/mteb/commit/5a023d6f661932f6bf92c4b3b435b0c128138f75 It should include all necessary changes to use `Enums` for `similarity_...
sam-hey created a comment on a pull request on lightonai/pylate
I added them as a review, thinking you would be able to see them, but it seems they are in a pending state, so that’s not the case. I also added a comment, which should be visible to you. > You al...
sam-hey created a comment on a pull request on lightonai/pylate
It seems that my comments are in Pending state, so they can't be seen at the moment. I have just one small regrade: I wouldn’t categorize this as a warning, as it’s expected behavior. It’s merel...
sam-hey created a comment on a pull request on lightonai/pylate
A huge thank you from my side! I did add two comments, but they're just small nitpicks :) Otherwise, I tested it locally, and it looks good to me. Thanks a lot for the effort! So just the Ruff v...
sam-hey created a comment on a pull request on lightonai/pylate
I want to highlight this for two main reasons: First, I encountered an issue with the configuration not being loaded properly, and second, I wasn’t sure what exactly was being loaded that could be ...
sam-hey created a comment on a pull request on lightonai/pylate
I applied your feedback to make the warning message more precise and also fixed some documentation.
sam-hey created a comment on an issue on embeddings-benchmark/mteb
Thanks a lot for your effort so far! I would probably go with setting all non-valid operations to None or NaN, as this clearly indicates the absence of a valid value. However, let's see what @orion...
sam-hey created a comment on an issue on embeddings-benchmark/mteb
Ok, thanks a lot for your feedback! I will take a look at it!
sam-hey created a comment on an issue on embeddings-benchmark/mteb
I'm not sure if I'm missing something, but in my opinion, it should be possible to simply inherit from one of the Adapter classes in the `ModelWrapper` (SentenceTransformerWrapper etc.) class. This...
sam-hey created a comment on an issue on embeddings-benchmark/mteb
I would like to propose an improvement to MTEB by introducing an Adapter Class between Model Classes and Tasks. Currently, the logic in [RetrievalEvaluator](https://github.com/embeddings-benchmark/...
sam-hey created a comment on an issue on embeddings-benchmark/mteb
Hello @imadtyx, Thanks a lot for working on this! As @Samoed pointed out, it's not possible to achieve this by simply searching for the text. I suggest reducing the scope of the PR to first just...
sam-hey created a comment on an issue on lightonai/pylate
Yes, this is exactly what I mean. No problem, I totally understand the time aspect—just knowing that it will come eventually is reassuring. My experiments showed a small difference, but it wasn’t ...
sam-hey opened a pull request on lightonai/pylate
ref: ruff python 3.8 & update warning
Align Ruff with setup.py: Set Python version to **3.8** for Ruff to ensure consistency with `setup.py` https://github.com/lightonai/pylate/blob/c6a22fcdc313ee1c2ced454f77bd4c199cb964b4/setup.py...
sam-hey pushed 56 commits to custom_search sam-hey/mteb
- fix: Ensure seed is based on RNG State (#1193) e2520df
- Consolidate Retrieval/Reranking/Instruction Variants (#1359) * update * merged retrieval; working * update tas... 9c58518
- fix: Unsure TaskResults can handle runtime and version being unspecified 2a8a370
- Merge branch 'v2.0.0' of https://github.com/embeddings-benchmark/mteb into v2.0.0 dea2b77
- fix: remove NaN handling for retrieval 23d6cb2
- Merge branch 'main' into v2.0.0 8868cd4
- fix: Merge main into v2 (#1454) * fix: Count unique texts, data leaks in calculate metrics (#1438) * add more sta... 5470c88
- feat: enable codecarbon by default (#1428) * enable codecarbon by default * lint * update flag * add allow_... 70a3ff2
- Add decriptive stat almost to all datasets (#1466) * run tasks * remove test script * lint * remove cache ... 0e9b6fd
- fix: Fix test for empty descriptive tasks (#1413) * fix test * skip mock * add message to assert * fix tes... 0a5bedb
- fix: pin datasets version <3.0.0 (#1471) 6da2a1a
- feat: Multilingual retrieval loader (#1473) * multilingual loader * lint a27de33
- fix: add citations to ModelMeta (#1477) * add citations * fix typo 0df0210
- Add descriptive stats to mising tasks and add number of qrels (#1476) * add code for comupting number of qrels * ... 0abe1a0
- 1475 add descriptive stats to all tasks v2 (#1482) * add code for comupting number of qrels * BibleNLPBitextMinin... a7a5214
- fix: Fix `BrightRetrieval` calculate stats (#1484) * fix bright loader * lint * fix comment 99247b2
- Merge main v2 (#1504) * fix: Count unique texts, data leaks in calculate metrics (#1438) * add more stat * ad... 022d355
- Fix: retrieval stats (#1496) * fix bright loader * lint * fix comment * fix stats * fix retrieval stats ... 6383950
- fix: hatespeech filipino (#1522) * fix FilipinoHateSpeechClassification * update tests d54fb75
- feat: Forbid task metadata and add upload functions (#1362) * init * find all wierd repos * move to mteb Wikipedia... dec5d6a
- and 36 more ...
sam-hey created a comment on an issue on lightonai/pylate
Hey, thanks for the response! I should probably rephrase my question a bit. What I mean is the recall for the index—because, as I understand it, with the index approach, many MaxSim operations aren...
sam-hey created a comment on a pull request on embeddings-benchmark/mteb
> Thanks @sam-hey for starting! > > Like I said last time, I think it would make more sense to define a new category of search for sparse models. That way we can add more than just BM25S (Pyseri...
sam-hey created a review comment on a pull request on embeddings-benchmark/mteb
Yes, I see your point. I'll integrate it into the interface.
sam-hey created a comment on a pull request on embeddings-benchmark/mteb
> Shouldn't this be merged into the v2 branch? It doesn’t break any existing functions, so I wouldn’t say so.