Ecosyste.ms: Timeline
Browse the timeline of events for every public repo on GitHub. Data updated hourly from GH Archive.
Narsil created a review comment on a pull request on huggingface/text-generation-inference
I don't see it.
drbh created a branch on huggingface/text-generation-inference
return-streaming-error-in-openai-client-compatible-format - Large Language Model Text Generation Inference
sam-ulrich1 opened an issue on huggingface/text-generation-inference
Optionally log input tokens/prompt
### Feature request Optionally log the input prompt/tokens for improved debugging. ### Motivation I am currently attempting to debug why in a prod env I am getting garbage but when replicating t...
drbh pushed 1 commit to pr-2634-ci-branch huggingface/text-generation-inference
- fix: adjust tool choice type in test 16e4c38
Narsil created a comment on a pull request on huggingface/text-generation-inference
IT's merged from an updated PR I prepared for CI (https://github.com/huggingface/text-generation-inference/pull/2665) (only minor fixes have been updated in the control flow and adding a few comme...
Narsil created a comment on a pull request on huggingface/text-generation-inference
As a side note, when I'm using ALL cores over both sockets, I get ~110ms/token. Not sure if the value is normal but I figure the socket crossing has overhead compensating the speedup maybe ? Cou...
Narsil pushed 1 commit to main huggingface/text-generation-inference
- CI job. Gpt awq 4 (#2665) * add gptq and awq int4 support in intel platform Signed-off-by: Wang, Yi A <yi.a.wang@... 153ff37
Narsil closed a pull request on huggingface/text-generation-inference
CI job. Gpt awq 4
# What does this PR do? <!-- Congratulations! You've made it this far! You're not quite done yet though. Once merged, your PR is going to appear in the release notes with the title you set, ...
Narsil pushed 1 commit to gpt_awq_4 huggingface/text-generation-inference
- Update server/text_generation_server/layers/gptq/__init__.py 0229c71
Narsil created a review comment on a pull request on huggingface/text-generation-inference
```suggestion # Remove g_idx[0] to adapt the check with TP>1. (g_idx - g_idx[0]).cpu(), ```
Narsil created a comment on a pull request on huggingface/text-generation-inference
> are you using ping cpu cores like " --cpuset-cpus=0-55"? or could you adjust the cpu cores. Indeed I adapted since I don't have the same number of cores. I tried to numactl the docker command ...
danieldk created a review comment on a pull request on huggingface/text-generation-inference
Yeah makes sense, would be a good to have a short comment here though so that future-us don't have to re-parse what is going on here.
mht-sharma created a review on a pull request on huggingface/text-generation-inference
Thanks @danieldk, LGTM This make things clearer
danieldk pushed 1 commit to maintenance/reciprocal-handling huggingface/text-generation-inference
- Make handling of FP8 scales more consisent Change `fp8_quantize` so that we can pass around reciprocals everywhere, ... f5b0946
drbh pushed 1 commit to pr-2634-ci-branch huggingface/text-generation-inference
- fix: add missing snapshot file 6102a95
drbh created a review comment on a pull request on huggingface/text-generation-inference
good catch, updated to `serde(rename)` and added a test to ensure that the `"none"` values is respected correctly
drbh created a review comment on a pull request on huggingface/text-generation-inference
yes thank you, updated the correct argument in the latest commit
drbh created a review comment on a pull request on huggingface/text-generation-inference
oh yes that worked, I was originally under the impression the Option was still needed to handle all http request but this is much cleaner. Thank you