Ecosyste.ms: Timeline
Browse the timeline of events for every public repo on GitHub. Data updated hourly from GH Archive.
shamanez opened a pull request on arcee-ai/optillm-upstream
Add both serial and parrallel optillm method evaluations with arcee endpoint.
## Evaluate optillm with Arcee-Atlas To evaluate optillm with Arcee-Atlas, we provide a script to generate a dataset and another to evaluate the results. ## API kets Please set your API kets i...
shamanez pushed 1 commit to evals-atals-arcee arcee-ai/optillm-upstream
- added the gptqa diamand dataset 236f0e4
shamanez pushed 1 commit to evals-atals-arcee arcee-ai/optillm-upstream
- added parrallel evals 8ad5b40
shamanez pushed 1 commit to evals-atals-arcee arcee-ai/optillm-upstream
- added gptqa eval script b7ffbbc
shamanez created a branch on arcee-ai/optillm-upstream
evals-atals-arcee - Optimizing inference proxy for LLMs
shamanez opened a pull request on arcee-ai/lm-evaluation-harness
Evaluate Blueberry with the Eval Harness
- Modifed the Vllm class and added a custom decoding mechanism. - Can interchange between greedy and the blueberry ``` lm_eval --model vllm \ --model_args pretrained="Qwen/Qwen2.5-32B-I...
shamanez created a branch on arcee-ai/lm-evaluation-harness
blue-berry - A framework for few-shot evaluation of language models.
shamanez opened an issue on EleutherAI/lm-evaluation-harness
bbh_zeroshot fails during to a custom filter issue.
I am trying to run this setup: ``` lm_eval --model vllm \ --model_args pretrained="Qwen/Qwen2.5-0.5B-Instruct",tensor_parallel_size=2,dtype=auto,gpu_memory_utilization=0.8 \ --tasks b...
shamanez pushed 16 commits to main arcee-ai/optillm
- Update README.md 2d3b951
- Update eval_frames_benchmark.py add more timeout as fetching and going over wikipedia pages can take time. 193ab3c
- intit 21215c6
- Update README.md c0cb613
- update 9afdadb
- Merge pull request #62 from codelion/feat-router Feat router 41821aa
- Delete wim.py ec85ad7
- Update README.md 5d471d1
- decoding updates 38830a7
- Update entropy_decoding.py aef846c
- Update entropy_decoding.py d04eea1
- Update README.md 95cc14d
- Update optillm.py fix CLI args 5d541ee
- Update optillm.py fix --approach flag 83b5341
- Update moa.py add more logging 0df5291
- override approach if set in messages 94fad78
shamanez pushed 27 commits to main shamanez/optillm
- Update README.md ef775aa
- Update README.md d027ac4
- updates a46533e
- Add executecode plugin 0e90c1b
- Update README.md 09499ad
- Update README.md f9bac97
- Update README.md 1b62fc4
- Update train_optillm_classifier.py bert large for classification b6e827c
- deps add dependencies 1d84b2f
- Update optillm.py 575692f
- fix bug in execute code 4d00381
- Update README.md 2d3b951
- Update eval_frames_benchmark.py add more timeout as fetching and going over wikipedia pages can take time. 193ab3c
- intit 21215c6
- Update README.md c0cb613
- update 9afdadb
- Merge pull request #62 from codelion/feat-router Feat router 41821aa
- Delete wim.py ec85ad7
- Update README.md 5d471d1
- decoding updates 38830a7
- and 7 more ...
shamanez closed an issue on codelion/optillm
Thanks for adding the entropy based sampling, by any chance do you have a comprasion with other alternative methods?
Entropy sampling method comparison.
shamanez opened an issue on codelion/optillm
Thanks for adding the entropy based sampling, by any chance do you have a comprasion with other alternative methods?
Entropy sampling method comparison.
shamanez created a comment on an issue on arcee-ai/DAM
Thanks, @ArcherShirou, for exploring our codebase. > would like to inquire if there are plans to support the Qwen2.5 && Qwen2 series or other popular models from the open-source community, such ...
shamanez created a review on a pull request on arcee-ai/DAM
You also need to add the modified modeling.py file to make this work. Ex: checkout the changes we did for the Llama3 - https://github.com/arcee-ai/DAM/blob/main/dam/modeling/llama3/modeling.py
shamanez created a review on a pull request on arcee-ai/DAM
Thanks a lot. Just one quick thing: did you test the training of the DAM with the merged model? Just want to test the end2end compatibility.
shamanez created a comment on an issue on arcee-ai/DAM
Adding new models is super easy and a two-minute thing. @thomasgauthier, maybe we can add a description to the README. @SolshineCode In the meantime, feel free to do a PR :) . Thanks again for ...
shamanez created a comment on an issue on arcee-ai/DAM
Can you please try this command? `python dam/merge.py mistralai/Mistral-7B-v0.1 augmxnt/shisa-gamma-7b-v1 WizardLM/WizardMath-7B-V1.1 arcee-train/Abel-7B-002-truncated-embeds --device cuda --outpu...
shamanez created a comment on an issue on arcee-ai/DAM
Hi @SolshineCode, thanks for trying our work. I think the problem is with the [modeling files](https://github.com/arcee-ai/DAM/tree/main/dam/modeling). As you can see, we support Mistral and Llama...
shamanez pushed 1 commit to main arcee-ai/DAM
- Update README.md (#42) * Update README.md * Update README.md * Update README.md * Update README.md ca90e81