Ecosyste.ms: Timeline
Browse the timeline of events for every public repo on GitHub. Data updated hourly from GH Archive.
YangWang92 created a comment on an issue on microsoft/VPTQ
https://github.com/microsoft/VPTQ/tree/main/csrc Thanks!
YangWang92 created a comment on an issue on THUDM/GLM-Edge
> 这部分由于我无法拿到对应的来自高通的开源工具包，因此，我们也没办法将对应属于高通的工具包进行开源 It seems that running models on QNN doesn't require the "open-source" Qualcomm toolkit, right?
YangWang92 opened an issue on THUDM/GLM-Edge
Reuqest: QNN Qualcomm inference example
### Feature request / 功能建议 Hi all, the README mentions testing on the Qualcomm 8 Elite (Gen4) platform with all models running on the NPU. Is there an early demo available for testing? Which p...
YangWang92 created a comment on an issue on microsoft/VPTQ
I plan to write a blog later to explain the algorithm details in depth. The algorithm in the paper is written quite concisely, so feel free to ask more questions!
YangWang92 created a comment on an issue on microsoft/VPTQ
Indeed, our code has a somewhat research-oriented style, haha, so some parts might not be very clear. Let me quickly explain: 1. **`quant_data`**: This is used to compute the proxy error for eval...
YangWang92 closed an issue on microsoft/VPTQ
tow stage code
1.In the two-stage quantization, two similar functions, the init_centroids_indices&& init_res_centroids_indices function and the quantize_vector &&quantize_residual_vector function, are defined res...
YangWang92 created a comment on an issue on microsoft/VPTQ
Additionally, I speculate that a 70B model at 2-bit might achieve stronger performance on certain benchmarks. Although I can’t prove this yet, I plan to conduct a thorough analysis on this in the f...
YangWang92 created a comment on an issue on microsoft/VPTQ
Yes, I really appreciate your question—it’s thought-provoking, and I’m seriously reflecting on it while trying to pursue rigorous research. If you're interested, we could collaborate directly. You ...
YangWang92 closed an issue on microsoft/VPTQ
Question about the design motivation behind VPTQ
Thank you for your insightful and thought-provoking work. I have a question regarding the motivation behind low-bit quantization and its potential as a solution for enabling extremely low-bit quant...
YangWang92 created a comment on an issue on microsoft/VPTQ
Yes, in the paper, we included layer-wise fine-tuning. However, we recently found that running end-to-end fine-tuning performs better than layer-wise fine-tuning. I removed the code for layer-wise ...
YangWang92 closed an issue on microsoft/VPTQ
where are the layer-wise fine-tune codes in algorithm branch?
In section 4, you suggest to perform layer-wise fine-tune after perform vptq stage, but I cannot find any code in algorithm branch.
YangWang92 created a comment on an issue on microsoft/VPTQ
The llama 3-8b models in the paper have been fine-tuned, especially around ~2-bit, where even just a few hundred iterations of end-to-end fine-tuning significantly improved model accuracy. We plan ...
YangWang92 closed an issue on microsoft/VPTQ
Question about result reproduction
 <!-- Failed to upload "image.png" --> I tried this config to quantize the model but didn't get as good res...
YangWang92 created a comment on an issue on microsoft/VPTQ
> Thank you for your quick response. I set `--vector_lens -1 12` because, in line 226 of `./vptq/quantizer.py`, it notes: > > if num_centroids == -1: # Do not quantize, keep original data > I ass...
YangWang92 created a comment on an issue on microsoft/VPTQ
This configuration looks a bit odd. When we set `--npercent 1`, it extracts 1% of the outliers to build a separate lookup table. However, with `--vector_lens -1 12`, the vector length of the outlie...
YangWang92 created a comment on an issue on microsoft/VPTQ
> > Hi [@ShawnzzWu](https://github.com/ShawnzzWu) > > Would you mind sharing your quantized model so I can debug into it? > > Sorry, for information security reasons, I'm not allowed to share my f...
YangWang92 created a comment on an issue on microsoft/VPTQ
> I've been trying to quantize and run the Meta-Llama-3.1-8B-Instruct-2.3bit model with group number set to 4, and successfully run the model when k1(centroids) is 4096 as in the paper. However, an...