Paper: https://arxiv.org/abs/2308.13137
Code: https://github.com/OpenGVLab/OmniQuant
To run this model, refer https://github.com/OpenGVLab/OmniQuant/blob/main/runing_quantized_mixtral_7bx8.ipynb for more details.
Paper: https://arxiv.org/abs/2308.13137
Code: https://github.com/OpenGVLab/OmniQuant
To run this model, refer https://github.com/OpenGVLab/OmniQuant/blob/main/runing_quantized_mixtral_7bx8.ipynb for more details.