Hugging Face's logo

cecibas
/
Midnight-Miqu-70B-v1.5-4bit

Text Generation
Transformers
PyTorch
llama
conversational
text-generation-inference
Inference Endpoints
4-bit precision
awq
Model card Files Community
Edit model card

awq quant made with lmdeploy v0.4.2:

lmdeploy lite auto_awq sophosympatheia/Midnight-Miqu-70B-v1.5 --work-dir Midnight-Miqu-70B-v1.5-4bit
Downloads last month
191,253
Inference API
Text Generation
Examples
Input a message to start chatting with cecibas/Midnight-Miqu-70B-v1.5-4bit.