Edit model card
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Quantization made by Richard Erkhov.

Github

Discord

Request more models

vicuna-160m - GGUF

Name Quant method Size
vicuna-160m.Q2_K.gguf Q2_K 0.07GB
vicuna-160m.IQ3_XS.gguf IQ3_XS 0.07GB
vicuna-160m.IQ3_S.gguf IQ3_S 0.07GB
vicuna-160m.Q3_K_S.gguf Q3_K_S 0.07GB
vicuna-160m.IQ3_M.gguf IQ3_M 0.08GB
vicuna-160m.Q3_K.gguf Q3_K 0.08GB
vicuna-160m.Q3_K_M.gguf Q3_K_M 0.08GB
vicuna-160m.Q3_K_L.gguf Q3_K_L 0.08GB
vicuna-160m.IQ4_XS.gguf IQ4_XS 0.09GB
vicuna-160m.Q4_0.gguf Q4_0 0.09GB
vicuna-160m.IQ4_NL.gguf IQ4_NL 0.09GB
vicuna-160m.Q4_K_S.gguf Q4_K_S 0.09GB
vicuna-160m.Q4_K.gguf Q4_K 0.1GB
vicuna-160m.Q4_K_M.gguf Q4_K_M 0.1GB
vicuna-160m.Q4_1.gguf Q4_1 0.1GB
vicuna-160m.Q5_0.gguf Q5_0 0.11GB
vicuna-160m.Q5_K_S.gguf Q5_K_S 0.11GB
vicuna-160m.Q5_K.gguf Q5_K 0.11GB
vicuna-160m.Q5_K_M.gguf Q5_K_M 0.11GB
vicuna-160m.Q5_1.gguf Q5_1 0.12GB
vicuna-160m.Q6_K.gguf Q6_K 0.12GB
vicuna-160m.Q8_0.gguf Q8_0 0.16GB

Original model description:

license: apache-2.0 datasets: - anon8231489123/ShareGPT_Vicuna_unfiltered language: - en pipeline_tag: text-generation

Model description

This is a Vicuna-like model with only 160M parameters, which is fine-tuned from LLaMA-160m on ShareGPT data.

The training setup follows the Vicuna suite.

The model is mainly developed as a base Small Speculative Model in MCSD paper. As a comparison, it can be better aligned to the Vicuna models than LLaMA-160m with little loss of alignment to the LLaMA models.

Draft Model Target Model Alignment
LLaMA-68/160M LLaMA-13/33B πŸ˜ƒ
LLaMA-68/160M Vicuna-13/33B 😟
Vicuna-68/160M LLaMA-13/33B πŸ˜ƒ
Vicuna-68/160M Vicuna-13/33B πŸ˜ƒ
Downloads last month
282
GGUF
Model size
162M params
Architecture
llama

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference API
Unable to determine this model's library. Check the docs .