YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Custom GGUF quants of Google’s gemma-2-2b-it, where the Output Tensors are quantized to Q8_0 or kept at F32 while the Embeddings are kept at F32. 🧠🔥🚀

Notes: Great SMOL LLM for on-device inference for mobile devices. 😋

GGUF

Model size

2.61B params

Architecture

gemma2

4-bit

6-bit

8-bit

Inference API

Unable to determine this model's library. Check the docs .