Hugging Face's logo

MykytaPetrenko
/
Hermes-3-Llama-3.1-8B-GPTQ-4bit

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community
Edit model card

Model Card for Model ID

Model Details

Model Description

This repo contains GPTQ 8bit version of for NousResearch Hermes 3 - Llama-3.1 8B.

I created this to use with ExLlama2 within oobabooga web-ui

Downloads last month
10
Safetensors
Model size
1.99B params
Tensor type
FP16
·
I32
·
Inference Examples
Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.