Error when using Inference API

#19
by Krooz - opened
response = client.chat_completion(
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    max_tokens=100,
    model = "meta-llama/Llama-3.2-1B",
    seed=42,
    stream=False
)

print(response)

When i try to use InferenceClient for accessing this model, i was getting the following error,

HfHubHTTPError: 422 Client Error: Unprocessable Entity for url: https://api-inference.huggingface.co/models/meta-llama/Llama-3.2-1B/v1/chat/completions (Request ID: 8wEASJgGRRA0nxDgqjZO0)

Template error: template not found

This is the base model which doesn't have a chat template because it is not trained for chatting.

Try using this model: meta-llama/Llama-3.2-1B-Instruct

Sign up or log in to comment