schmuell/TinyLlama-1.1B-Chat-v1.0-int4

https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0 with ONNX weights to be compatible with Transformers.js.

Usage (Transformers.js)

If you haven't already, you can install the Transformers.js JavaScript library from NPM using:

npm i @xenova/transformers

Example: Text generation with Xenova/TinyLlama-1.1B-Chat-v1.0.

import { pipeline } from '@xenova/transformers';

// Create a text-generation pipeline
const generator = await pipeline('text-generation', 'Xenova/TinyLlama-1.1B-Chat-v1.0');

// Define the list of messages
const messages = [
    { "role": "system", "content": "You are a friendly assistant." },
    { "role": "user", "content": "Explain thermodynamics in simple terms." },
]

// Construct the prompt
const prompt = generator.tokenizer.apply_chat_template(messages, {
    tokenize: false, add_generation_prompt: true,
});

// Generate a response
const result = await generator(prompt, {
    max_new_tokens: 256,
    temperature: 0.7,
    do_sample: true,
    top_k: 50,
});
console.log(result);
// [
//   {
//     generated_text: '<|system|>\n' +
//       'You are a friendly assistant.\n' +
//       '<|user|>\n' +
//       'Explain thermodynamics in simple terms.\n' +
//       '<|assistant|>\n' +
//       'Thermodynamics is a branch of physics that deals with the study of heat and its transfer, including the relationship between matter and energy, the concept of chemical equilibrium, and the effects of temperature on chemical and physical processes. In thermodynamics, the properties of matter (such as heat capacity, specific heat, and entropy) are considered and their behavior is studied in relation to the temperature.\n\n' +
//       'Here are some simple steps to explain thermodynamics in simple terms:\n\n' +
//       '1. Energy: Energy is the ability to do work. It is the ability to transfer heat or do other thermodynamic processes. Some common forms of energy are heat, light, electricity, and chemical energy.\n\n' +
//       '2. Heat: Heat is a form of energy that can be transferred from one place to another. It is the ability to induce a change in the temperature of a body or system.\n\n' +
//       '3. Heat capacity: Heat capacity is the amount of heat required to raise the temperature of a system by 1 degree Kelvin (K). It is a measure of the ability of a material to absorb and dissipate thermal energy.\n\n' +
//       '4. Specific heat: Specific heat is the heat required to raise the'
//   }
// ]

---

Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using 🤗 Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).