Hugging Face's logo

teleprint-me
/
phi-3-mini-128k-instruct

Text Generation
GGUF
English
nlp
code
Inference Endpoints
conversational
Model card Files Community
1
phi-3-mini-128k-instruct / README.md
aberrio's picture
aberrio
Update README.md
e4263cd verified
preview code
|
raw
history blame contribute delete
4.29 kB
metadata
license: mit
license_link: https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/resolve/main/LICENSE
library: llama.cpp
library_link: https://github.com/ggerganov/llama.cpp
base_model:
  - microsoft/Phi-3-mini-128k-instruct
language:
  - en
pipeline_tag: text-generation
tags:
  - nlp
  - code
  - gguf

Phi-3-Mini-128K-Instruct

Model Information

Phi-3-Mini-128K-Instruct is a 3.8 billion-parameter, lightweight, instruction-tuned model from Microsoft, belonging to the Phi-3 family. It has been optimized for long-context comprehension and efficient handling of complex, reasoning-dense tasks. The model supports a context length of up to 128K tokens, making it particularly suitable for scenarios involving extended conversations or long-form content generation.

  • Name: Phi-3-Mini-128K-Instruct
  • Parameter Size: 3.8 billion
  • Model Family: Phi-3
  • Architecture: Transformer with an enhanced focus on efficient context handling.
  • Purpose: Multilingual dialogue generation, text generation, code completion, and summarization.
  • Training Data: A combination of synthetic data and filtered, publicly available website data, with an emphasis on reasoning-dense properties.
  • Supported Languages: English (primary language).
  • Release Date: September 18, 2024
  • Context Length: 128K tokens (other versions include a 4K variant)
  • Knowledge Cutoff: July 2023

Quantized Model Files

Phi-3 is available in several formats, catering to different computational needs and resource constraints:

  • ggml-model-q8_0.gguf: 8-bit quantization, providing robust performance with a file size of 3.8 GB, suitable for resource-constrained environments.
  • ggml-model-f16.gguf: 16-bit floating-point format, offering enhanced precision at a larger file size of 7.2 GB.

These formats ensure that the Phi-3 Mini-128K can be adapted to a variety of systems, from low-power devices to high-end servers, making it a versatile option for deployments.

Core Library

Phi-3-Mini-128K-Instruct can be deployed using llama.cpp or transformers, with support for high-efficiency long-context inference.

  • Primary Framework: llama.cpp
  • Alternate Frameworks:
    • transformers for integrations into the Hugging Face ecosystem.
    • vLLM for efficient inference with optimized memory usage.

Library and Model Links:

Safety and Responsible Use

The Phi-3-Mini-128K-Instruct is part of the Phi model family, known for its rigorous dataset curation focused on educational and non-toxic sources. Due to its careful design, the Phi-3 series generally avoids generating harmful or biased outputs. This makes it a reliable choice for safety-critical applications and environments where ethical standards are paramount.

Training Philosophy

The Phi-3 series models are intentionally trained on textbooks, research papers, and high-quality language corpora, avoiding sources that might introduce harmful, biased, or inappropriate content. As a result, Phi-3 maintains a strong adherence to safe and controlled responses, even when handling sensitive topics or instructions.

Risk Profile and Use Recommendations

While no AI model is entirely risk-free, Phi-3's safety features minimize the likelihood of producing unwanted or offensive outputs. However, it is still recommended that users conduct scenario-specific testing to verify its behavior in deployment environments. For additional confidence, consider the following guidelines:

  • Intended Use: Education, research, and general-purpose dialogue systems.
  • Deployment: Suitable for low-risk applications where adherence to ethical and safety guidelines is crucial.
  • Community Testing and Feedback: Open to user feedback to improve safety benchmarks further and align with best practices.

For more information on Phi's safety approach, refer to Phi-3 Technical Report.