|
--- |
|
license: mit |
|
license_link: https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/resolve/main/LICENSE |
|
library: llama.cpp |
|
library_link: https://github.com/ggerganov/llama.cpp |
|
base_model: |
|
- microsoft/Phi-3-mini-128k-instruct |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
tags: |
|
- nlp |
|
- code |
|
- gguf |
|
--- |
|
|
|
## Phi-3-Mini-128K-Instruct |
|
|
|
### Model Information |
|
Phi-3-Mini-128K-Instruct is a 3.8 billion-parameter, lightweight, instruction-tuned model from Microsoft, belonging to the Phi-3 family. It has been optimized for long-context comprehension and efficient handling of complex, reasoning-dense tasks. The model supports a context length of up to 128K tokens, making it particularly suitable for scenarios involving extended conversations or long-form content generation. |
|
|
|
- **Name**: Phi-3-Mini-128K-Instruct |
|
- **Parameter Size**: 3.8 billion |
|
- **Model Family**: Phi-3 |
|
- **Architecture**: Transformer with an enhanced focus on efficient context handling. |
|
- **Purpose**: Multilingual dialogue generation, text generation, code completion, and summarization. |
|
- **Training Data**: A combination of synthetic data and filtered, publicly available website data, with an emphasis on reasoning-dense properties. |
|
- **Supported Languages**: English (primary language). |
|
- **Release Date**: September 18, 2024 |
|
- **Context Length**: 128K tokens (other versions include a [4K variant](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)) |
|
- **Knowledge Cutoff**: July 2023 |
|
|
|
### Quantized Model Files |
|
Phi-3 is available in several formats, catering to different computational needs and resource constraints: |
|
|
|
- **ggml-model-q8_0.gguf**: 8-bit quantization, providing robust performance with a file size of 3.8 GB, suitable for resource-constrained environments. |
|
- **ggml-model-f16.gguf**: 16-bit floating-point format, offering enhanced precision at a larger file size of 7.2 GB. |
|
|
|
These formats ensure that the Phi-3 Mini-128K can be adapted to a variety of systems, from low-power devices to high-end servers, making it a versatile option for deployments. |
|
|
|
### Core Library |
|
Phi-3-Mini-128K-Instruct can be deployed using `llama.cpp` or `transformers`, with support for high-efficiency long-context inference. |
|
|
|
- **Primary Framework**: `llama.cpp` |
|
- **Alternate Frameworks**: |
|
- `transformers` for integrations into the Hugging Face ecosystem. |
|
- `vLLM` for efficient inference with optimized memory usage. |
|
|
|
**Library and Model Links**: |
|
- **Model Base**: [microsoft/Phi-3-mini-128k-instruct](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) |
|
- **Resources and Technical Documentation**: |
|
- [Phi-3 Microsoft Blog](https://aka.ms/phi3blog-april) |
|
- [Phi-3 Technical Report](https://aka.ms/phi3-tech-report) |
|
|
|
## Safety and Responsible Use |
|
|
|
The Phi-3-Mini-128K-Instruct is part of the Phi model family, known for its rigorous dataset curation focused on educational and non-toxic sources. Due to its careful design, the Phi-3 series generally avoids generating harmful or biased outputs. This makes it a reliable choice for safety-critical applications and environments where ethical standards are paramount. |
|
|
|
### Training Philosophy |
|
The Phi-3 series models are intentionally trained on textbooks, research papers, and high-quality language corpora, avoiding sources that might introduce harmful, biased, or inappropriate content. As a result, Phi-3 maintains a strong adherence to safe and controlled responses, even when handling sensitive topics or instructions. |
|
|
|
### Risk Profile and Use Recommendations |
|
While no AI model is entirely risk-free, Phi-3's safety features minimize the likelihood of producing unwanted or offensive outputs. However, it is still recommended that users conduct scenario-specific testing to verify its behavior in deployment environments. For additional confidence, consider the following guidelines: |
|
|
|
- **Intended Use**: Education, research, and general-purpose dialogue systems. |
|
- **Deployment**: Suitable for low-risk applications where adherence to ethical and safety guidelines is crucial. |
|
- **Community Testing and Feedback**: Open to user feedback to improve safety benchmarks further and align with best practices. |
|
|
|
For more information on Phi's safety approach, refer to [Phi-3 Technical Report](https://aka.ms/phi3-tech-report). |
|
|
