Quantization made by Richard Erkhov.

Qwen2.5-Math-72B - GGUF

Model creator: https://huggingface.co/Qwen/
Original model: https://huggingface.co/Qwen/Qwen2.5-Math-72B/

Name	Quant method	Size
Qwen2.5-Math-72B.Q2_K.gguf	Q2_K	27.76GB
Qwen2.5-Math-72B.IQ3_XS.gguf	IQ3_XS	30.59GB
Qwen2.5-Math-72B.IQ3_S.gguf	IQ3_S	32.12GB
Qwen2.5-Math-72B.Q3_K_S.gguf	Q3_K_S	32.12GB
Qwen2.5-Math-72B.IQ3_M.gguf	IQ3_M	33.07GB
Qwen2.5-Math-72B.Q3_K.gguf	Q3_K	35.11GB
Qwen2.5-Math-72B.Q3_K_M.gguf	Q3_K_M	35.11GB
Qwen2.5-Math-72B.Q3_K_L.gguf	Q3_K_L	36.79GB
Qwen2.5-Math-72B.IQ4_XS.gguf	IQ4_XS	37.4GB
Qwen2.5-Math-72B.Q4_0.gguf	Q4_0	38.4GB
Qwen2.5-Math-72B.IQ4_NL.gguf	IQ4_NL	38.9GB
Qwen2.5-Math-72B.Q4_K_S.gguf	Q4_K_S	40.88GB
Qwen2.5-Math-72B.Q4_K.gguf	Q4_K	44.16GB
Qwen2.5-Math-72B.Q4_K_M.gguf	Q4_K_M	44.16GB
Qwen2.5-Math-72B.Q4_1.gguf	Q4_1	42.56GB
Qwen2.5-Math-72B.Q5_0.gguf	Q5_0	46.72GB
Qwen2.5-Math-72B.Q5_K_S.gguf	Q5_K_S	47.85GB
Qwen2.5-Math-72B.Q5_K.gguf	Q5_K	50.71GB
Qwen2.5-Math-72B.Q5_K_M.gguf	Q5_K_M	50.71GB
Qwen2.5-Math-72B.Q5_1.gguf	Q5_1	50.88GB
Qwen2.5-Math-72B.Q6_K.gguf	Q6_K	59.93GB
Qwen2.5-Math-72B.Q8_0.gguf	Q8_0	71.96GB

Original model description:

license: other license_name: qwen base_model: Qwen/Qwen2.5-72B license_link: https://huggingface.co/Qwen/Qwen2.5-Math-72B/blob/main/LICENSE language: - en pipeline_tag: text-generation library_name: transformers

Qwen2.5-Math-72B

🚨 Qwen2.5-Math mainly supports solving English and Chinese math problems through CoT and TIR. We do not recommend using this series of models for other tasks.

Introduction

In August 2024, we released the first series of mathematical LLMs - Qwen2-Math - of our Qwen family. A month later, we have upgraded it and open-sourced Qwen2.5-Math series, including base models Qwen2.5-Math-1.5B/7B/72B, instruction-tuned models Qwen2.5-Math-1.5B/7B/72B-Instruct, and mathematical reward model Qwen2.5-Math-RM-72B.

Unlike Qwen2-Math series which only supports using Chain-of-Thught (CoT) to solve English math problems, Qwen2.5-Math series is expanded to support using both CoT and Tool-integrated Reasoning (TIR) to solve math problems in both Chinese and English. The Qwen2.5-Math series models have achieved significant performance improvements compared to the Qwen2-Math series models on the Chinese and English mathematics benchmarks with CoT.

While CoT plays a vital role in enhancing the reasoning capabilities of LLMs, it faces challenges in achieving computational accuracy and handling complex mathematical or algorithmic reasoning tasks, such as finding the roots of a quadratic equation or computing the eigenvalues of a matrix. TIR can further improve the model's proficiency in precise computation, symbolic manipulation, and algorithmic manipulation. Qwen2.5-Math-1.5B/7B/72B-Instruct achieve 79.7, 85.3, and 87.8 respectively on the MATH benchmark using TIR.

Model Details

For more details, please refer to our blog post and GitHub repo.

Requirements

transformers>=4.37.0 for Qwen2.5-Math models. The latest version is recommended.

🚨 This is a must because transformers integrated Qwen2 codes since 4.37.0.

For requirements on GPU memory and the respective throughput, see similar results of Qwen2 here.

Quick Start

Qwen2.5-Math-72B-Instruct is an instruction model for chatting;

Qwen2.5-Math-72B is a base model typically used for completion and few-shot inference, serving as a better starting point for fine-tuning.

Citation

If you find our work helpful, feel free to give us a citation.

@article{yang2024qwen25mathtechnicalreportmathematical,
  title={Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement}, 
  author={An Yang and Beichen Zhang and Binyuan Hui and Bofei Gao and Bowen Yu and Chengpeng Li and Dayiheng Liu and Jianhong Tu and Jingren Zhou and Junyang Lin and Keming Lu and Mingfeng Xue and Runji Lin and Tianyu Liu and Xingzhang Ren and Zhenru Zhang},
  journal={arXiv preprint arXiv:2409.12122},
  year={2024}
}