metadata
tags:
- npu
- amd
- llama3
- Ryzen AI
This model is meta-llama/Meta-Llama-3.1-8B-Instruct AWQ quantized and converted version to run on the NPU installed Ryzen AI PC, for example, Ryzen 9 7940HS Processor.
For set up Ryzen AI for LLMs in window 11, see Running LLM on AMD NPU Hardware.
The following sample assumes that the setup on the above page has been completed.
This model has only been tested on RyzenAI for Windows 11. It does not work in Linux environments such as WSL.
Sample script will be uploaded tomorrow.