@yanghaojin on Hugging Face: "Full parameter fine-tuning of the LLaMA-3 8B model using a single GTX 3090 GPU…"

Hugging Face

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Back to feed

yanghaojin

posted an update May 2

Post

2088

Full parameter fine-tuning of the LLaMA-3 8B model using a single GTX 3090 GPU with 24GB of graphics memory?

Please check out our tool for fine-tuning, inferencing, and evaluating GreenBitAI's low-bit LLMs:
https://github.com/GreenBitAI/green-bit-llm
Model Zoo:
https://huggingface.co/GreenBitAI

yanghaojin

May 2

Command for reproducing this run 😉 :
CUDA_VISIBLE_DEVICES=0 WANDB_DISABLED=true python -m sft.finetune --model GreenBitAI/Llama-3-8B-layer-mix-bpw-2.2 --tune-qweight-only --galore --galore-rank 64 --optimizer adamw8bit --batch-size 1 --seqlen 96

jqodiriy

May 10

How you prepare dataset for finetuning llama3?
Could you show the structure of your dataset and how you fine-tune using that dataset?

jqodiriy

May 10

How you prepare dataset for finetuning llama3?
Could you show the structure of your dataset and how you fine-tune using that dataset?

In this post