Create README.md

README.md

@@ -0,0 +1 @@
+LoRA adapter based on GradientAI's 1M context Llama-3 8B Instruct finetune. I found that rank 1024 is not sufficient to capture the delta weights in the q_proj and o_proj, so I've created seperate adapters for those modules vs the k-v projection modules.