source code for fine-tuning
Hi Steve,
I hope you're doing great! I've been checking out your work and it's really interesting. I've even tried some of it myself and it worked out pretty well 👍.
Right now, I'm getting more into research in this area and I was hoping you could help me out. Would you be able to share any source code for fine-tuning or maybe some tutorials you have? It would really help me dive deeper.
Thanks a lot for considering my request. Looking forward to hearing from you!
Best,
Hello Radchaneeporn,
I used a tool called axolotl to streamline the fine tuning process. Check it out, https://github.com/OpenAccess-AI-Collective/axolotl
The starter guide I used was this, https://mlabonne.github.io/blog/posts/A_Beginners_Guide_to_LLM_Finetuning.html.
You can find the axolotl configuration file I wrote for mello here, https://github.com/steve-cse/mello/blob/master/mello.yml.
From my personal point of view cleaning/preparing the dataset and setting the right values during fine tuning such as epochs etc.. is more intricate and takes more time.
Hope this helps ⚡
Thank you so much, I appreciate your response :D