Further Fine-Tune ddh0/Mistral-10.7B-Instruct-v0.2 on IKM to Make Nexus-IKM-Mistral-Instruct-v0.2-10.7B
Severian, ddh0 made a Depth UpScaled version of Mistral-7B-Instruct-v0.2 (ddh0/Mistral-10.7B-Instruct-v0.2). It just needs to be further fine-tuned - perhaps on your IKM dataset? If you're interested, I hope you'll further fine-tune it as well and see what it can do. It would be awesome to see it outperform your 7B version. Cheers! ddh0/Mistral-10.7B-Instruct-v0.2
Ps... I know this format and wording is very similar to my other post. Cheers! 😬
You got it! Love the different versions and new methods we have. I think one powerful thing to come of training all these different models on the IKM dataset will be to see how it affects each one, and whether there is validity to what I'm doing!
I agree! And, it's going to be a blast testing the models and seeing if your hypothesis gets proven. 😋
Severian, ddh0/Mistral-10.7B-Instruct-v0.2 is being remade. The model is going to be Depth Up-Scaled just like Joseph717171/Mistral-10.7B-v0.2, which follows the paper: SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling and the very well written explanation on Depth Up-Scaling by Rohan Paul on X. I apologize for the inconvenience. ddh0/Mistral-10.7B-Instruct-v0.2 was merged like froggeric/WestLake-10.7B-v2, which still seems like a good model - it just doesn't follow the paper, and is therefore not considered the same. 🤔😬
ddh0/Mistral-10.7B-Instruct-v0.2 has been fixed and has been re-uploaded, and is ready to go. 🎉😁