Progress on v4?
Scene's moving fast. Just wondering if it's gonna happen - my dreams have bamboozled me TWICE that v4 (including v5 and v7) has released lol.
The short answer is yes, I'm still working on it, and I have a few models already that I'm happy with and can upload at any time, but I want to take a little bit more time to optimize. I'll upload something probably within a week or less.
The long answer is that I went back and retested all of the models I was using, looked at some new models, and tried out every merge method available in mergekit. I have a set of models that I know through my experience in merging are good, what merge method I want to use, as well as the order to place them in to get the best results. I've evaluated well over 100 merges since I uploaded V3, and this takes a lot of time with merging, quantizing, and testing. I decided to pace myself so I don't get burnout; otherwise, I would have already uploaded something by now. I just really don't want to repeat my mistake with V2 and V3; yes, they had improvements in certain areas, but I should have just labeled them as experiments instead of iterations.
That's great to hear! Indeed, make sure you don't burn out.
Although, to be fair, v2 and v3 are still great models. V2 has no major issues if you use DRY repetition penalty, as does V3. In my tests I've been constantly bouncing between the two, there's only been small conditional issues (I've not experimented with them, or LLMs in general, in quite a bit).
I suppose it would be possible for you to seperate your next models by giving them a new title, perhaps. Unless, of course, you'd like to make my dreams a reality (v7, lol).
Good luck, looking forward to future releases!