outstanding
this is almost Noromaid level in RP , more tuning and maybe a 6x8b MOE and it should be real close to the 70b RP models , great work
would more experts really be necessary / make it better if it's just tuned for RP?
would more experts really be necessary / make it better if it's just tuned for RP?
i would think so , i mean claude 3 opus is great all around model and great at RP so my logic here is the model needs to be good in general and specialise in RP lets say 3 experts for general purpose and 3 just made for RP , but thats just what i think would work @JayhC
I'd like to finetune it but i have no idea where people get their 50M+ high quality RP datasets, tried to make my own out of my best RP sessions, but i only got a file which size is around one megabyte D:
Of course there is limarp and other public RP/ERP datasets, but i feel like using most of them would make the model more dumb/predictable instead.
This is very good, quite smart and very flexible. Excellent start with llama3!