state of the kimiko chat format
I've been building a chatbot which I've standardized on an extended version of what I understand to be the Kimiko chat format. My chat format builds upon your <<AIBOT>> syntax and adds a perspective tag. So I can do things like:
<<slimeq/CHAT>> (a publicly visible text message)<<slimeq/SPEAKING>> (an internal, narrative message spoken by a character in my story)<<slimeq/THOUGHT>> (an internal, narrative message thought by a character in my story, which nobody else will see)
To accomplish this I've been building a dataset and training a lora. My first attempt was a fine tune of the original nous-hermes kimiko, and then I moved on to MythoMax and then Tiefighter, both of which immediately understood the Kimiko chat format and were able to adapt effectively.
Now though I'm attempting to move to a mistral base and I've found that this chat format doesn't really work at all. I'm wondering what your plans are @nRuaif , are you abandoning this chat format or planning to train it into a mistral model? It seems like a chatML conversion would be relatively easy, just curious if you've run into this problem yet
I was experimenting with stuff back then with custom format but I get many complains about it so I switched to that Vicuna 1.1.
I will think about ChatML, not really my favorite format but seems like everyone is using it.