German
Hi Drummer,
You probably haven't thought about this aspect at all, but the Rocinante models 1.0, 1.1 and v2c are unusually good with German. As long as you don't raise the repetition penalty above 1, they produce a pretty solid grammar and phrasing. Even though they seem to be more adept and elegant in English than in German.
Thought you might be interested.
Guten tag und danke! I've seen users set rep pen from 1.07 to 1.13, and I think 1.3 to 2.0 is just too much.
You'd want to try the other anti-repetition samplers like DRY (0.8 mult) or XTC. Kobold supports this out of the box: https://github.com/LostRuins/koboldcpp/releases/tag/v1.74
"RP-Wizard": Haven't heard of this one.
I have an idea why my model might be good at it, but could you confirm if other nemo models aren't as good? How does the german output compare? Are we talking about moist?
yup. I can also confirm that Rocinante is fairly good at german. Subjectively a bit better than a few other nemos I tried.. It can do pretty good translation from en to ger. Can even keep the "Höflichkeitsform" in RP chat, which I thought would be hard for LLMs since that specific language feature does not exist in english, but handles it pretty well. For me personnally GGUF Q5 of Rocinante (1&2) is pretty decent with fully german chat character descriptions. As Huegli said, not quite as good as english, occasionally messes up a word or sneaks in an english one, but definitely usable.
German chat bot descriptions are hard to come by though, if anyone knows a place where they are shared, feel free to post it here. :-)
@Huegli @ElevenGames I can incorporate more German examples in my dataset if you all want. I see around a dozen examples currently and I can boost that to several hundred. Could you tell me what kind of special characters (accented letters, etc) I might encounter in German writing? Also, does Roci print out these German letters properly like ä, ö, ü & ß? If not, are they substituted?
@TheDrummer yes, you are right about the specific letters, I think those are the ones. I can also see all of them already in Roci's output as well. No substitutions, at least in a context with fully german text, they are used correctly. Would be nice if you add more german text to the dataset, I am in favor of it. :-)
@TheDrummer This would be amazing. Those special German letters, you mentioned, cause some problems sometimes, but no big ones. The model tends to substitute those letters with their base letters. I.E. instead of "könnte", it sometimes says "konnte", instead of "Hütte", it may say "Hutte". But even though this happens from time to time, it still uses the correct letters more frequently than the substitutes. The v2c Version is a bit worse then 1.11, but it still is usable.
You wrote about DRY and XTC. Right now I am experimenting with DRY instead of the standard penalty system with good results so far. But I didn't find XTC yet. Could you please hint me on how to use this one in SillyTavern? I need a good amount of repetition penalty, because v2c sometimes tends to repeat the whole last reply. I just was caught in such a repetition loop, and DRY helped me out without interfering with the German language.
@ElevenGames In my experience, rocinante doesn't care about the language of the cards as much as you would expect. I once just wrote: "#instruction: Schreibe von nun an in Deutsch" (From now on write in German) in the middle of a story, and this was it. All German from this moment on. Another approach would be to simply translate just the starting dialog (and maybe the sample texts) of the card into German, and rocinante will follow up accordingly. You could even ask rocinante to do this for you and take it's translation. This is done within one minute.
@TheDrummer Oh,and for the merge. I was referring to this one: https://huggingface.co/DavidAU/MN-Rocinante-18.5B-v1.1-Story-Wizard-ED1-Instruct-GGUF. But it simply feels way less capable than the original model. It's not a bad model. But it feels... I would say: Less creative, and it doesn't follow subtle hints as well as the original one. You have brewed something very special with Rocinante! It's far from being perfect. But it's so much fun to play around with it, in all three versions so far. I also tried out your Theia, which is also an awesome model, But I quickly came back to Rocinante, which is my favorite for now. I just don't know, which version I prefer.
@Huegli yes, I agree, that the language switch is easily possible like that. And the translation, of course, works quite well, except if the text contains character tags or the typical personality description structures. That can cause issues, because the LLM if trained for RP, tends to adopt that as a character and "speaks for them" instead of translating. I have a function that replaces some of the typical tags, translates and then puts them back in. :D I am only doing these things, because what i've found is that the letter substitution or english terms almost disappear, if you really give it a fully german context (including instruct prompt, everything).