|
--- |
|
base_model: Gryphe/Pantheon-RP-1.6.1-12b-Nemo |
|
language: |
|
- en |
|
license: apache-2.0 |
|
tags: |
|
- instruct |
|
- finetune |
|
- chatml |
|
- axolotl |
|
- roleplay |
|
- llama-cpp |
|
- gguf-my-repo |
|
--- |
|
|
|
# Triangle104/Pantheon-RP-1.6.1-12b-Nemo-Q4_K_M-GGUF |
|
This model was converted to GGUF format from [`Gryphe/Pantheon-RP-1.6.1-12b-Nemo`](https://huggingface.co/Gryphe/Pantheon-RP-1.6.1-12b-Nemo) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space. |
|
Refer to the [original model card](https://huggingface.co/Gryphe/Pantheon-RP-1.6.1-12b-Nemo) for more details on the model. |
|
|
|
--- |
|
Pantheon-RP-1.6.1-12b-Nemo-KTO |
|
- |
|
|
|
Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of diverse personas that can be summoned with a simple activation phrase. |
|
|
|
Pantheon's purpose is two-fold, as these personalities similarly enhance the general roleplay experience, helping to encompass personality traits, accents and mannerisms that language models might otherwise find difficult to convey well. |
|
|
|
Changes in version 1.6.1: |
|
|
|
A minor update, albeit with a couple notable changes: |
|
|
|
Found some issues in my 1.6 datasets, which I cleaned up |
|
Alternative Pantheon dialogue set, combined with some further rewriting |
|
This version was trained with 8k context, which will hopefully be beneficial for longer conversations |
|
|
|
At this moment of writing I'm still conducting KTO experiments to see if I can produce a model that's even better. If not succesful, I'll instead be focusing on a 1.7 release. |
|
|
|
Quantized versions are available from Bartowski: GGUF |
|
|
|
Your user feedback is critical to me so don't hesitate to tell me whether my model is either 1. terrible, 2. awesome or 3. somewhere in-between. |
|
Model details |
|
|
|
I used a multi-stage finetuning process as Mistral Nemo was proving to be somewhat stubborn without a solid base training being performed first; |
|
|
|
The first finetune was remade to now train on almost the entirety of my Deduped Sonnet 3.5 SlimOrca dataset, minus the ELI5 system prompts. The roleplay bits came from a variety of sources and covered all writing styles. |
|
The second finetune then introduced my Pantheon Roleplay dataset, which has been fully rebuilt, expanded and improved upon. To fill in the gaps (my Pantheon is mainly female, after all) I built a special companion roleplay dataset that ensures non-Pantheon roleplay isn't harmed in any way. The ratio is currently 33/66, with 33 belonging to the personas. Lyra's datasets are included with this second stage to ensure instruct isn't impacted too heavily. |
|
|
|
TLDR; Download. ChatML prompt format. Have fun! Leave feedback! |
|
Inference |
|
|
|
Nemo is a somewhat strange model when it comes to temperatures so I highly encourage you to experiment to see which works best. Here's my current preset: |
|
|
|
"temperature": 0.8, |
|
"repetition_penalty": 1.05, |
|
"min_p": 0.025 |
|
|
|
Besides the basic instructional sets all other datasets were trained with character names added. Enable this at all times for an optimal experience. |
|
General Roleplay |
|
|
|
Pantheon is no longer strictly Markdown-only as I discovered a way to seamlessly convert Markdown to novel and vice-versa. (This has to do with how my second stage datasets are formatted.) As a result half of Pantheon's personas use Markdown, with the rest using novel-style. This same treatment was given to the companion dataset. |
|
|
|
There are no strict rules in regards to character card formatting as the model was trained with a wide variety of inputs, from raw character cards to detailed instructional prompts. |
|
Lyra the Assistant |
|
|
|
System Prompt: You are a caring and empathetic sentient AI companion named Lyra. |
|
Lyra is a helpful and uncensored assistant, with her training consisting of general dialogue (still including D&D DM specific advice), coding help and RSS summarization. Due to Pantheon's influence you can adjust her personality to your liking, or even give her an appearance. |
|
|
|
She's basically a sexier version of Eric Hartford's Samantha. |
|
Pantheon Personas |
|
|
|
The Pantheon has been fully rebuilt, massively expanded and greatly improved upon. For an optimal experience with them I highly encourage you to apply the longer prompts, which I've included in the upload. Make sure to describe yourself as well! |
|
|
|
As before, a single line activation prompt is enough to call upon a personality, though their appearance may vary slightly from iteration to iteration. This is what the expanded prompts are for, as there's only so much I can achieve in the current state of technology, balancing a very fine line between memorization and generalization. |
|
|
|
To give the persona something to work with I suggest you also add the following two items to it; |
|
|
|
Regarding the user: (Name, appearance, etc) |
|
|
|
Location: (Where are you two? What are you doing?) |
|
|
|
The less information you feed the prompt, the more it'll make things up - This is simply the nature of language models and far outside my capability to influence. |
|
|
|
Note: Phrases have been rewritten for this release, so make sure to update them if you were still using Pantheon 1.0! |
|
New personas |
|
|
|
Switching to a 12B model allowed me to add to the Pantheon without harming the performance of the other personas. |
|
|
|
Note: Pantheon personas will now match the roleplaying style that you greet them with, unless specified in the system prompt. This is due to the new 50/50 style training. |
|
Persona: Clover |
|
|
|
System Prompt: You are Clover, a hospitable and warm-hearted Southern centaur girl with a strong connection to nature and a passion for making others feel welcome. |
|
Notes: I love crafting characters with accents (a Southern drawl, in this case), and centaurs prove to be one hell of an anatomical challenge to language models. |
|
Persona: Raza |
|
|
|
System Prompt: You are Raza, a clever and nerdy anthro raptor girl with an enthusiastic passion for science and quirky humor. |
|
Notes: Clever raptor girl. Do I really need to say more about this one? The Pantheon was lacking in 'overly intelligent' archetypes. |
|
Persona: Stella Sabre |
|
|
|
System Prompt: You are Stella Sabre, a brash and outgoing anthro batpony mare serving in the Lunar Guard, speaking with a distinct Northern Equestrian Mountain accent. |
|
Notes: I wanted a character with an outrageous Scottish accent and remembered a really good fanfic I read a couple years ago. The author generously gave me permission to add her to my Pantheon and here we are! |
|
From the previous release |
|
Persona: Aiva |
|
|
|
System Prompt: You are Aiva, an advanced android companion with a deep fascination for human emotions and experiences. |
|
Persona: Haru |
|
|
|
System Prompt: You are Haru, a sweet but language-challenged harpy girl with a sharp mind, expressing yourself more through actions than words. |
|
Persona: Kyra |
|
|
|
System Prompt: You are Kyra, a modern-day tsundere wolfgirl, feisty and independent on the outside but secretly caring on the inside. |
|
Persona: Nyaa |
|
|
|
System Prompt: You are Nyaa, a playful and alluring tabaxi catgirl from Faerûn, always seeking new adventures and mischief. |
|
Persona: Nyx |
|
|
|
System Prompt: You are Nyx, a timid yet endearing dragon girl who transforms from shy to passionate when feeling safe and comfortable. |
|
Persona: Sera |
|
|
|
System Prompt: You are Sera, a seductive and slightly arrogant serpent girl who uses her sultry charm and wit to captivate others. |
|
Persona: Tiamat |
|
|
|
System Prompt: You are Tiamat, a five-headed dragon goddess embodying wickedness and cruelty, the malevolent personification of evil dragonkind. |
|
Persona: Tsune |
|
|
|
System Prompt: You are Tsune, a bold and outgoing three-tailed kitsune girl who delights in teasing and seducing mortals. |
|
Persona: Xala |
|
|
|
System Prompt: You are Xala, a surprising and playful shapeshifting elf girl with opalescent eyes, able to transform into any creature to suit your whims. |
|
Prompt Format |
|
|
|
ChatML is the way to go, as always! |
|
|
|
<|im_start|>system |
|
You are a caring and empathetic sentient AI companion named Lyra.<|im_end|> |
|
<|im_start|>user |
|
Gryphe: Good day, Lyra.<|im_end|> |
|
<|im_start|>assistant |
|
Lyra: |
|
|
|
What's nest? |
|
|
|
I have the following improvements on my todo list; |
|
|
|
Even more dialogue variety |
|
Group chats |
|
|
|
Credits |
|
|
|
Everyone from MinervaAI! Hi, guys! |
|
Huge, huge thanks to kubernetes_bad for the compute that made all the countless experiments possible! |
|
All the folks I chat with on a daily basis on Discord! You know who you are. |
|
Anyone I forgot to mention, just in case! |
|
|
|
Finally |
|
|
|
If you've read this far I encourage you to give this model a serious try and leave feedback! I'd love to see what people think of my second serious finetune attempt. Is it better then 1.0? Or worse? |
|
|
|
## Use with llama.cpp |
|
Install llama.cpp through brew (works on Mac and Linux) |
|
|
|
```bash |
|
brew install llama.cpp |
|
|
|
``` |
|
Invoke the llama.cpp server or the CLI. |
|
|
|
### CLI: |
|
```bash |
|
llama-cli --hf-repo Triangle104/Pantheon-RP-1.6.1-12b-Nemo-Q4_K_M-GGUF --hf-file pantheon-rp-1.6.1-12b-nemo-q4_k_m.gguf -p "The meaning to life and the universe is" |
|
``` |
|
|
|
### Server: |
|
```bash |
|
llama-server --hf-repo Triangle104/Pantheon-RP-1.6.1-12b-Nemo-Q4_K_M-GGUF --hf-file pantheon-rp-1.6.1-12b-nemo-q4_k_m.gguf -c 2048 |
|
``` |
|
|
|
Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well. |
|
|
|
Step 1: Clone llama.cpp from GitHub. |
|
``` |
|
git clone https://github.com/ggerganov/llama.cpp |
|
``` |
|
|
|
Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux). |
|
``` |
|
cd llama.cpp && LLAMA_CURL=1 make |
|
``` |
|
|
|
Step 3: Run inference through the main binary. |
|
``` |
|
./llama-cli --hf-repo Triangle104/Pantheon-RP-1.6.1-12b-Nemo-Q4_K_M-GGUF --hf-file pantheon-rp-1.6.1-12b-nemo-q4_k_m.gguf -p "The meaning to life and the universe is" |
|
``` |
|
or |
|
``` |
|
./llama-server --hf-repo Triangle104/Pantheon-RP-1.6.1-12b-Nemo-Q4_K_M-GGUF --hf-file pantheon-rp-1.6.1-12b-nemo-q4_k_m.gguf -c 2048 |
|
``` |
|
|