|
|
|
--- |
|
|
|
base_model: mistralai/Mistral-Nemo-Base-2407 |
|
license: cc-by-nc-4.0 |
|
tags: |
|
- general-purpose |
|
- roleplay |
|
- storywriting |
|
- merge |
|
- finetune |
|
library_name: transformers |
|
|
|
--- |
|
|
|
 |
|
|
|
# QuantFactory/Chronos-Gold-12B-1.0-GGUF |
|
This is quantized version of [elinas/Chronos-Gold-12B-1.0](https://huggingface.co/elinas/Chronos-Gold-12B-1.0) created using llama.cpp |
|
|
|
# Original Model Card |
|
|
|
|
|
# Chronos Gold 12B-1.0 |
|
|
|
 |
|
|
|
Chronos Gold 12B 1.0 is a very unique model that applies to domain areas such as |
|
geneal chatbot functionatliy, *roleplay*, and storywriting. The model has been observed to write up to 2250 tokens in a single sequence. The model was trained at a |
|
sequence length of 16384 (16k) and will still retain the *apparent* 128k context length from Mistral-Nemo. |
|
|
|
The base model is `mistralai/Mistral-Nemo-Base-2407` which was heavily modified to produce a more coherent model, comparable to much larger models. |
|
|
|
**Chronos Gold 12B-1.0** re-creates the uniqueness of the original Chronos with significiantly enhanced prompt adherence (following), coherence, a modern dataset, as well as supporting a majority of "character card" formats in applications like SillyTavern. |
|
|
|
It went through an iterative and objective merge process as my previous models and was further finetuned on a dataset curated for it. |
|
|
|
The specifics of the model will not be disclosed at the time due to dataset ownership. |
|
|
|
## Instruct Template |
|
|
|
This model uses `ChatML` - below is an example. It is a preset in many frontends. |
|
|
|
``` |
|
<|im_start|>system |
|
A system prompt describing how you'd like your bot to act.<|im_end|> |
|
<|im_start|>user |
|
Hello there!<|im_end|> |
|
<|im_start|>assistant |
|
I can assist you or we can disucss other things?<|im_end|> |
|
<|im_start|>user |
|
I was wondering how transformers work?<|im_end|> |
|
<|im_start|>assistant |
|
``` |
|
|
|
## Quantization |
|
|
|
#### LlamaCPP |
|
[GGUFs from @bartowski](https://huggingface.co/bartowski/Chronos-Gold-12B-1.0-GGUF) |
|
|
|
#### Exllama2 |
|
[4.5bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-4.5bpw-h6-exl2) |
|
|
|
[5.5bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-5.5bpw-h6-exl2) |
|
|
|
[6.5bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-6.5bpw-h6-exl2) |
|
|
|
[8.0bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-8.0bpw-h8-exl2) |
|
|
|
#### FP8 |
|
[FP8 Quant by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-FP8) |
|
|
|
## Sampling Settings |
|
Nemo is a bit sensitive to high temperatures, so I use lower. Here are my settings: |
|
``` |
|
Temp - 0.7 (0.9 max) |
|
Presence Penalty - 1.0 |
|
Repetition Penalty range - 2800 |
|
Min P - 0.10 |
|
``` |
|
|
|
## Additional Details |
|
This model was created by **elinas** on discord. |
|
|
|
This is one of multiple models to come out in the series by size and model architecture, so look forward to it! |
|
|
|
Please be mindful of the license. This is strictly non-commercial, but free to use at your own leisure personally. Outputs generated by the model are not reflective of my views. |
|
|
|
Contact me on Discord for inquiries. |
|
|
