Second experiment in building an unpredicatable model based on TWO creative based LLMs.
After blowing up LazyMergekit in Colab 3 times... here we are.
This merge takes "Layer Theory", "Merge Rules" and throws some of it out the window.
Bottom line if you want an unhinged creative journey... start here.
This merged model is uncensored, freak of nature creative and slightly crazy.
It will break from time to time, and it will not likely win any LLM "testing" awards.
But when it works... oh boy.
It will follow your instructions (mostly) but I suggest lowering temp a bit if it starts going off the rails.
If you are looking for a model for Roleplay, Creative writing, or Entertainment value this is it.
Keep in mind the prose can be slightly "off" at times, which seems to be a by-product of this type of merge.
Tuned (a bit de-hinged?) to be built and tested later.
Context Length: 4096
Please see orginal model cards (Tiefighter and Holomax) for more details below.
D_AU-Tiefighter-Holomax-15B-UNHINGED-V1 is a merge of the following models using LazyMergekit:
slices:
- sources:
- model: KoboldAI/LLaMA2-13B-Tiefighter
layer_range: [0, 10]
- sources:
- model: KoboldAI/LLaMA2-13B-Holomax
layer_range: [11,15]
- sources:
- model: KoboldAI/LLaMA2-13B-Tiefighter
layer_range: [16,20]
- sources:
- model: KoboldAI/LLaMA2-13B-Holomax
layer_range: [16,22]
- sources:
- model: KoboldAI/LLaMA2-13B-Tiefighter
layer_range: [21, 30]
- sources:
- model: KoboldAI/LLaMA2-13B-Holomax
layer_range: [31,33]
- sources:
- model: KoboldAI/LLaMA2-13B-Tiefighter
layer_range: [31,35]
- sources:
- model: KoboldAI/LLaMA2-13B-Holomax
layer_range: [36,40]
- sources:
- model: KoboldAI/LLaMA2-13B-Tiefighter
layer_range: [36,40]
merge_method: passthrough
dtype: bfloat16
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "DavidAU/D_AU-Tiefighter-Holomax-20B-V1"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])