Update README.md
Browse files
README.md
CHANGED
@@ -11,6 +11,18 @@ tags:
|
|
11 |
|
12 |
I like to call this model "The little professor". It is simply a MOE merge of lora merged models across Llama2 and Mistral. I am using this as a test case to move to larger models and get my gate discrimination set correctly. This model is best suited for knowledge related use cases, I did not give it a specific workload target as I did with some of the other models in the "Owl Series".
|
13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
# Prompting
|
15 |
|
16 |
## Prompt Template for alpaca style
|
|
|
11 |
|
12 |
I like to call this model "The little professor". It is simply a MOE merge of lora merged models across Llama2 and Mistral. I am using this as a test case to move to larger models and get my gate discrimination set correctly. This model is best suited for knowledge related use cases, I did not give it a specific workload target as I did with some of the other models in the "Owl Series".
|
13 |
|
14 |
+
This model is merged from the following sources:
|
15 |
+
|
16 |
+
[Westlake-7B](https://huggingface.co/senseable/Westlake-7B)
|
17 |
+
[WestLake-7B-v2](https://huggingface.co/senseable/WestLake-7B-v2)
|
18 |
+
[openchat-nectar-0.5](https://huggingface.co/andysalerno/openchat-nectar-0.5)
|
19 |
+
[WestSeverus-7B-DPO-v2](https://huggingface.co/FelixChao/WestSeverus-7B-DPO-v2)
|
20 |
+
[WestSeverus-7B-DPO](https://huggingface.co/PetroGPT/WestSeverus-7B-DPO)
|
21 |
+
|
22 |
+
Unless those models are "contaminated" this one is not. This is a proof of concept version of this series and you can find others where I am tuning my own models and using moe mergekit to combine them to make moe models that I can run on lower tier hardware with better results.
|
23 |
+
|
24 |
+
The goal here is to create specialized models that can collaborate and run as one model.
|
25 |
+
|
26 |
# Prompting
|
27 |
|
28 |
## Prompt Template for alpaca style
|