Undi95 commited on
Commit
66acb1f
1 Parent(s): 06f2047

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -33
README.md CHANGED
@@ -10,41 +10,13 @@ tags:
10
  - merge
11
 
12
  ---
13
- # out
14
 
15
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
 
17
- ## Merge Details
18
- ### Merge Method
19
 
20
- This model was merged using the della merge method using [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) as a base.
21
 
22
- ### Models Merged
23
-
24
- The following models were included in the merge:
25
- * [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B)
26
- * [Undi95/LocalC-12B-e2.0](https://huggingface.co/Undi95/LocalC-12B-e2.0)
27
- * [intervitens/mini-magnum-12b-v1.1](https://huggingface.co/intervitens/mini-magnum-12b-v1.1)
28
-
29
- ### Configuration
30
-
31
- The following YAML configuration was used to produce this model:
32
-
33
- ```yaml
34
- base_model: mistralai/Mistral-Nemo-Instruct-2407
35
- merge_method: della
36
- dtype: bfloat16
37
- models:
38
- - model: intervitens/mini-magnum-12b-v1.1
39
- parameters:
40
- weight: 1.0
41
- - model: Undi95/LocalC-12B-e2.0
42
- parameters:
43
- weight: 1.0
44
- - model: NeverSleep/Lumimaid-v0.2-12B
45
- parameters:
46
- weight: 1.0
47
- - model: mistralai/Mistral-Nemo-Instruct-2407
48
- parameters:
49
- weight: 1.0
50
  ```
 
 
 
10
  - merge
11
 
12
  ---
 
13
 
14
+ Merge of Lumimaid and Magnum as requested by somes.
15
 
16
+ I used the new DELLA merge method in mergekit and added a finetune of Nemo only on Claude input, trained on 16k ctx, in the mix.
 
17
 
18
+ # Prompt template: Mistral
19
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  ```
21
+ <s>[INST] {input} [/INST] {output}</s>
22
+ ```