Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,70 @@
+---
+base_model:
+- berkeley-nest/Starling-LM-7B-alpha
+- NousResearch/Nous-Hermes-2-Mistral-7B-DPO
+- senseable/WestLake-7B-v2
+- openchat/openchat-3.5-0106
+library_name: transformers
+tags:
+- mergekit
+- merge
+
+---
+# merged
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+## Merge Details
+### Merge Method
+
+This model was merged using the task_swapping merge method using [senseable/WestLake-7B-v2](https://huggingface.co/senseable/WestLake-7B-v2) as a base.
+
+### Models Merged
+
+The following models were included in the merge:
+* [berkeley-nest/Starling-LM-7B-alpha](https://huggingface.co/berkeley-nest/Starling-LM-7B-alpha)
+* [NousResearch/Nous-Hermes-2-Mistral-7B-DPO](https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO)
+* [openchat/openchat-3.5-0106](https://huggingface.co/openchat/openchat-3.5-0106)
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+base_model:
+  model:
+    path: senseable/WestLake-7B-v2
+dtype: bfloat16
+merge_method: task_swapping
+slices:
+- sources:
+  - layer_range: [0, 32]
+    model:
+      model:
+        path: berkeley-nest/Starling-LM-7B-alpha
+    parameters:
+      diagonal_offset: 2.0
+      weight: 0.72
+  - layer_range: [0, 32]
+    model:
+      model:
+        path: openchat/openchat-3.5-0106
+    parameters:
+      diagonal_offset: 4.0
+      random_mask: 0.166
+      random_mask_seed: 19519.0
+      weight: 0.4
+  - layer_range: [0, 32]
+    model:
+      model:
+        path: NousResearch/Nous-Hermes-2-Mistral-7B-DPO
+    parameters:
+      diagonal_offset: 4.0
+      random_mask: 0.125
+      random_mask_seed: 990090.0
+      weight: 0.666
+  - layer_range: [0, 32]
+    model:
+      model:
+        path: senseable/WestLake-7B-v2
+```