Hugging Face's logo

schnapper79
/
lumikabra-123B_v0.2

Text Generation
Transformers
Safetensors
mistral
mergekit
lumikabra-123B
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
schnapper79 commited on
Commit
d471b06
•
1 Parent(s): c36957a

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +50 -0
  2. config.json +27 -0
  3. mergekit_config.yml +20 -0
  4. model-00001-of-00051.safetensors +3 -0
  5. model-00002-of-00051.safetensors +3 -0
  6. model-00003-of-00051.safetensors +3 -0
  7. model-00004-of-00051.safetensors +3 -0
  8. model-00005-of-00051.safetensors +3 -0
  9. model-00006-of-00051.safetensors +3 -0
  10. model-00007-of-00051.safetensors +3 -0
  11. model-00008-of-00051.safetensors +3 -0
  12. model-00009-of-00051.safetensors +3 -0
  13. model-00010-of-00051.safetensors +3 -0
  14. model-00011-of-00051.safetensors +3 -0
  15. model-00012-of-00051.safetensors +3 -0
  16. model-00013-of-00051.safetensors +3 -0
  17. model-00014-of-00051.safetensors +3 -0
  18. model-00015-of-00051.safetensors +3 -0
  19. model-00016-of-00051.safetensors +3 -0
  20. model-00017-of-00051.safetensors +3 -0
  21. model-00018-of-00051.safetensors +3 -0
  22. model-00019-of-00051.safetensors +3 -0
  23. model-00020-of-00051.safetensors +3 -0
  24. model-00021-of-00051.safetensors +3 -0
  25. model-00022-of-00051.safetensors +3 -0
  26. model-00023-of-00051.safetensors +3 -0
  27. model-00024-of-00051.safetensors +3 -0
  28. model-00025-of-00051.safetensors +3 -0
  29. model-00026-of-00051.safetensors +3 -0
  30. model-00027-of-00051.safetensors +3 -0
  31. model-00028-of-00051.safetensors +3 -0
  32. model-00029-of-00051.safetensors +3 -0
  33. model-00030-of-00051.safetensors +3 -0
  34. model-00031-of-00051.safetensors +3 -0
  35. model-00032-of-00051.safetensors +3 -0
  36. model-00033-of-00051.safetensors +3 -0
  37. model-00034-of-00051.safetensors +3 -0
  38. model-00035-of-00051.safetensors +3 -0
  39. model-00036-of-00051.safetensors +3 -0
  40. model-00037-of-00051.safetensors +3 -0
  41. model-00038-of-00051.safetensors +3 -0
  42. model-00039-of-00051.safetensors +3 -0
  43. model-00040-of-00051.safetensors +3 -0
  44. model-00041-of-00051.safetensors +3 -0
  45. model-00042-of-00051.safetensors +3 -0
  46. model-00043-of-00051.safetensors +3 -0
  47. model-00044-of-00051.safetensors +3 -0
  48. model-00045-of-00051.safetensors +3 -0
  49. model-00046-of-00051.safetensors +3 -0
  50. model-00047-of-00051.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # lumikabra_v0.2
10
+
11
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
+
13
+ ## Merge Details
14
+ ### Merge Method
15
+
16
+ This model was merged using the della_linear merge method using /workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407 as a base.
17
+
18
+ ### Models Merged
19
+
20
+ The following models were included in the merge:
21
+ * /workspace/text-generation-webui/models/anthracite-org_magnum-v2-123b
22
+ * /workspace/text-generation-webui/models/FluffyKaeloky_Luminum-v0.1-123B
23
+ * /workspace/text-generation-webui/models/migtissera_Tess-3-Mistral-Large-2-123B
24
+
25
+ ### Configuration
26
+
27
+ The following YAML configuration was used to produce this model:
28
+
29
+ ```yaml
30
+ models:
31
+ - model: /workspace/text-generation-webui/models/anthracite-org_magnum-v2-123b
32
+ parameters:
33
+ weight: 0.24
34
+ density: 0.5
35
+ - model: /workspace/text-generation-webui/models/FluffyKaeloky_Luminum-v0.1-123B
36
+ parameters:
37
+ weight: 0.34
38
+ density: 0.8
39
+ - model: /workspace/text-generation-webui/models/migtissera_Tess-3-Mistral-Large-2-123B
40
+ parameters:
41
+ weight: 0.24
42
+ density: 0.9
43
+ merge_method: della_linear
44
+ base_model: /workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407
45
+ parameters:
46
+ epsilon: 0.05
47
+ lambda: 1
48
+ int8_mask: true
49
+ dtype: bfloat16
50
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "/workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 88,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "bfloat16",
24
+ "transformers_version": "4.44.2",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ models:
2
+ - model: /workspace/text-generation-webui/models/anthracite-org_magnum-v2-123b
3
+ parameters:
4
+ weight: 0.24
5
+ density: 0.5
6
+ - model: /workspace/text-generation-webui/models/FluffyKaeloky_Luminum-v0.1-123B
7
+ parameters:
8
+ weight: 0.34
9
+ density: 0.8
10
+ - model: /workspace/text-generation-webui/models/migtissera_Tess-3-Mistral-Large-2-123B
11
+ parameters:
12
+ weight: 0.24
13
+ density: 0.9
14
+ merge_method: della_linear
15
+ base_model: /workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407
16
+ parameters:
17
+ epsilon: 0.05
18
+ lambda: 1
19
+ int8_mask: true
20
+ dtype: bfloat16
model-00001-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:717b699733f103c255516a028c1ac9cad0a1a5feca2c99463b794a160f2b1b39
3
+ size 4378928504
model-00002-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:19f6bed32952489043e3f8f0b5e61dd30154b26eb1eefb64e2147040ad50a992
3
+ size 4907411088
model-00003-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99aa318100ffab74deb323ab6acb906df3483348751e2d0b76806c9446a88815
3
+ size 4806747904
model-00004-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:682f586d0e0360539133a9a9e38c5eda46ce32ecc55ac1f5caac10e20c410cbf
3
+ size 4831938544
model-00005-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6b587316ef6d7793b45c43bc0d5d80b6fafc63932960229d5abde9752e4f171
3
+ size 4831938552
model-00006-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bf10dbd07b79589f7af9f9cdee98c9e8433ff6f384acee0d1fd6e55b95708ea1
3
+ size 4907411096
model-00007-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6820a8504ebe3a1791d15465c995044c77d068c6feabe1c78889a4920e3b46e9
3
+ size 4806747904
model-00008-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d6922c4ee9b15b1c012f4424e60a51d5f0835c52635221c256e27a0a8ffeda4
3
+ size 4831938536
model-00009-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b10669fc5f06bf87fd99b38345a9b03af705623b9ac89651de04dd63e6e6aadf
3
+ size 4831938552
model-00010-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f095baf2b51167cfa8df768598df4745baaa69276e2768436bfbbd2706fc011c
3
+ size 4907411096
model-00011-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4c6fe9a42b7b3ac3d3813296987472bf14927f2c722483ef3d2a3ac2a8081c6
3
+ size 4806747904
model-00012-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:396c19fd2d98d30673dc56778b3d32be64f5a3054a97153502ca09ad41620f24
3
+ size 4831938544
model-00013-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:896b88368c4dbf3881796e60827726760dba5c3bdb4f6b33c1760fd090cad168
3
+ size 4831938552
model-00014-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1041aad77d6b3004879d2d2e7191198b0735ff0a6c1e828eb49d086c7d9891a
3
+ size 4907411088
model-00015-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b8e73a47806f8b95985cce191eb9f6b6c8b0b1aba66d5649ce2fde2db5ea90b3
3
+ size 4806747904
model-00016-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b8a1e4bfdd03fd89722d38dc57cee2daac6d57b63b8877a144b54195c4483f7d
3
+ size 4831938544
model-00017-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d54810a98cb2e6703fb4e86ec0a5a7ad60a44026507783a8ab16dc2bd5197a2
3
+ size 4831938552
model-00018-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8288468e6de1361cbfa0ff7f590e34139fa465bccb27ec4158c1d4a2e0be0d5
3
+ size 4907411096
model-00019-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c008e21f5df5f1db64122f492edf57a004d0c4d05ca70f8f27e8bc40f1ff5d63
3
+ size 4806747904
model-00020-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12494b46644dac0f64507eae739023ce28bf7afbccc0326a85e74d86053360e0
3
+ size 4831938544
model-00021-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c5aef27634e33603db92a976ccab46a8988947e3f0205f1a1a282cd8a6d0ee1
3
+ size 4831938544
model-00022-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d238574936f696abe11b644f34ac1b9157cfb0387b7693c6a72fd593ba00d6c
3
+ size 4907411096
model-00023-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d772d634bd7e0cf34144fcc6304d48c161e0e2fc4c64986b6ab2f236587db122
3
+ size 4806747904
model-00024-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e60fb1e7ef90053e6b14fb8a4409a4cb615f3afd4ca7c9f4c22db43f4b7d588
3
+ size 4831938544
model-00025-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eaf9196f123753c4659b5669242acab46b02f0819f16d96e83e97ef3cd31b9ad
3
+ size 4831938552
model-00026-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ecfad0a5ff2faac39df87ca268deac405940785124539029dd08111c0b6bc151
3
+ size 4907411096
model-00027-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c615747e21565121bc9f4b8139fe11dc881eaedb918aa03c1364c01371aae92e
3
+ size 4806747896
model-00028-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5902088b5706ad5f3cbdb6456150c4f8f4f81d6d92ea20a83f55566b21cdf126
3
+ size 4831938544
model-00029-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7bf140d6881a1e055604a42be29e96205be68048b5c482d7204ddf3c61a80a7
3
+ size 4831938552
model-00030-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9853b6b80a0f233ff663a5d534a836c7b41350795f8ec1a38e991bdbdc41a5a7
3
+ size 4907411096
model-00031-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3b587890be993d85c9908225d6d14c9b06e44057e01fb1a12a37f9fe031fd598
3
+ size 4806747904
model-00032-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b52e7ed39a348b89c86dd356c3e65cc8965d9a66e4e1ead76423aeedc222734
3
+ size 4831938544
model-00033-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4cdee11ee83de352d9bf4088c3ba370f0f3e5acb7ce14d894358cde1ace7c73b
3
+ size 4831938544
model-00034-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3dd0c20e5fc390521b5d4b594eb4e7913fe49e24b13eea31840b619c3b8add6e
3
+ size 4907411096
model-00035-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13dffe0a8151e4b74b40817a3d810a02b49dbbd8c1f8b5cfa8dde1dd070017a6
3
+ size 4806747904
model-00036-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4a6768d63f160b8953d3bbd5d8b907bde6712a6b5d4b0fc0884bd27584e2c99
3
+ size 4831938544
model-00037-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0de3201e8a643f98668ad2d6e61d58d47e53ea905288e8cca27460981ed73e7c
3
+ size 4831938552
model-00038-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:483f053670fd640db3aa6a3e701be1eabbf495563356247d986a475aa3e3383f
3
+ size 4907411096
model-00039-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d4e0c79c9fc91371b80109794f3101febd4cc26266311de78a44089e12c846c
3
+ size 4806747904
model-00040-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b368dc19648a6ab44d02e7ae1cd15351a102e6eadd51dfd0cf8cbcee4567c37b
3
+ size 4831938544
model-00041-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a37694fc452e8a1f2f705f6cfd5c72a8e17679fc1ba0e1dd874ff954fe439f48
3
+ size 4831938552
model-00042-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fee6cc124a82b45232a9f60e66a09879a23469c57f2c0932f0b5b90c93da7220
3
+ size 4907411096
model-00043-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f5abb98225c676e74adcafcc3d4a59a9721c15ee03de68348846488c7a9f7eb
3
+ size 4806747904
model-00044-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42eea501572482f6fd33156cbf020711268ae5d14806d7d2657a15d56949c3e6
3
+ size 4831938544
model-00045-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a25119209ef8ab17522c7343bb4de8685f39c7fc79f410f697ee9dd72f0b1cf6
3
+ size 4831938552
model-00046-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a7bc248187811fad9246d675471fb657be2a808971df4a68e0b19f74d1de480b
3
+ size 4907411088
model-00047-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78472f91a358aabba45a4d44de1fe3ee97a626717886c9d4e20fea5ddd084ba6
3
+ size 4806747904