theprint commited on
Commit
f55b78a
1 Parent(s): fe3baa5

Update README.md

Browse files

Copied from the original model,
Incl. missing meta data.

Files changed (1) hide show
  1. README.md +66 -0
README.md CHANGED
@@ -1,3 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  <!-- ### quantize_version: 2 -->
2
  <!-- ### output_tensor_quantised: 1 -->
3
  <!-- ### convert_type: hf -->
 
1
+ ---
2
+ base_model:
3
+ - ajibawa-2023/Code-Llama-3-8B
4
+ - jondurbin/bagel-8b-v1.0
5
+ tags:
6
+ - merge
7
+ - mergekit
8
+ - lazymergekit
9
+ - theprint/Code-Llama-Bagel-8B
10
+ - ajibawa-2023/Code-Llama-3-8B
11
+ - jondurbin/bagel-8b-v1.0
12
+ license: llama3
13
+ ---
14
+
15
+ # Code-Llama-Bagel-8B
16
+
17
+ Code-Llama-Bagel-8B is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
18
+ * [ajibawa-2023/Code-Llama-3-8B](https://huggingface.co/ajibawa-2023/Code-Llama-3-8B)
19
+ * [jondurbin/bagel-8b-v1.0](https://huggingface.co/jondurbin/bagel-8b-v1.0)
20
+
21
+ ## 🧩 Configuration
22
+
23
+ ```yaml
24
+ slices:
25
+ - sources:
26
+ - model: ajibawa-2023/Code-Llama-3-8B
27
+ layer_range: [0, 32]
28
+ - model: jondurbin/bagel-8b-v1.0
29
+ layer_range: [0, 32]
30
+ merge_method: slerp
31
+ base_model: ajibawa-2023/Code-Llama-3-8B
32
+ parameters:
33
+ t:
34
+ - filter: self_attn
35
+ value: [0, 0.5, 0.3, 0.7, 1]
36
+ - filter: mlp
37
+ value: [1, 0.5, 0.7, 0.3, 0]
38
+ - value: 0.5
39
+ dtype: bfloat16
40
+ ```
41
+
42
+ ## 💻 Usage
43
+
44
+ ```python
45
+ !pip install -qU transformers accelerate
46
+
47
+ from transformers import AutoTokenizer
48
+ import transformers
49
+ import torch
50
+
51
+ model = "theprint/Code-Llama-Bagel-8B"
52
+ messages = [{"role": "user", "content": "What is a large language model?"}]
53
+
54
+ tokenizer = AutoTokenizer.from_pretrained(model)
55
+ prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
56
+ pipeline = transformers.pipeline(
57
+ "text-generation",
58
+ model=model,
59
+ torch_dtype=torch.float16,
60
+ device_map="auto",
61
+ )
62
+
63
+ outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
64
+ print(outputs[0]["generated_text"])
65
+ ```
66
+
67
  <!-- ### quantize_version: 2 -->
68
  <!-- ### output_tensor_quantised: 1 -->
69
  <!-- ### convert_type: hf -->