File size: 2,959 Bytes
f8a0f5a 2abad12 f8a0f5a 8812c0a f8a0f5a 928ff38 f8a0f5a eb9805e f8a0f5a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
---
license: apache-2.0
tags:
- moe
- frankenmoe
- merge
- mergekit
- lazymergekit
- aloobun/Cypher-Mini-Laser-1.8B
- aloobun/Cypher-CoT-Laser-1.8B
- laser
base_model:
- aloobun/Cypher-Mini-Laser-1.8B
- aloobun/Cypher-CoT-Laser-1.8B
datasets:
- kaist-ai/CoT-Collection
- Locutusque/Hercules-v3.0
---
This work is based on implementation of the LASER technique, by @fernandofernandes & @ehartford.
I 'lasered' two models first and then merged them.
# Cypher-Laser-Mixtral-2x1.8B-v0.1
Cypher-Laser-Mixtral-2x1.8B-v0.1 is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [aloobun/Cypher-Mini-Laser-1.8B](https://huggingface.co/aloobun/Cypher-Mini-Laser-1.8B)
* [aloobun/Cypher-CoT-Laser-1.8B](https://huggingface.co/aloobun/Cypher-CoT-Laser-1.8B)
## 🧩 Configuration
```yaml
base_model: aloobun/Cypher-Mini-Laser-1.8B
gate_mode: hidden
dtype: bfloat16
experts:
- source_model: aloobun/Cypher-Mini-Laser-1.8B
positive_prompts:
- "Write a Python script that sorts a list of integers using the bubble sort algorithm."
- "Write a JavaScript function that redirects a web page to another page after 5 seconds."
- "Describe the steps to troubleshoot a fluid dynamics issue with a water fountain."
- "Write a short story about a knight's quest to find a lost treasure, and then summarize it in one paragraph."
- "Summarize the following article with details and clarity."
- "Tell me about your favorite book and why you like it."
- source_model: aloobun/Cypher-CoT-Laser-1.8B
positive_prompts:
- "Liam saw an animal running on the farm. Q: Is it true that The animal could be a horse."
- "Based on the following paragraph can we conclude that the sentence below is true?"
- "According to the article, how do dolphins communicate with each other?"
- "Solve this math problem Solve 7644 = 4648*d - 4557*d for d."
- "If we have 3 marbles, and two roll under the counter, and one is found, how many marbles are there?"
- "What is the result of 25 divided by 5?"
- "Is it morally justifiable to lie to protect someone's feelings?"
- "Determine if the sentence is true based on the text below. Choose from options."
- "What might a person do if they forget their umbrella on a rainy day?"
- "Which of the following is an example of renewable energy: a) Coal, b) Solar, c) Oil, d) Natural gas?"
- "What is the capital of Canada? a) Toronto, b) Ottawa, c) Montreal, d) Vancouver."
- "Which of these animals is a mammal? a) Snake, b) Dolphin, c) Turtle, d) Frog."
- "Given a story, answer the question about the story."
- "Given a prompt and four completions, select the completion that is the most plausible in continuing or answering the prompt."
```
## Format:
```
<|system|></s><|prompt|></s><|answer|>
``` |