|
--- |
|
language: |
|
- ko |
|
- en |
|
license: mit |
|
--- |
|
|
|
# Model Card for free-evo-qwen72b-v0.8 |
|
|
|
## Developed by : [Freewheelin](https://freewheelin-recruit.oopy.io/) AI Technical Team |
|
|
|
## 1st place : 2024 4th May - avg. 81.28 [Open Llm Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) |
|
but this kicked away. maybe the explanation was not enough. |
|
|
|
## Method |
|
- We were inspired by this [Sakana project](https://sakana.ai/evolutionary-model-merge/) |
|
|
|
## Process |
|
- you need two models with the same architecture |
|
- 1. choose one model and finetune the model to make a gap between the original one and fine-tuned one. it doesn't matter the evaluation score is higher or lower. |
|
- 2. merge two of them |
|
- 3. evaluate the merged model |
|
- 4. finetune a specific evaluation part if you need to increase score of the part of the model. (sure it's not gonna work like you think. but try it) |
|
- 5. merge again |
|
- 6. evaluate again |
|
- 7. keep going until evaluate avg is higher then original one |
|
|
|
that's it. simple. |
|
|
|
## Base Architecture |
|
- QWEN2 |
|
|
|
## Base Models |
|
- several QWEN2 based models |
