README.md

metadata

language:
  - ko
  - en
license: mit

Model Card for free-evo-qwen72b-v0.8

Developed by : Freewheelin AI Technical Team

1st place : 2024 4th May - avg. 81.28 Open Llm Leaderboard

but this kicked away. maybe the explanation was not enough.

Method

• We were inspired by this Sakana project

Process

• you need two models with the same architecture
• 1. choose one model and fine-tune a model to make a gap between the original one and fine-tuned one.
• 2. merge two of them
• 3. evaluate the merged model
• 4. fine-tune a specific evaluation part of the model
• 5. evaluate again
• 6. merge again
• 5. evaluate again
• 6. keep going until evaluate avg is higher then original one

that's it. simple.

Base Architecture

• QWEN2

Base Models

• several QWEN2 based models