Update README.md
Browse files
README.md
CHANGED
|
@@ -17,8 +17,6 @@ The model is based on a "zeroed" passthrough merge of [Llama-3-15B-Instruct-zero
|
|
| 17 |
|
| 18 |
This was primarily an experiment to see how a passthrough merge will respond to further finetuning, though this was done on a small dataset.
|
| 19 |
|
| 20 |
-
The goal was to make a "mid" sized model like Meta has released in the past and the merge method was inspired by [mlabonne's Llama-3-120B](https://huggingface.co/mlabonne/Meta-Llama-3-120B-Instruct).
|
| 21 |
-
|
| 22 |
The model was finetuned on **8192 context length** and is likely reliable using RoPE up to 32k.
|
| 23 |
|
| 24 |
Further finetuning this model or finetuning the [base model](https://huggingface.co/elinas/Llama-3-15B-Instruct-zeroed) on more samples is encouraged.
|
|
|
|
| 17 |
|
| 18 |
This was primarily an experiment to see how a passthrough merge will respond to further finetuning, though this was done on a small dataset.
|
| 19 |
|
|
|
|
|
|
|
| 20 |
The model was finetuned on **8192 context length** and is likely reliable using RoPE up to 32k.
|
| 21 |
|
| 22 |
Further finetuning this model or finetuning the [base model](https://huggingface.co/elinas/Llama-3-15B-Instruct-zeroed) on more samples is encouraged.
|