llama-3-8b-finetuned-peft-exp

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.1404
+- Loss: 0.1550
 
 ## Model description
 
@@ -52,11 +52,11 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
-| 0.1223        | 0.1998 | 90   | 0.1680          |
-| 0.1614        | 0.3996 | 180  | 0.1531          |
-| 0.1621        | 0.5993 | 270  | 0.1549          |
-| 0.2369        | 0.7991 | 360  | 0.1443          |
-| 0.1496        | 0.9989 | 450  | 0.1404          |
+| 0.0902        | 0.1998 | 90   | 0.1856          |
+| 0.1458        | 0.3996 | 180  | 0.1749          |
+| 0.2055        | 0.5993 | 270  | 0.1664          |
+| 0.1414        | 0.7991 | 360  | 0.1581          |
+| 0.1347        | 0.9989 | 450  | 0.1550          |
 
 
 ### Framework versions

adapter_config.json

@@ -21,12 +21,12 @@
   "revision": null,
   "target_modules": [
     "down_proj",
-    "k_proj",
+    "up_proj",
     "v_proj",
+    "o_proj",
     "q_proj",
     "gate_proj",
-    "up_proj",
-    "o_proj"
+    "k_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fd6a35221a98099f33d85c3c0a9405aa74ac080560e40a92ebe11219a9f6df8d
+oid sha256:7357ca6c884c4f37f830b269d1ab2df933fb29ed9386aaf6c6f167be5e024c33
 size 2185326944

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bc6daa4908be123ed36b20847a1ac85a541ef074093784808a519ec03c76420f
+oid sha256:ed5e945f2800a83e0c3185e0e81196c8dc2602e1c556c52440bf497f4709ab1a
 size 5368