End of training

README.md

@@ -19,6 +19,8 @@ should probably proofread and complete it, then remove this comment. -->
 # SmolLM-135M-Instruct-Finetune-LoRA
 
 This model is a fine-tuned version of [HuggingFaceTB/SmolLM-135M](https://huggingface.co/HuggingFaceTB/SmolLM-135M) on the generator dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.4752
 
 ## Model description
 
@@ -49,11 +51,19 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_ratio: 0.1
-- training_steps: 1
+- num_epochs: 4
 - mixed_precision_training: Native AMP
 
 ### Training results
 
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 1.9806        | 0.6173 | 25   | 1.9171          |
+| 1.8272        | 1.2346 | 50   | 1.7049          |
+| 1.6412        | 1.8519 | 75   | 1.5713          |
+| 1.5453        | 2.4691 | 100  | 1.5098          |
+| 1.5006        | 3.0864 | 125  | 1.4822          |
+| 1.4829        | 3.7037 | 150  | 1.4752          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "up_proj",
-    "k_proj",
-    "o_proj",
     "down_proj",
-    "q_proj",
     "v_proj",
-    "gate_proj"
+    "gate_proj",
+    "k_proj",
+    "o_proj",
+    "up_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4fe7e9521eca7961cd31bd7700cbef38e9ae1b6e91059c464843c6045fc6f83f
+oid sha256:c3b4ca791c9f9c01295edcaef3a43f2136420e2dade49afd94618c13ef947b6b
 size 19593064

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2e5ebb42d904b2d2cff340cde81eabbc832efe72d870e31b37eef616aa413661
+oid sha256:f5cba0bfc5734dbb897f1356225cd298207d7c656f42a7fa9bb6c108759ff5ea
 size 5496