ai-maker-space/llama2-instruct-tune-100s

README.md

@@ -14,7 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [NousResearch/Llama-2-7b-hf](https://huggingface.co/NousResearch/Llama-2-7b-hf) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.7400
+- Loss: 1.6733
 
 ## Model description
 
@@ -40,17 +40,37 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
 - lr_scheduler_warmup_steps: 0.03
-- training_steps: 100
+- training_steps: 500
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.9478        | 0.0   | 20   | 1.8131          |
-| 1.7993        | 0.0   | 40   | 1.7797          |
-| 1.8768        | 0.0   | 60   | 1.7643          |
-| 1.8495        | 0.01  | 80   | 1.7529          |
-| 1.8518        | 0.01  | 100  | 1.7400          |
+| 1.9478        | 0.0   | 20   | 1.8129          |
+| 1.7992        | 0.0   | 40   | 1.7797          |
+| 1.8765        | 0.0   | 60   | 1.7642          |
+| 1.8488        | 0.01  | 80   | 1.7527          |
+| 1.8512        | 0.01  | 100  | 1.7384          |
+| 1.8856        | 0.01  | 120  | 1.7136          |
+| 1.8429        | 0.01  | 140  | 1.7035          |
+| 1.8258        | 0.01  | 160  | 1.6970          |
+| 1.8125        | 0.01  | 180  | 1.6923          |
+| 1.7902        | 0.01  | 200  | 1.6898          |
+| 1.8622        | 0.02  | 220  | 1.6895          |
+| 1.8823        | 0.02  | 240  | 1.6867          |
+| 1.7728        | 0.02  | 260  | 1.6837          |
+| 1.6331        | 0.02  | 280  | 1.6820          |
+| 1.8399        | 0.02  | 300  | 1.6821          |
+| 1.735         | 0.02  | 320  | 1.6810          |
+| 1.8192        | 0.02  | 340  | 1.6804          |
+| 1.7609        | 0.03  | 360  | 1.6771          |
+| 1.6517        | 0.03  | 380  | 1.6768          |
+| 1.6272        | 0.03  | 400  | 1.6738          |
+| 1.7947        | 0.03  | 420  | 1.6751          |
+| 1.7794        | 0.03  | 440  | 1.6740          |
+| 1.6371        | 0.03  | 460  | 1.6726          |
+| 1.7126        | 0.03  | 480  | 1.6730          |
+| 1.7321        | 0.04  | 500  | 1.6733          |
 
 
 ### Framework versions

adapter_config.json

@@ -16,8 +16,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "v_proj"
+    "v_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM"
 }

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ff30ee137e3cc8a5da81daaea62b250bd10b757fe624c225ec8d04416057691b
+oid sha256:73f613e808fdcd86d2fda293b7e25d1788b940476c988d517ae2dbe25ca2db50
 size 134235048

runs/Nov14_23-41-36_4a9ba01a8270/events.out.tfevents.1700005296.4a9ba01a8270.21049.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53aa6e152adf610705984c8017ed221593fb44d7848a6784f4c05b741db16cf9
+size 19580

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ceefebd3833df8f7f9fa6d2d6e7038c3725c2411a9fadbce22254cf24a1b63c6
+oid sha256:95aa17d15e436395bb17415ed4cb606077c6ea609cce8bb1f8da04d026217b9d
 size 4600