Model save

README.md

@@ -0,0 +1,74 @@
+---
+license: apache-2.0
+base_model: alignment-handbook/zephyr-7b-sft-full
+tags:
+- generated_from_trainer
+model-index:
+- name: spin-margin2
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# spin-margin2
+
+This model is a fine-tuned version of [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0010
+- Rewards/real: -0.7975
+- Rewards/generated: -20.4822
+- Rewards/accuracies: 1.0
+- Rewards/margins: 19.6846
+- Logps/generated: -303.8466
+- Logps/real: -141.0674
+- Logits/generated: -2.6068
+- Logits/real: -2.3492
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-07
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 4
+- total_train_batch_size: 32
+- total_eval_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 1
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Rewards/real | Rewards/generated | Rewards/accuracies | Rewards/margins | Logps/generated | Logps/real | Logits/generated | Logits/real |
+|:-------------:|:-----:|:----:|:---------------:|:------------:|:-----------------:|:------------------:|:---------------:|:---------------:|:----------:|:----------------:|:-----------:|
+| 0.0043        | 0.19  | 100  | 0.0049          | 0.9120       | -9.6012           | 1.0                | 10.5132         | -195.0367       | -123.9721  | -2.7982          | -2.5652     |
+| 0.0034        | 0.39  | 200  | 0.0024          | -0.0739      | -14.1834          | 1.0                | 14.1095         | -240.8593       | -133.8314  | -2.8109          | -2.5347     |
+| 0.0007        | 0.58  | 300  | 0.0012          | -0.2381      | -16.9127          | 1.0                | 16.6746         | -268.1524       | -135.4731  | -2.7308          | -2.4046     |
+| 0.0016        | 0.78  | 400  | 0.0010          | -1.1878      | -19.5719          | 1.0                | 18.3841         | -294.7439       | -144.9703  | -2.6559          | -2.3917     |
+| 0.0001        | 0.97  | 500  | 0.0010          | -0.7975      | -20.4822          | 1.0                | 19.6846         | -303.8466       | -141.0674  | -2.6068          | -2.3492     |
+
+
+### Framework versions
+
+- Transformers 4.37.0
+- Pytorch 2.1.2+cu121
+- Datasets 2.14.6
+- Tokenizers 0.15.2

all_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "train_loss": 0.016583090304290803,
+    "train_runtime": 3283.9246,
+    "train_samples": 16467,
+    "train_samples_per_second": 5.014,
+    "train_steps_per_second": 0.157
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.37.0"
+}

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e98e3d2581137a81621547ef6c0ad50dcd903475eb5c2a356ecf404e6a543c8
+size 4943162336

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41b565c1c77fd26bce5fee9886091f03e8ae36522e056b2023be74791f895c16
+size 4999819336

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1523f9bd758ce036e2719e3a589b44dd58e1156c757a71ac3db4cb80cf6c270
+size 4540516344

model.safetensors.index.json

@@ -0,0 +1,298 @@
+{
+  "metadata": {
+    "total_size": 14483464192
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00003-of-00003.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.norm.weight": "model-00003-of-00003.safetensors"
+  }
+}

runs/Jun18_10-29-25_gilbreth-j001.rcac.purdue.edu/events.out.tfevents.1718721100.gilbreth-j001.rcac.purdue.edu.219991.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3b46c6b3d7d75401c87e7a9e0a91ebf38d9a29524f9af9fb7e42ec84173e8c2e
-size 40271
+oid sha256:96e0c37b6d9118359a0f3f130cbb4fb84579c39cfb97293a80af9be97821eac0
+size 41256

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "train_loss": 0.016583090304290803,
+    "train_runtime": 3283.9246,
+    "train_samples": 16467,
+    "train_samples_per_second": 5.014,
+    "train_steps_per_second": 0.157
+}

trainer_state.json

@@ -0,0 +1,838 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 100,
+  "global_step": 515,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0,
+      "learning_rate": 9.615384615384615e-09,
+      "logits/generated": -3.0575106143951416,
+      "logits/real": -2.816924571990967,
+      "logps/generated": -89.89034271240234,
+      "logps/real": -110.7723617553711,
+      "loss": 0.6931,
+      "rewards/accuracies": 0.0,
+      "rewards/generated": 0.0,
+      "rewards/margins": 0.0,
+      "rewards/real": 0.0,
+      "step": 1
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 9.615384615384616e-08,
+      "logits/generated": -3.0530381202697754,
+      "logits/real": -2.897934675216675,
+      "logps/generated": -101.54727935791016,
+      "logps/real": -141.96624755859375,
+      "loss": 0.5193,
+      "rewards/accuracies": 0.7638888955116272,
+      "rewards/generated": -0.3860962390899658,
+      "rewards/margins": 0.4838888943195343,
+      "rewards/real": 0.09779267013072968,
+      "step": 10
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.9230769230769231e-07,
+      "logits/generated": -3.022646188735962,
+      "logits/real": -2.780233860015869,
+      "logps/generated": -117.37088775634766,
+      "logps/real": -120.82357025146484,
+      "loss": 0.1046,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -2.058361291885376,
+      "rewards/margins": 2.6513800621032715,
+      "rewards/real": 0.5930188894271851,
+      "step": 20
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 2.884615384615384e-07,
+      "logits/generated": -2.9492268562316895,
+      "logits/real": -2.746804714202881,
+      "logps/generated": -131.04525756835938,
+      "logps/real": -121.73081970214844,
+      "loss": 0.0414,
+      "rewards/accuracies": 0.9750000238418579,
+      "rewards/generated": -3.5284512042999268,
+      "rewards/margins": 4.389330863952637,
+      "rewards/real": 0.8608797788619995,
+      "step": 30
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 3.8461538461538463e-07,
+      "logits/generated": -2.8972926139831543,
+      "logits/real": -2.75730562210083,
+      "logps/generated": -132.66815185546875,
+      "logps/real": -132.48605346679688,
+      "loss": 0.0317,
+      "rewards/accuracies": 0.987500011920929,
+      "rewards/generated": -4.410510063171387,
+      "rewards/margins": 5.345829963684082,
+      "rewards/real": 0.9353200793266296,
+      "step": 40
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 4.807692307692307e-07,
+      "logits/generated": -2.9142377376556396,
+      "logits/real": -2.6744868755340576,
+      "logps/generated": -152.9608612060547,
+      "logps/real": -120.39155578613281,
+      "loss": 0.0178,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -5.3825178146362305,
+      "rewards/margins": 6.37641716003418,
+      "rewards/real": 0.9938996434211731,
+      "step": 50
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 4.913606911447084e-07,
+      "logits/generated": -2.8876500129699707,
+      "logits/real": -2.6662659645080566,
+      "logps/generated": -169.52001953125,
+      "logps/real": -121.60459899902344,
+      "loss": 0.0134,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -6.808724880218506,
+      "rewards/margins": 7.904421806335449,
+      "rewards/real": 1.0956966876983643,
+      "step": 60
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 4.80561555075594e-07,
+      "logits/generated": -2.8779220581054688,
+      "logits/real": -2.714576005935669,
+      "logps/generated": -173.93214416503906,
+      "logps/real": -140.5163116455078,
+      "loss": 0.0088,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -7.277545928955078,
+      "rewards/margins": 8.348401069641113,
+      "rewards/real": 1.0708551406860352,
+      "step": 70
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 4.6976241900647946e-07,
+      "logits/generated": -2.854962110519409,
+      "logits/real": -2.5673232078552246,
+      "logps/generated": -187.35971069335938,
+      "logps/real": -115.74288177490234,
+      "loss": 0.008,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -8.015608787536621,
+      "rewards/margins": 8.94044017791748,
+      "rewards/real": 0.9248324632644653,
+      "step": 80
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 4.5896328293736496e-07,
+      "logits/generated": -2.8408970832824707,
+      "logits/real": -2.5543339252471924,
+      "logps/generated": -190.44345092773438,
+      "logps/real": -117.4663314819336,
+      "loss": 0.0096,
+      "rewards/accuracies": 0.987500011920929,
+      "rewards/generated": -8.589609146118164,
+      "rewards/margins": 9.519355773925781,
+      "rewards/real": 0.9297460317611694,
+      "step": 90
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 4.481641468682505e-07,
+      "logits/generated": -2.8061957359313965,
+      "logits/real": -2.5320754051208496,
+      "logps/generated": -201.36314392089844,
+      "logps/real": -106.45611572265625,
+      "loss": 0.0043,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -9.835786819458008,
+      "rewards/margins": 10.548227310180664,
+      "rewards/real": 0.7124411463737488,
+      "step": 100
+    },
+    {
+      "epoch": 0.19,
+      "eval_logits/generated": -2.7982144355773926,
+      "eval_logits/real": -2.5651581287384033,
+      "eval_logps/generated": -195.03672790527344,
+      "eval_logps/real": -123.97211456298828,
+      "eval_loss": 0.004925635177642107,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/generated": -9.601181983947754,
+      "eval_rewards/margins": 10.513165473937988,
+      "eval_rewards/real": 0.9119827151298523,
+      "eval_runtime": 115.6828,
+      "eval_samples_per_second": 14.522,
+      "eval_steps_per_second": 0.458,
+      "step": 100
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 4.3736501079913605e-07,
+      "logits/generated": -2.7716643810272217,
+      "logits/real": -2.528700351715088,
+      "logps/generated": -192.03431701660156,
+      "logps/real": -115.5088119506836,
+      "loss": 0.0027,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -9.888772964477539,
+      "rewards/margins": 10.59119987487793,
+      "rewards/real": 0.7024269104003906,
+      "step": 110
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 4.265658747300216e-07,
+      "logits/generated": -2.772347927093506,
+      "logits/real": -2.439427137374878,
+      "logps/generated": -199.63067626953125,
+      "logps/real": -117.44853210449219,
+      "loss": 0.008,
+      "rewards/accuracies": 0.987500011920929,
+      "rewards/generated": -10.372490882873535,
+      "rewards/margins": 10.862470626831055,
+      "rewards/real": 0.48998016119003296,
+      "step": 120
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 4.157667386609071e-07,
+      "logits/generated": -2.790419101715088,
+      "logits/real": -2.504594564437866,
+      "logps/generated": -182.69247436523438,
+      "logps/real": -117.50340270996094,
+      "loss": 0.0116,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -9.6342134475708,
+      "rewards/margins": 9.86939811706543,
+      "rewards/real": 0.2351863831281662,
+      "step": 130
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 4.0496760259179263e-07,
+      "logits/generated": -2.848449230194092,
+      "logits/real": -2.592134475708008,
+      "logps/generated": -209.822998046875,
+      "logps/real": -132.05592346191406,
+      "loss": 0.0088,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -10.387749671936035,
+      "rewards/margins": 10.231754302978516,
+      "rewards/real": -0.15599501132965088,
+      "step": 140
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 3.941684665226782e-07,
+      "logits/generated": -2.863227367401123,
+      "logits/real": -2.6038713455200195,
+      "logps/generated": -205.6251678466797,
+      "logps/real": -132.90298461914062,
+      "loss": 0.0035,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -10.28604507446289,
+      "rewards/margins": 10.251394271850586,
+      "rewards/real": -0.03464997559785843,
+      "step": 150
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 3.8336933045356367e-07,
+      "logits/generated": -2.836599111557007,
+      "logits/real": -2.6022212505340576,
+      "logps/generated": -199.63819885253906,
+      "logps/real": -126.2606201171875,
+      "loss": 0.005,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -10.73370361328125,
+      "rewards/margins": 10.730461120605469,
+      "rewards/real": -0.0032417401671409607,
+      "step": 160
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 3.7257019438444927e-07,
+      "logits/generated": -2.848566770553589,
+      "logits/real": -2.5174214839935303,
+      "logps/generated": -235.12161254882812,
+      "logps/real": -115.94686126708984,
+      "loss": 0.0017,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -13.108288764953613,
+      "rewards/margins": 12.952964782714844,
+      "rewards/real": -0.155323326587677,
+      "step": 170
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 3.6177105831533476e-07,
+      "logits/generated": -2.8279058933258057,
+      "logits/real": -2.5158348083496094,
+      "logps/generated": -231.4835968017578,
+      "logps/real": -140.82273864746094,
+      "loss": 0.0033,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -12.741474151611328,
+      "rewards/margins": 12.89801025390625,
+      "rewards/real": 0.15653587877750397,
+      "step": 180
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 3.509719222462203e-07,
+      "logits/generated": -2.8390724658966064,
+      "logits/real": -2.552072048187256,
+      "logps/generated": -226.83334350585938,
+      "logps/real": -127.37821197509766,
+      "loss": 0.0011,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -12.685934066772461,
+      "rewards/margins": 12.80540657043457,
+      "rewards/real": 0.11947281658649445,
+      "step": 190
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 3.401727861771058e-07,
+      "logits/generated": -2.832120418548584,
+      "logits/real": -2.5636706352233887,
+      "logps/generated": -224.13436889648438,
+      "logps/real": -126.2430419921875,
+      "loss": 0.0034,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -12.46034049987793,
+      "rewards/margins": 12.773991584777832,
+      "rewards/real": 0.31365132331848145,
+      "step": 200
+    },
+    {
+      "epoch": 0.39,
+      "eval_logits/generated": -2.8109114170074463,
+      "eval_logits/real": -2.53472638130188,
+      "eval_logps/generated": -240.85931396484375,
+      "eval_logps/real": -133.8314208984375,
+      "eval_loss": 0.002396376570686698,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/generated": -14.183443069458008,
+      "eval_rewards/margins": 14.109496116638184,
+      "eval_rewards/real": -0.07394662499427795,
+      "eval_runtime": 113.7278,
+      "eval_samples_per_second": 14.772,
+      "eval_steps_per_second": 0.466,
+      "step": 200
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 3.2937365010799135e-07,
+      "logits/generated": -2.8051133155822754,
+      "logits/real": -2.4014732837677,
+      "logps/generated": -237.0082244873047,
+      "logps/real": -113.56056213378906,
+      "loss": 0.001,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -14.111905097961426,
+      "rewards/margins": 13.899911880493164,
+      "rewards/real": -0.21199479699134827,
+      "step": 210
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 3.1857451403887684e-07,
+      "logits/generated": -2.820549488067627,
+      "logits/real": -2.5103089809417725,
+      "logps/generated": -243.4583740234375,
+      "logps/real": -127.36563873291016,
+      "loss": 0.0018,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -14.133171081542969,
+      "rewards/margins": 14.2222900390625,
+      "rewards/real": 0.08911846578121185,
+      "step": 220
+    },
+    {
+      "epoch": 0.45,
+      "learning_rate": 3.077753779697624e-07,
+      "logits/generated": -2.8067831993103027,
+      "logits/real": -2.6124329566955566,
+      "logps/generated": -219.4794158935547,
+      "logps/real": -123.17138671875,
+      "loss": 0.0025,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -12.553444862365723,
+      "rewards/margins": 12.771233558654785,
+      "rewards/real": 0.21778886020183563,
+      "step": 230
+    },
+    {
+      "epoch": 0.47,
+      "learning_rate": 2.9697624190064794e-07,
+      "logits/generated": -2.8342039585113525,
+      "logits/real": -2.46539306640625,
+      "logps/generated": -229.17520141601562,
+      "logps/real": -126.71480560302734,
+      "loss": 0.0005,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -12.728944778442383,
+      "rewards/margins": 12.824361801147461,
+      "rewards/real": 0.09541518241167068,
+      "step": 240
+    },
+    {
+      "epoch": 0.49,
+      "learning_rate": 2.861771058315335e-07,
+      "logits/generated": -2.784558057785034,
+      "logits/real": -2.4345905780792236,
+      "logps/generated": -229.5637969970703,
+      "logps/real": -120.04652404785156,
+      "loss": 0.002,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -13.79577350616455,
+      "rewards/margins": 13.668333053588867,
+      "rewards/real": -0.12744084000587463,
+      "step": 250
+    },
+    {
+      "epoch": 0.5,
+      "learning_rate": 2.7537796976241903e-07,
+      "logits/generated": -2.7688822746276855,
+      "logits/real": -2.3783717155456543,
+      "logps/generated": -251.04879760742188,
+      "logps/real": -122.02119445800781,
+      "loss": 0.0023,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -14.788324356079102,
+      "rewards/margins": 14.605295181274414,
+      "rewards/real": -0.1830286979675293,
+      "step": 260
+    },
+    {
+      "epoch": 0.52,
+      "learning_rate": 2.645788336933045e-07,
+      "logits/generated": -2.761305093765259,
+      "logits/real": -2.324169635772705,
+      "logps/generated": -248.24539184570312,
+      "logps/real": -111.51519775390625,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -14.844027519226074,
+      "rewards/margins": 14.8836088180542,
+      "rewards/real": 0.03958074748516083,
+      "step": 270
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 2.5377969762419007e-07,
+      "logits/generated": -2.7696480751037598,
+      "logits/real": -2.422027826309204,
+      "logps/generated": -254.77340698242188,
+      "logps/real": -130.95404052734375,
+      "loss": 0.0009,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -15.440943717956543,
+      "rewards/margins": 15.30030632019043,
+      "rewards/real": -0.14063778519630432,
+      "step": 280
+    },
+    {
+      "epoch": 0.56,
+      "learning_rate": 2.4298056155507556e-07,
+      "logits/generated": -2.746011972427368,
+      "logits/real": -2.287593126296997,
+      "logps/generated": -258.4119873046875,
+      "logps/real": -124.4466781616211,
+      "loss": 0.0005,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -15.832595825195312,
+      "rewards/margins": 15.681103706359863,
+      "rewards/real": -0.15149332582950592,
+      "step": 290
+    },
+    {
+      "epoch": 0.58,
+      "learning_rate": 2.3218142548596113e-07,
+      "logits/generated": -2.7288124561309814,
+      "logits/real": -2.3387868404388428,
+      "logps/generated": -255.36669921875,
+      "logps/real": -116.48316955566406,
+      "loss": 0.0007,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -16.082441329956055,
+      "rewards/margins": 15.728803634643555,
+      "rewards/real": -0.3536352515220642,
+      "step": 300
+    },
+    {
+      "epoch": 0.58,
+      "eval_logits/generated": -2.730811357498169,
+      "eval_logits/real": -2.404576063156128,
+      "eval_logps/generated": -268.1523742675781,
+      "eval_logps/real": -135.4730987548828,
+      "eval_loss": 0.0012407265603542328,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/generated": -16.912744522094727,
+      "eval_rewards/margins": 16.67462921142578,
+      "eval_rewards/real": -0.23811471462249756,
+      "eval_runtime": 114.2693,
+      "eval_samples_per_second": 14.702,
+      "eval_steps_per_second": 0.464,
+      "step": 300
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 2.2138228941684665e-07,
+      "logits/generated": -2.705501079559326,
+      "logits/real": -2.2757084369659424,
+      "logps/generated": -258.28076171875,
+      "logps/real": -140.76162719726562,
+      "loss": 0.0011,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -15.845602035522461,
+      "rewards/margins": 14.796327590942383,
+      "rewards/real": -1.0492744445800781,
+      "step": 310
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 2.1058315334773217e-07,
+      "logits/generated": -2.7162063121795654,
+      "logits/real": -2.4163174629211426,
+      "logps/generated": -282.51336669921875,
+      "logps/real": -138.6903839111328,
+      "loss": 0.0006,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -18.408849716186523,
+      "rewards/margins": 17.277786254882812,
+      "rewards/real": -1.131063461303711,
+      "step": 320
+    },
+    {
+      "epoch": 0.64,
+      "learning_rate": 1.9978401727861772e-07,
+      "logits/generated": -2.6899831295013428,
+      "logits/real": -2.407248020172119,
+      "logps/generated": -282.01873779296875,
+      "logps/real": -141.7494354248047,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -18.30569076538086,
+      "rewards/margins": 17.74685287475586,
+      "rewards/real": -0.5588369965553284,
+      "step": 330
+    },
+    {
+      "epoch": 0.66,
+      "learning_rate": 1.8898488120950324e-07,
+      "logits/generated": -2.694230556488037,
+      "logits/real": -2.284055233001709,
+      "logps/generated": -272.35986328125,
+      "logps/real": -123.73674011230469,
+      "loss": 0.0004,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -17.68344497680664,
+      "rewards/margins": 16.862354278564453,
+      "rewards/real": -0.8210934400558472,
+      "step": 340
+    },
+    {
+      "epoch": 0.68,
+      "learning_rate": 1.7818574514038876e-07,
+      "logits/generated": -2.713329315185547,
+      "logits/real": -2.368961811065674,
+      "logps/generated": -274.2613830566406,
+      "logps/real": -129.2357635498047,
+      "loss": 0.0009,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -17.50069808959961,
+      "rewards/margins": 16.7661190032959,
+      "rewards/real": -0.7345799207687378,
+      "step": 350
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 1.673866090712743e-07,
+      "logits/generated": -2.701970100402832,
+      "logits/real": -2.498610258102417,
+      "logps/generated": -284.69195556640625,
+      "logps/real": -152.46725463867188,
+      "loss": 0.0016,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -18.89683723449707,
+      "rewards/margins": 18.188879013061523,
+      "rewards/real": -0.7079570293426514,
+      "step": 360
+    },
+    {
+      "epoch": 0.72,
+      "learning_rate": 1.5658747300215982e-07,
+      "logits/generated": -2.6658008098602295,
+      "logits/real": -2.3703696727752686,
+      "logps/generated": -280.6600036621094,
+      "logps/real": -144.48782348632812,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -18.335285186767578,
+      "rewards/margins": 17.47960090637207,
+      "rewards/real": -0.8556815385818481,
+      "step": 370
+    },
+    {
+      "epoch": 0.74,
+      "learning_rate": 1.4578833693304534e-07,
+      "logits/generated": -2.7102785110473633,
+      "logits/real": -2.3787968158721924,
+      "logps/generated": -288.45074462890625,
+      "logps/real": -136.98178100585938,
+      "loss": 0.0018,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -18.772268295288086,
+      "rewards/margins": 18.083477020263672,
+      "rewards/real": -0.6887896060943604,
+      "step": 380
+    },
+    {
+      "epoch": 0.76,
+      "learning_rate": 1.3498920086393086e-07,
+      "logits/generated": -2.6787545680999756,
+      "logits/real": -2.271860122680664,
+      "logps/generated": -288.81396484375,
+      "logps/real": -126.51753234863281,
+      "loss": 0.0009,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -18.791013717651367,
+      "rewards/margins": 17.96844482421875,
+      "rewards/real": -0.8225716352462769,
+      "step": 390
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 1.241900647948164e-07,
+      "logits/generated": -2.6518869400024414,
+      "logits/real": -2.3778929710388184,
+      "logps/generated": -278.9534912109375,
+      "logps/real": -152.96298217773438,
+      "loss": 0.0016,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -18.41701316833496,
+      "rewards/margins": 16.954509735107422,
+      "rewards/real": -1.4625046253204346,
+      "step": 400
+    },
+    {
+      "epoch": 0.78,
+      "eval_logits/generated": -2.655914068222046,
+      "eval_logits/real": -2.3916683197021484,
+      "eval_logps/generated": -294.743896484375,
+      "eval_logps/real": -144.9702606201172,
+      "eval_loss": 0.000959120923653245,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/generated": -19.5718994140625,
+      "eval_rewards/margins": 18.384069442749023,
+      "eval_rewards/real": -1.1878323554992676,
+      "eval_runtime": 113.966,
+      "eval_samples_per_second": 14.741,
+      "eval_steps_per_second": 0.465,
+      "step": 400
+    },
+    {
+      "epoch": 0.8,
+      "learning_rate": 1.1339092872570194e-07,
+      "logits/generated": -2.668544292449951,
+      "logits/real": -2.383988380432129,
+      "logps/generated": -284.5706481933594,
+      "logps/real": -138.00421142578125,
+      "loss": 0.0011,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -18.296131134033203,
+      "rewards/margins": 16.981504440307617,
+      "rewards/real": -1.3146272897720337,
+      "step": 410
+    },
+    {
+      "epoch": 0.82,
+      "learning_rate": 1.0259179265658747e-07,
+      "logits/generated": -2.637368679046631,
+      "logits/real": -2.379403829574585,
+      "logps/generated": -290.41619873046875,
+      "logps/real": -135.14498901367188,
+      "loss": 0.001,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -19.232669830322266,
+      "rewards/margins": 18.452062606811523,
+      "rewards/real": -0.7806074619293213,
+      "step": 420
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 9.179265658747299e-08,
+      "logits/generated": -2.6258468627929688,
+      "logits/real": -2.2875823974609375,
+      "logps/generated": -289.2123718261719,
+      "logps/real": -127.7003402709961,
+      "loss": 0.0007,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -19.344642639160156,
+      "rewards/margins": 18.49654769897461,
+      "rewards/real": -0.8480979204177856,
+      "step": 430
+    },
+    {
+      "epoch": 0.85,
+      "learning_rate": 8.099352051835854e-08,
+      "logits/generated": -2.6232876777648926,
+      "logits/real": -2.2331902980804443,
+      "logps/generated": -286.9706726074219,
+      "logps/real": -136.14852905273438,
+      "loss": 0.0004,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -19.171314239501953,
+      "rewards/margins": 17.70499038696289,
+      "rewards/real": -1.4663217067718506,
+      "step": 440
+    },
+    {
+      "epoch": 0.87,
+      "learning_rate": 7.019438444924406e-08,
+      "logits/generated": -2.6160647869110107,
+      "logits/real": -2.321377992630005,
+      "logps/generated": -288.0433044433594,
+      "logps/real": -141.96792602539062,
+      "loss": 0.0004,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -19.229713439941406,
+      "rewards/margins": 18.14931869506836,
+      "rewards/real": -1.0803945064544678,
+      "step": 450
+    },
+    {
+      "epoch": 0.89,
+      "learning_rate": 5.939524838012959e-08,
+      "logits/generated": -2.6172728538513184,
+      "logits/real": -2.3889567852020264,
+      "logps/generated": -304.689697265625,
+      "logps/real": -138.3423614501953,
+      "loss": 0.0024,
+      "rewards/accuracies": 0.987500011920929,
+      "rewards/generated": -20.097713470458984,
+      "rewards/margins": 19.377609252929688,
+      "rewards/real": -0.7201052308082581,
+      "step": 460
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 4.8596112311015116e-08,
+      "logits/generated": -2.6321027278900146,
+      "logits/real": -2.3017470836639404,
+      "logps/generated": -284.72479248046875,
+      "logps/real": -128.301025390625,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -19.106754302978516,
+      "rewards/margins": 18.5189208984375,
+      "rewards/real": -0.5878311991691589,
+      "step": 470
+    },
+    {
+      "epoch": 0.93,
+      "learning_rate": 3.779697624190065e-08,
+      "logits/generated": -2.61067795753479,
+      "logits/real": -2.3235392570495605,
+      "logps/generated": -293.27984619140625,
+      "logps/real": -133.7152862548828,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -19.834014892578125,
+      "rewards/margins": 18.92249870300293,
+      "rewards/real": -0.9115169644355774,
+      "step": 480
+    },
+    {
+      "epoch": 0.95,
+      "learning_rate": 2.6997840172786175e-08,
+      "logits/generated": -2.620330333709717,
+      "logits/real": -2.346421241760254,
+      "logps/generated": -295.2581787109375,
+      "logps/real": -136.19390869140625,
+      "loss": 0.0005,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -19.7529296875,
+      "rewards/margins": 18.93564224243164,
+      "rewards/real": -0.8172855377197266,
+      "step": 490
+    },
+    {
+      "epoch": 0.97,
+      "learning_rate": 1.6198704103671707e-08,
+      "logits/generated": -2.619122266769409,
+      "logits/real": -2.424999475479126,
+      "logps/generated": -298.48040771484375,
+      "logps/real": -157.33090209960938,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -20.18784523010254,
+      "rewards/margins": 19.563133239746094,
+      "rewards/real": -0.6247111558914185,
+      "step": 500
+    },
+    {
+      "epoch": 0.97,
+      "eval_logits/generated": -2.606785535812378,
+      "eval_logits/real": -2.3491668701171875,
+      "eval_logps/generated": -303.8465576171875,
+      "eval_logps/real": -141.06735229492188,
+      "eval_loss": 0.0009754680213518441,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/generated": -20.482166290283203,
+      "eval_rewards/margins": 19.684627532958984,
+      "eval_rewards/real": -0.7975420951843262,
+      "eval_runtime": 113.9889,
+      "eval_samples_per_second": 14.738,
+      "eval_steps_per_second": 0.465,
+      "step": 500
+    },
+    {
+      "epoch": 0.99,
+      "learning_rate": 5.3995680345572344e-09,
+      "logits/generated": -2.629978895187378,
+      "logits/real": -2.2564597129821777,
+      "logps/generated": -287.25323486328125,
+      "logps/real": -123.89505767822266,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/generated": -18.566009521484375,
+      "rewards/margins": 17.70358657836914,
+      "rewards/real": -0.8624227643013,
+      "step": 510
+    },
+    {
+      "epoch": 1.0,
+      "step": 515,
+      "total_flos": 0.0,
+      "train_loss": 0.016583090304290803,
+      "train_runtime": 3283.9246,
+      "train_samples_per_second": 5.014,
+      "train_steps_per_second": 0.157
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 515,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 100,
+  "total_flos": 0.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}