metadata

license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
  - generated_from_trainer
datasets:
  - essays_su_g
metrics:
  - accuracy
model-index:
  - name: longformer-spans
    results:
      - task:
          name: Token Classification
          type: token-classification
        dataset:
          name: essays_su_g
          type: essays_su_g
          config: spans
          split: test
          args: spans
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.9360779676806765

longformer-spans

This model is a fine-tuned version of allenai/longformer-base-4096 on the essays_su_g dataset. It achieves the following results on the evaluation set:

Loss: 0.1786
B: {'precision': 0.8126064735945485, 'recall': 0.9008498583569405, 'f1-score': 0.8544558889386475, 'support': 1059.0}
I: {'precision': 0.9468377121729875, 'recall': 0.9617069701280228, 'f1-score': 0.9542144187884605, 'support': 17575.0}
O: {'precision': 0.9307744259342638, 'recall': 0.8915363881401617, 'f1-score': 0.9107329698771959, 'support': 9275.0}
Accuracy: 0.9361
Macro avg: {'precision': 0.8967395372339334, 'recall': 0.918031072208375, 'f1-score': 0.9064677592014346, 'support': 27909.0}
Weighted avg: {'precision': 0.9364060284323041, 'recall': 0.9360779676806765, 'f1-score': 0.9359789133327677, 'support': 27909.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	B	I	O	Accuracy	Macro avg	Weighted avg
No log	1.0	41	0.2836	{'precision': 0.7968253968253968, 'recall': 0.4740321057601511, 'f1-score': 0.5944345766725874, 'support': 1059.0}	{'precision': 0.9161490683229814, 'recall': 0.9399715504978663, 'f1-score': 0.9279074339315303, 'support': 17575.0}	{'precision': 0.8717421866551314, 'recall': 0.8691105121293801, 'f1-score': 0.8704243602202786, 'support': 9275.0}	0.8987	{'precision': 0.8615722172678364, 'recall': 0.7610380561291326, 'f1-score': 0.7975887902747987, 'support': 27909.0}	{'precision': 0.8968636193428943, 'recall': 0.8987423411802644, 'f1-score': 0.8961505359950553, 'support': 27909.0}
No log	2.0	82	0.2022	{'precision': 0.7857142857142857, 'recall': 0.8621340887629839, 'f1-score': 0.8221521837010356, 'support': 1059.0}	{'precision': 0.9355464420305919, 'recall': 0.9605120910384068, 'f1-score': 0.9478649035627053, 'support': 17575.0}	{'precision': 0.9268068482132598, 'recall': 0.8696495956873316, 'f1-score': 0.8973189453776839, 'support': 9275.0}	0.9266	{'precision': 0.8826891919860458, 'recall': 0.8974319251629076, 'f1-score': 0.889112010880475, 'support': 27909.0}	{'precision': 0.9269566686171867, 'recall': 0.926582822745351, 'f1-score': 0.9262968240005718, 'support': 27909.0}
No log	3.0	123	0.1852	{'precision': 0.7871125611745514, 'recall': 0.9112370160528801, 'f1-score': 0.8446389496717724, 'support': 1059.0}	{'precision': 0.9400952275495515, 'recall': 0.966145092460882, 'f1-score': 0.9529421668490613, 'support': 17575.0}	{'precision': 0.940726133859181, 'recall': 0.8743935309973045, 'f1-score': 0.9063477872150201, 'support': 9275.0}	0.9336	{'precision': 0.8893113075277613, 'recall': 0.9172585465036889, 'f1-score': 0.9013096345786179, 'support': 27909.0}	{'precision': 0.9345000078114988, 'recall': 0.9335698161883264, 'f1-score': 0.9333479148838716, 'support': 27909.0}
No log	4.0	164	0.1786	{'precision': 0.8126064735945485, 'recall': 0.9008498583569405, 'f1-score': 0.8544558889386475, 'support': 1059.0}	{'precision': 0.9468377121729875, 'recall': 0.9617069701280228, 'f1-score': 0.9542144187884605, 'support': 17575.0}	{'precision': 0.9307744259342638, 'recall': 0.8915363881401617, 'f1-score': 0.9107329698771959, 'support': 9275.0}	0.9361	{'precision': 0.8967395372339334, 'recall': 0.918031072208375, 'f1-score': 0.9064677592014346, 'support': 27909.0}	{'precision': 0.9364060284323041, 'recall': 0.9360779676806765, 'f1-score': 0.9359789133327677, 'support': 27909.0}

Framework versions

Transformers 4.37.2
Pytorch 2.2.0+cu121
Datasets 2.17.0
Tokenizers 0.15.2