metadata

base_model: allenai/longformer-base-4096
tags:
  - generated_from_trainer
datasets:
  - essays_su_g
metrics:
  - accuracy
model-index:
  - name: longformer-spans
    results:
      - task:
          name: Token Classification
          type: token-classification
        dataset:
          name: essays_su_g
          type: essays_su_g
          config: spans
          split: train[80%:100%]
          args: spans
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.9313516057786306

longformer-spans

This model is a fine-tuned version of allenai/longformer-base-4096 on the essays_su_g dataset. It achieves the following results on the evaluation set:

Loss: 0.1886
B: {'precision': 0.8005115089514067, 'recall': 0.900287631831256, 'f1-score': 0.8474729241877257, 'support': 1043.0}
I: {'precision': 0.9321724709784411, 'recall': 0.9719308357348703, 'f1-score': 0.9516365688487585, 'support': 17350.0}
O: {'precision': 0.947941598851125, 'recall': 0.8585519184912205, 'f1-score': 0.9010351495848026, 'support': 9226.0}
Accuracy: 0.9314
Macro avg: {'precision': 0.8935418595936575, 'recall': 0.9102567953524489, 'f1-score': 0.9000482142070956, 'support': 27619.0}
Weighted avg: {'precision': 0.9324680497596853, 'recall': 0.9313516057786306, 'f1-score': 0.9307997762237281, 'support': 27619.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	B	I	O	Accuracy	Macro avg	Weighted avg
No log	1.0	41	0.3465	{'precision': 0.7459016393442623, 'recall': 0.174496644295302, 'f1-score': 0.2828282828282829, 'support': 1043.0}	{'precision': 0.8462454712392674, 'recall': 0.9827665706051874, 'f1-score': 0.90941091762447, 'support': 17350.0}	{'precision': 0.9458898422363686, 'recall': 0.7408411012356384, 'f1-score': 0.8309020179917336, 'support': 9226.0}	0.8714	{'precision': 0.8460123176066329, 'recall': 0.6327014387120425, 'f1-score': 0.6743804061481621, 'support': 27619.0}	{'precision': 0.8757418451178571, 'recall': 0.8714290886708426, 'f1-score': 0.8595232027867116, 'support': 27619.0}
No log	2.0	82	0.2059	{'precision': 0.7637130801687764, 'recall': 0.8676893576222435, 'f1-score': 0.8123877917414721, 'support': 1043.0}	{'precision': 0.9387513394619593, 'recall': 0.9593659942363112, 'f1-score': 0.9489467232975115, 'support': 17350.0}	{'precision': 0.9291049063541308, 'recall': 0.8764361586819857, 'f1-score': 0.9020023425734843, 'support': 9226.0}	0.9282	{'precision': 0.8771897753282888, 'recall': 0.9011638368468469, 'f1-score': 0.8877789525374893, 'support': 27619.0}	{'precision': 0.9289188728159687, 'recall': 0.9282016003475868, 'f1-score': 0.9281081765661735, 'support': 27619.0}
No log	3.0	123	0.1926	{'precision': 0.7828618968386023, 'recall': 0.9022051773729626, 'f1-score': 0.8383073496659242, 'support': 1043.0}	{'precision': 0.9354406344242153, 'recall': 0.9654178674351584, 'f1-score': 0.950192874971636, 'support': 17350.0}	{'precision': 0.9381976266008695, 'recall': 0.8654888358985476, 'f1-score': 0.9003777414444383, 'support': 9226.0}	0.9296	{'precision': 0.8855000526212291, 'recall': 0.9110372935688895, 'f1-score': 0.896292655360666, 'support': 27619.0}	{'precision': 0.9305996331758, 'recall': 0.9296498787066875, 'f1-score': 0.9293271294770207, 'support': 27619.0}
No log	4.0	164	0.1886	{'precision': 0.8005115089514067, 'recall': 0.900287631831256, 'f1-score': 0.8474729241877257, 'support': 1043.0}	{'precision': 0.9321724709784411, 'recall': 0.9719308357348703, 'f1-score': 0.9516365688487585, 'support': 17350.0}	{'precision': 0.947941598851125, 'recall': 0.8585519184912205, 'f1-score': 0.9010351495848026, 'support': 9226.0}	0.9314	{'precision': 0.8935418595936575, 'recall': 0.9102567953524489, 'f1-score': 0.9000482142070956, 'support': 27619.0}	{'precision': 0.9324680497596853, 'recall': 0.9313516057786306, 'f1-score': 0.9307997762237281, 'support': 27619.0}

Framework versions

Transformers 4.37.2
Pytorch 2.2.0+cu121
Datasets 2.17.0
Tokenizers 0.15.2