README.md

metadata

license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
  - generated_from_trainer
datasets:
  - essays_su_g
metrics:
  - accuracy
model-index:
  - name: longformer-spans
    results:
      - task:
          name: Token Classification
          type: token-classification
        dataset:
          name: essays_su_g
          type: essays_su_g
          config: spans
          split: test
          args: spans
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.9420975312623168

longformer-spans

This model is a fine-tuned version of allenai/longformer-base-4096 on the essays_su_g dataset. It achieves the following results on the evaluation set:

• Loss: 0.1719
• B: {'precision': 0.852017937219731, 'recall': 0.8970727101038716, 'f1-score': 0.8739650413983441, 'support': 1059.0}
• I: {'precision': 0.9538791159224177, 'recall': 0.9626173541963016, 'f1-score': 0.9582283141230779, 'support': 17575.0}
• O: {'precision': 0.9301170236255244, 'recall': 0.9083557951482479, 'f1-score': 0.919107620138548, 'support': 9275.0}
• Accuracy: 0.9421
• Macro avg: {'precision': 0.912004692255891, 'recall': 0.9226819531494738, 'f1-score': 0.91710032521999, 'support': 27909.0}
• Weighted avg: {'precision': 0.9421171612017244, 'recall': 0.9420975312623168, 'f1-score': 0.9420299823117623, 'support': 27909.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss	B	I	O	Accuracy	Macro avg	Weighted avg
No log	1.0	41	0.2928	{'precision': 0.8236434108527132, 'recall': 0.40132200188857414, 'f1-score': 0.5396825396825397, 'support': 1059.0}	{'precision': 0.9120444175691276, 'recall': 0.9440113798008535, 'f1-score': 0.9277526142146172, 'support': 17575.0}	{'precision': 0.8748098239513149, 'recall': 0.8679245283018868, 'f1-score': 0.87135357471451, 'support': 9275.0}	0.8981	{'precision': 0.8701658841243853, 'recall': 0.7377526366637716, 'f1-score': 0.7795962428705557, 'support': 27909.0}	{'precision': 0.8963158883521046, 'recall': 0.8981332186749794, 'f1-score': 0.8942842957405419, 'support': 27909.0}
No log	2.0	82	0.1943	{'precision': 0.8109318996415771, 'recall': 0.8545797922568461, 'f1-score': 0.8321839080459771, 'support': 1059.0}	{'precision': 0.9395201599466845, 'recall': 0.9625604551920341, 'f1-score': 0.9509007616424496, 'support': 17575.0}	{'precision': 0.9288721975645841, 'recall': 0.88, 'f1-score': 0.9037758830694275, 'support': 9275.0}	0.9310	{'precision': 0.8931080857176151, 'recall': 0.8990467491496267, 'f1-score': 0.895620184252618, 'support': 27909.0}	{'precision': 0.9311022725713902, 'recall': 0.9310258339603712, 'f1-score': 0.9307350661061193, 'support': 27909.0}
No log	3.0	123	0.1853	{'precision': 0.799163179916318, 'recall': 0.9017941454202077, 'f1-score': 0.847382431233363, 'support': 1059.0}	{'precision': 0.9557297671201291, 'recall': 0.9433854907539118, 'f1-score': 0.9495175099504624, 'support': 17575.0}	{'precision': 0.9017723681400811, 'recall': 0.9106199460916442, 'f1-score': 0.9061745614505659, 'support': 9275.0}	0.9309	{'precision': 0.8855551050588427, 'recall': 0.9185998607552546, 'f1-score': 0.9010248342114636, 'support': 27909.0}	{'precision': 0.9318572209382959, 'recall': 0.9309183417535563, 'f1-score': 0.9312378547962845, 'support': 27909.0}
No log	4.0	164	0.1717	{'precision': 0.825491873396065, 'recall': 0.9112370160528801, 'f1-score': 0.8662477558348295, 'support': 1059.0}	{'precision': 0.9546820940389087, 'recall': 0.957724039829303, 'f1-score': 0.9562006476168834, 'support': 17575.0}	{'precision': 0.9242507410253595, 'recall': 0.9077088948787062, 'f1-score': 0.915905134899913, 'support': 9275.0}	0.9393	{'precision': 0.9014749028201111, 'recall': 0.9255566502536298, 'f1-score': 0.9127845127838753, 'support': 27909.0}	{'precision': 0.9396667497821657, 'recall': 0.9393385646207316, 'f1-score': 0.9393959970436956, 'support': 27909.0}
No log	5.0	205	0.1734	{'precision': 0.8358078602620087, 'recall': 0.9036827195467422, 'f1-score': 0.868421052631579, 'support': 1059.0}	{'precision': 0.9562692176289717, 'recall': 0.9555618776671408, 'f1-score': 0.9559154167971085, 'support': 17575.0}	{'precision': 0.9189306672462508, 'recall': 0.9116981132075471, 'f1-score': 0.915300102830546, 'support': 9275.0}	0.9390	{'precision': 0.903669248379077, 'recall': 0.9236475701404768, 'f1-score': 0.9132121907530778, 'support': 27909.0}	{'precision': 0.9392896184942356, 'recall': 0.9390160880002867, 'f1-score': 0.9390977748647152, 'support': 27909.0}
No log	6.0	246	0.1677	{'precision': 0.8308759757155247, 'recall': 0.9046270066100094, 'f1-score': 0.8661844484629294, 'support': 1059.0}	{'precision': 0.9521587587137396, 'recall': 0.9636984352773826, 'f1-score': 0.9578938438480898, 'support': 17575.0}	{'precision': 0.9325379125780553, 'recall': 0.9016711590296496, 'f1-score': 0.9168448171901551, 'support': 9275.0}	0.9408	{'precision': 0.9051908823357732, 'recall': 0.9233322003056804, 'f1-score': 0.9136410365003914, 'support': 27909.0}	{'precision': 0.9410361167307384, 'recall': 0.9408434555161418, 'f1-score': 0.9407721278437462, 'support': 27909.0}
No log	7.0	287	0.1719	{'precision': 0.852017937219731, 'recall': 0.8970727101038716, 'f1-score': 0.8739650413983441, 'support': 1059.0}	{'precision': 0.9538791159224177, 'recall': 0.9626173541963016, 'f1-score': 0.9582283141230779, 'support': 17575.0}	{'precision': 0.9301170236255244, 'recall': 0.9083557951482479, 'f1-score': 0.919107620138548, 'support': 9275.0}	0.9421	{'precision': 0.912004692255891, 'recall': 0.9226819531494738, 'f1-score': 0.91710032521999, 'support': 27909.0}	{'precision': 0.9421171612017244, 'recall': 0.9420975312623168, 'f1-score': 0.9420299823117623, 'support': 27909.0}

Framework versions

• Transformers 4.37.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.0
• Tokenizers 0.15.2