Russian NER model fine-tuned on RURED2. https://github.com/denis-gordeev/rured2
If you have any questions, message me at https://t.me/nlp_party This model outputs multiple possible labels for a single token. So for proper usage you can use it like in the following code:
import torch
from torch import nn
from transformers import (AutoTokenizer, AutoModelForTokenClassification,
TrainingArguments, Trainer)
model_name = "denis-gordeev/rured2-ner-microsoft-mdeberta-v3-base"
model = AutoModelForTokenClassification.from_pretrained(
model_name).to('cuda')
tokenizer = AutoTokenizer.from_pretrained(model_name)
def predict(text:str, glue_tokens=False, output_together=True, glue_words=True):
sigmoid = nn.Sigmoid()
tokenized = tokenizer(text)
input_ids = torch.tensor(
[tokenized["input_ids"]], dtype=torch.long
).to("cuda")
token_type_ids = torch.tensor(
[tokenized["token_type_ids"]], dtype=torch.long
).to("cuda")
attention_mask = torch.tensor(
[tokenized["attention_mask"]], dtype=torch.long
).to("cuda")
preds = model(**{"input_ids": input_ids, "token_type_ids": token_type_ids, "attention_mask": attention_mask})
logits = sigmoid(preds.logits)
output_tokens = []
output_preds = []
id_to_label = {int(k): v for k, v in model.config.id2label.items()}
for i, token in enumerate(input_ids[0]):
if token > 3:
class_ids = (logits[0][i] > 0.5).nonzero()
if class_ids.shape[0] >= 1:
class_names = [id_to_label[int(cl)] for cl in class_ids]
else:
class_names = [id_to_label[int(logits[0][i].argmax())]]
converted_token = tokenizer.convert_ids_to_tokens([token])[0]
new_word_bool = converted_token.startswith("▁")
converted_token = converted_token.replace("▁", "")
if glue_words and not(new_word_bool) and output_tokens:
output_tokens[-1] += converted_token
else:
output_tokens.append(converted_token)
output_preds.append(class_names)
else:
class_names = []
if output_together:
return [[output_tokens[t_i], output_preds[t_i]] for t_i in range(len(output_tokens))]
return output_tokens, output_preds
denis-gordeev/rured2-ner-microsoft-mdeberta-v3-base
This model is a fine-tuned version of microsoft/mdeberta-v3-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.0096
- F1 Micro: 0.5837
- O F1 Micro: 0.6370
- O Recall Micro: 0.9242
- O Precision Micro: 0.4860
- B-person F1 Micro: 0.9639
- B-person Recall Micro: 0.9816
- B-person Precision Micro: 0.9468
- B-norp F1 Micro: 0.6190
- B-norp Recall Micro: 0.8667
- B-norp Precision Micro: 0.4815
- B-commodity F1 Micro: 0.7553
- B-commodity Recall Micro: 0.9470
- B-commodity Precision Micro: 0.6281
- B-date F1 Micro: 0.8386
- B-date Recall Micro: 0.8471
- B-date Precision Micro: 0.8304
- I-date F1 Micro: 0.6419
- I-date Recall Micro: 0.9492
- I-date Precision Micro: 0.4849
- B-country F1 Micro: 0.6152
- B-country Recall Micro: 0.9765
- B-country Precision Micro: 0.4490
- B-economic Sector F1 Micro: 0.5576
- B-economic Sector Recall Micro: 0.5897
- B-economic Sector Precision Micro: 0.5287
- I-economic Sector F1 Micro: 0.2517
- I-economic Sector Recall Micro: 0.6667
- I-economic Sector Precision Micro: 0.1551
- B-news Source F1 Micro: 0.7988
- B-news Source Recall Micro: 0.8327
- B-news Source Precision Micro: 0.7677
- B-profession F1 Micro: 0.8088
- B-profession Recall Micro: 0.9464
- B-profession Precision Micro: 0.7061
- I-news Source F1 Micro: 0.4808
- I-news Source Recall Micro: 0.8400
- I-news Source Precision Micro: 0.3368
- I-person F1 Micro: 0.3381
- I-person Recall Micro: 0.996
- I-person Precision Micro: 0.2036
- B-organization F1 Micro: 0.8350
- B-organization Recall Micro: 0.8993
- B-organization Precision Micro: 0.7794
- I-profession F1 Micro: 0.2462
- I-profession Recall Micro: 0.8030
- I-profession Precision Micro: 0.1454
- B-event F1 Micro: 0.5658
- B-event Recall Micro: 0.5436
- B-event Precision Micro: 0.5899
- B-city F1 Micro: 0.625
- B-city Recall Micro: 0.8904
- B-city Precision Micro: 0.4815
- B-gpe F1 Micro: 0.6760
- B-gpe Recall Micro: 0.9380
- B-gpe Precision Micro: 0.5284
- I-event F1 Micro: 0.2577
- I-event Recall Micro: 0.3776
- I-event Precision Micro: 0.1956
- B-group F1 Micro: 0.6667
- B-group Recall Micro: 0.75
- B-group Precision Micro: 0.6
- B-ordinal F1 Micro: 0.5306
- B-ordinal Recall Micro: 0.8125
- B-ordinal Precision Micro: 0.3939
- B-product F1 Micro: 0.6683
- B-product Recall Micro: 0.8232
- B-product Precision Micro: 0.5625
- I-organization F1 Micro: 0.3128
- I-organization Recall Micro: 0.8425
- I-organization Precision Micro: 0.1921
- B-money F1 Micro: 0.8530
- B-money Recall Micro: 0.8947
- B-money Precision Micro: 0.8151
- I-money F1 Micro: 0.6259
- I-money Recall Micro: 0.9644
- I-money Precision Micro: 0.4632
- B-currency F1 Micro: 0.7441
- B-currency Recall Micro: 0.9658
- B-currency Precision Micro: 0.6052
- B-percent F1 Micro: 0.8639
- B-percent Recall Micro: 0.8902
- B-percent Precision Micro: 0.8391
- I-percent F1 Micro: 0.6995
- I-percent Recall Micro: 0.9846
- I-percent Precision Micro: 0.5424
- I-group F1 Micro: 0.1844
- I-group Recall Micro: 0.4836
- I-group Precision Micro: 0.1139
- B-cardinal F1 Micro: 0.6903
- B-cardinal Recall Micro: 0.7358
- B-cardinal Precision Micro: 0.65
- B-law F1 Micro: 0.3704
- B-law Recall Micro: 0.3571
- B-law Precision Micro: 0.3846
- I-law F1 Micro: 0.3246
- I-law Recall Micro: 0.3936
- I-law Precision Micro: 0.2761
- B-fac F1 Micro: 0.6910
- B-fac Recall Micro: 0.6910
- B-fac Precision Micro: 0.6910
- I-fac F1 Micro: 0.3007
- I-fac Recall Micro: 0.7151
- I-fac Precision Micro: 0.1904
- B-age F1 Micro: 0.8649
- B-age Recall Micro: 0.7619
- B-age Precision Micro: 1.0
- I-city F1 Micro: 0.1047
- I-city Recall Micro: 0.6429
- I-city Precision Micro: 0.0570
- B-work Of Art F1 Micro: 0.3158
- B-work Of Art Recall Micro: 0.375
- B-work Of Art Precision Micro: 0.2727
- I-work Of Art F1 Micro: 0.3721
- I-work Of Art Recall Micro: 0.5
- I-work Of Art Precision Micro: 0.2963
- B-region F1 Micro: 0.8070
- B-region Recall Micro: 0.7731
- B-region Precision Micro: 0.8440
- I-region F1 Micro: 0.2817
- I-region Recall Micro: 0.8197
- I-region Precision Micro: 0.1701
- I-cardinal F1 Micro: 0.3851
- I-cardinal Recall Micro: 0.4831
- I-cardinal Precision Micro: 0.3202
- I-currency F1 Micro: 0.0
- I-currency Recall Micro: 0.0
- I-currency Precision Micro: 0.0
- B-quantity F1 Micro: 0.7311
- B-quantity Recall Micro: 0.7311
- B-quantity Precision Micro: 0.7311
- I-quantity F1 Micro: 0.4889
- I-quantity Recall Micro: 0.7989
- I-quantity Precision Micro: 0.3522
- B-crime F1 Micro: 0.3736
- B-crime Recall Micro: 0.4048
- B-crime Precision Micro: 0.3469
- I-crime F1 Micro: 0.3245
- I-crime Recall Micro: 0.5648
- I-crime Precision Micro: 0.2276
- B-trade Agreement F1 Micro: 0.7170
- B-trade Agreement Recall Micro: 0.7037
- B-trade Agreement Precision Micro: 0.7308
- B-nationality F1 Micro: 0.0
- B-nationality Recall Micro: 0.0
- B-nationality Precision Micro: 0.0
- B-family F1 Micro: 0.5
- B-family Recall Micro: 0.8889
- B-family Precision Micro: 0.3478
- I-family F1 Micro: 0.0
- I-family Recall Micro: 0.0
- I-family Precision Micro: 0.0
- I-product F1 Micro: 0.2021
- I-product Recall Micro: 0.6824
- I-product Precision Micro: 0.1186
- B-time F1 Micro: 0.6538
- B-time Recall Micro: 0.6296
- B-time Precision Micro: 0.68
- I-time F1 Micro: 0.6118
- I-time Recall Micro: 0.9811
- I-time Precision Micro: 0.4444
- I-commodity F1 Micro: 0.0444
- I-commodity Recall Micro: 0.1667
- I-commodity Precision Micro: 0.0256
- B-application F1 Micro: 0.0
- B-application Recall Micro: 0.0
- B-application Precision Micro: 0.0
- I-application F1 Micro: 0.0
- I-application Recall Micro: 0.0
- I-application Precision Micro: 0.0
- I-country F1 Micro: 0.1695
- I-country Recall Micro: 0.7895
- I-country Precision Micro: 0.0949
- B-award F1 Micro: 0.5455
- B-award Recall Micro: 0.4615
- B-award Precision Micro: 0.6667
- I-award F1 Micro: 0.4459
- I-award Recall Micro: 0.8049
- I-award Precision Micro: 0.3084
- I-gpe F1 Micro: 0.3284
- I-gpe Recall Micro: 0.9167
- I-gpe Precision Micro: 0.2
- B-location F1 Micro: 0.4885
- B-location Recall Micro: 0.5161
- B-location Precision Micro: 0.4638
- I-location F1 Micro: 0.3189
- I-location Recall Micro: 0.6316
- I-location Precision Micro: 0.2133
- I-ordinal F1 Micro: 0.5
- I-ordinal Recall Micro: 0.4
- I-ordinal Precision Micro: 0.6667
- I-trade Agreement F1 Micro: 0.1163
- I-trade Agreement Recall Micro: 0.3846
- I-trade Agreement Precision Micro: 0.0685
- B-religion F1 Micro: 0.0
- B-religion Recall Micro: 0.0
- B-religion Precision Micro: 0.0
- I-age F1 Micro: 0.4324
- I-age Recall Micro: 0.5714
- I-age Precision Micro: 0.3478
- B-investment Program F1 Micro: 0.0
- B-investment Program Recall Micro: 0.0
- B-investment Program Precision Micro: 0.0
- I-investment Program F1 Micro: 0.0
- I-investment Program Recall Micro: 0.0
- I-investment Program Precision Micro: 0.0
- B-borough F1 Micro: 0.7059
- B-borough Recall Micro: 0.6667
- B-borough Precision Micro: 0.75
- B-price F1 Micro: 0.0
- B-price Recall Micro: 0.0
- B-price Precision Micro: 0.0
- I-price F1 Micro: 0.0
- I-price Recall Micro: 0.0
- I-price Precision Micro: 0.0
- B-character F1 Micro: 0.0
- B-character Recall Micro: 0.0
- B-character Precision Micro: 0.0
- I-character F1 Micro: 0.0
- I-character Recall Micro: 0.0
- I-character Precision Micro: 0.0
- B-website F1 Micro: 0.0
- B-website Recall Micro: 0.0
- B-website Precision Micro: 0.0
- B-street F1 Micro: 0.4000
- B-street Recall Micro: 0.4286
- B-street Precision Micro: 0.375
- I-street F1 Micro: 0.3256
- I-street Recall Micro: 1.0
- I-street Precision Micro: 0.1944
- B-village F1 Micro: 0.6667
- B-village Recall Micro: 0.7
- B-village Precision Micro: 0.6364
- I-village F1 Micro: 0.2222
- I-village Recall Micro: 0.875
- I-village Precision Micro: 0.1273
- B-disease F1 Micro: 0.5965
- B-disease Recall Micro: 0.7083
- B-disease Precision Micro: 0.5152
- I-disease F1 Micro: 0.3704
- I-disease Recall Micro: 0.7812
- I-disease Precision Micro: 0.2427
- B-penalty F1 Micro: 0.1579
- B-penalty Recall Micro: 0.1579
- B-penalty Precision Micro: 0.1579
- I-penalty F1 Micro: 0.1674
- I-penalty Recall Micro: 0.3175
- I-penalty Precision Micro: 0.1136
- B-weapon F1 Micro: 0.6715
- B-weapon Recall Micro: 0.7302
- B-weapon Precision Micro: 0.6216
- I-weapon F1 Micro: 0.2455
- I-weapon Recall Micro: 0.5965
- I-weapon Precision Micro: 0.1545
- I-borough F1 Micro: 0.4091
- I-borough Recall Micro: 0.6923
- I-borough Precision Micro: 0.2903
- B-vehicle F1 Micro: 0.6349
- B-vehicle Recall Micro: 0.5882
- B-vehicle Precision Micro: 0.6897
- I-vehicle F1 Micro: 0.4174
- I-vehicle Recall Micro: 0.7273
- I-vehicle Precision Micro: 0.2927
- B-language F1 Micro: 0.0
- B-language Recall Micro: 0.0
- B-language Precision Micro: 0.0
- I-language F1 Micro: 0.0
- I-language Recall Micro: 0.0
- I-language Precision Micro: 0.0
- B-house F1 Micro: 0.0
- B-house Recall Micro: 0.0
- B-house Precision Micro: 0.0
- I-norp F1 Micro: 0.0
- I-norp Recall Micro: 0.0
- I-norp Precision Micro: 0.0
- I-house F1 Micro: 0.0
- I-house Recall Micro: 0.0
- I-house Precision Micro: 0.0
- I-website F1 Micro: 0.0
- I-website Recall Micro: 0.0
- I-website Precision Micro: 0.0
- F1 Macro: 0.3969
- Recall Macro: 0.5603
- Precision Macro: 0.3447
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10000
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | F1 Micro | O F1 Micro | O Recall Micro | O Precision Micro | B-person F1 Micro | B-person Recall Micro | B-person Precision Micro | B-norp F1 Micro | B-norp Recall Micro | B-norp Precision Micro | B-commodity F1 Micro | B-commodity Recall Micro | B-commodity Precision Micro | B-date F1 Micro | B-date Recall Micro | B-date Precision Micro | I-date F1 Micro | I-date Recall Micro | I-date Precision Micro | B-country F1 Micro | B-country Recall Micro | B-country Precision Micro | B-economic Sector F1 Micro | B-economic Sector Recall Micro | B-economic Sector Precision Micro | I-economic Sector F1 Micro | I-economic Sector Recall Micro | I-economic Sector Precision Micro | B-news Source F1 Micro | B-news Source Recall Micro | B-news Source Precision Micro | B-profession F1 Micro | B-profession Recall Micro | B-profession Precision Micro | I-news Source F1 Micro | I-news Source Recall Micro | I-news Source Precision Micro | I-person F1 Micro | I-person Recall Micro | I-person Precision Micro | B-organization F1 Micro | B-organization Recall Micro | B-organization Precision Micro | I-profession F1 Micro | I-profession Recall Micro | I-profession Precision Micro | B-event F1 Micro | B-event Recall Micro | B-event Precision Micro | B-city F1 Micro | B-city Recall Micro | B-city Precision Micro | B-gpe F1 Micro | B-gpe Recall Micro | B-gpe Precision Micro | I-event F1 Micro | I-event Recall Micro | I-event Precision Micro | B-group F1 Micro | B-group Recall Micro | B-group Precision Micro | B-ordinal F1 Micro | B-ordinal Recall Micro | B-ordinal Precision Micro | B-product F1 Micro | B-product Recall Micro | B-product Precision Micro | I-organization F1 Micro | I-organization Recall Micro | I-organization Precision Micro | B-money F1 Micro | B-money Recall Micro | B-money Precision Micro | I-money F1 Micro | I-money Recall Micro | I-money Precision Micro | B-currency F1 Micro | B-currency Recall Micro | B-currency Precision Micro | B-percent F1 Micro | B-percent Recall Micro | B-percent Precision Micro | I-percent F1 Micro | I-percent Recall Micro | I-percent Precision Micro | I-group F1 Micro | I-group Recall Micro | I-group Precision Micro | B-cardinal F1 Micro | B-cardinal Recall Micro | B-cardinal Precision Micro | B-law F1 Micro | B-law Recall Micro | B-law Precision Micro | I-law F1 Micro | I-law Recall Micro | I-law Precision Micro | B-fac F1 Micro | B-fac Recall Micro | B-fac Precision Micro | I-fac F1 Micro | I-fac Recall Micro | I-fac Precision Micro | B-age F1 Micro | B-age Recall Micro | B-age Precision Micro | I-city F1 Micro | I-city Recall Micro | I-city Precision Micro | B-work Of Art F1 Micro | B-work Of Art Recall Micro | B-work Of Art Precision Micro | I-work Of Art F1 Micro | I-work Of Art Recall Micro | I-work Of Art Precision Micro | B-region F1 Micro | B-region Recall Micro | B-region Precision Micro | I-region F1 Micro | I-region Recall Micro | I-region Precision Micro | I-cardinal F1 Micro | I-cardinal Recall Micro | I-cardinal Precision Micro | I-currency F1 Micro | I-currency Recall Micro | I-currency Precision Micro | B-quantity F1 Micro | B-quantity Recall Micro | B-quantity Precision Micro | I-quantity F1 Micro | I-quantity Recall Micro | I-quantity Precision Micro | B-crime F1 Micro | B-crime Recall Micro | B-crime Precision Micro | I-crime F1 Micro | I-crime Recall Micro | I-crime Precision Micro | B-trade Agreement F1 Micro | B-trade Agreement Recall Micro | B-trade Agreement Precision Micro | B-nationality F1 Micro | B-nationality Recall Micro | B-nationality Precision Micro | B-family F1 Micro | B-family Recall Micro | B-family Precision Micro | I-family F1 Micro | I-family Recall Micro | I-family Precision Micro | I-product F1 Micro | I-product Recall Micro | I-product Precision Micro | B-time F1 Micro | B-time Recall Micro | B-time Precision Micro | I-time F1 Micro | I-time Recall Micro | I-time Precision Micro | I-commodity F1 Micro | I-commodity Recall Micro | I-commodity Precision Micro | B-application F1 Micro | B-application Recall Micro | B-application Precision Micro | I-application F1 Micro | I-application Recall Micro | I-application Precision Micro | I-country F1 Micro | I-country Recall Micro | I-country Precision Micro | B-award F1 Micro | B-award Recall Micro | B-award Precision Micro | I-award F1 Micro | I-award Recall Micro | I-award Precision Micro | I-gpe F1 Micro | I-gpe Recall Micro | I-gpe Precision Micro | B-location F1 Micro | B-location Recall Micro | B-location Precision Micro | I-location F1 Micro | I-location Recall Micro | I-location Precision Micro | I-ordinal F1 Micro | I-ordinal Recall Micro | I-ordinal Precision Micro | I-trade Agreement F1 Micro | I-trade Agreement Recall Micro | I-trade Agreement Precision Micro | B-religion F1 Micro | B-religion Recall Micro | B-religion Precision Micro | I-age F1 Micro | I-age Recall Micro | I-age Precision Micro | B-investment Program F1 Micro | B-investment Program Recall Micro | B-investment Program Precision Micro | I-investment Program F1 Micro | I-investment Program Recall Micro | I-investment Program Precision Micro | B-borough F1 Micro | B-borough Recall Micro | B-borough Precision Micro | B-price F1 Micro | B-price Recall Micro | B-price Precision Micro | I-price F1 Micro | I-price Recall Micro | I-price Precision Micro | B-character F1 Micro | B-character Recall Micro | B-character Precision Micro | I-character F1 Micro | I-character Recall Micro | I-character Precision Micro | B-website F1 Micro | B-website Recall Micro | B-website Precision Micro | B-street F1 Micro | B-street Recall Micro | B-street Precision Micro | I-street F1 Micro | I-street Recall Micro | I-street Precision Micro | B-village F1 Micro | B-village Recall Micro | B-village Precision Micro | I-village F1 Micro | I-village Recall Micro | I-village Precision Micro | B-disease F1 Micro | B-disease Recall Micro | B-disease Precision Micro | I-disease F1 Micro | I-disease Recall Micro | I-disease Precision Micro | B-penalty F1 Micro | B-penalty Recall Micro | B-penalty Precision Micro | I-penalty F1 Micro | I-penalty Recall Micro | I-penalty Precision Micro | B-weapon F1 Micro | B-weapon Recall Micro | B-weapon Precision Micro | I-weapon F1 Micro | I-weapon Recall Micro | I-weapon Precision Micro | I-borough F1 Micro | I-borough Recall Micro | I-borough Precision Micro | B-vehicle F1 Micro | B-vehicle Recall Micro | B-vehicle Precision Micro | I-vehicle F1 Micro | I-vehicle Recall Micro | I-vehicle Precision Micro | B-language F1 Micro | B-language Recall Micro | B-language Precision Micro | I-language F1 Micro | I-language Recall Micro | I-language Precision Micro | B-house F1 Micro | B-house Recall Micro | B-house Precision Micro | I-norp F1 Micro | I-norp Recall Micro | I-norp Precision Micro | I-house F1 Micro | I-house Recall Micro | I-house Precision Micro | I-website F1 Micro | I-website Recall Micro | I-website Precision Micro | F1 Macro | Recall Macro | Precision Macro |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.0033 | 1.0 | 3014 | 0.0092 | 0.5876 | 0.6385 | 0.9365 | 0.4844 | 0.9705 | 0.9816 | 0.9596 | 0.6118 | 0.8667 | 0.4727 | 0.7873 | 0.9394 | 0.6776 | 0.8436 | 0.8571 | 0.8304 | 0.6416 | 0.9669 | 0.4801 | 0.6229 | 0.9831 | 0.4559 | 0.6024 | 0.6410 | 0.5682 | 0.2491 | 0.5965 | 0.1574 | 0.7954 | 0.8306 | 0.7630 | 0.8659 | 0.9184 | 0.8191 | 0.4757 | 0.8461 | 0.3309 | 0.3331 | 0.996 | 0.2 | 0.8277 | 0.9038 | 0.7633 | 0.2675 | 0.7652 | 0.1621 | 0.5617 | 0.5291 | 0.5987 | 0.7347 | 0.8630 | 0.6396 | 0.6989 | 0.9535 | 0.5516 | 0.2461 | 0.3922 | 0.1793 | 0.7073 | 0.7143 | 0.7004 | 0.592 | 0.7708 | 0.4805 | 0.7213 | 0.8049 | 0.6535 | 0.3127 | 0.8040 | 0.1941 | 0.88 | 0.9098 | 0.8521 | 0.6312 | 0.9502 | 0.4726 | 0.7622 | 0.9658 | 0.6295 | 0.8824 | 0.9146 | 0.8523 | 0.6952 | 1.0 | 0.5328 | 0.1762 | 0.4426 | 0.1100 | 0.6688 | 0.6604 | 0.6774 | 0.3846 | 0.3571 | 0.4167 | 0.2473 | 0.3617 | 0.1878 | 0.7348 | 0.7253 | 0.7445 | 0.3085 | 0.6977 | 0.1980 | 0.8333 | 0.7143 | 1.0 | 0.1010 | 0.7143 | 0.0543 | 0.3333 | 0.25 | 0.5 | 0.4242 | 0.4375 | 0.4118 | 0.7729 | 0.8151 | 0.7348 | 0.2865 | 0.8033 | 0.1744 | 0.4196 | 0.5085 | 0.3571 | 0.0 | 0.0 | 0.0 | 0.7177 | 0.7479 | 0.6899 | 0.4931 | 0.7933 | 0.3577 | 0.3789 | 0.4286 | 0.3396 | 0.3341 | 0.6574 | 0.2240 | 0.6415 | 0.6296 | 0.6538 | 0.0 | 0.0 | 0.0 | 0.4737 | 1.0 | 0.3103 | 0.0 | 0.0 | 0.0 | 0.2270 | 0.5647 | 0.1420 | 0.6667 | 0.6667 | 0.6667 | 0.5854 | 0.9057 | 0.4324 | 0.0741 | 0.1667 | 0.0476 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1867 | 0.7368 | 0.1069 | 0.4444 | 0.3077 | 0.8 | 0.4706 | 0.7805 | 0.3368 | 0.2619 | 0.9167 | 0.1528 | 0.4878 | 0.4839 | 0.4918 | 0.2997 | 0.6053 | 0.1991 | 0.25 | 0.2 | 0.3333 | 0.1154 | 0.2308 | 0.0769 | 0.0 | 0.0 | 0.0 | 0.3750 | 0.6429 | 0.2647 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.6667 | 0.6667 | 0.6667 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.5 | 0.5714 | 0.4444 | 0.3333 | 1.0 | 0.2 | 0.6222 | 0.7 | 0.56 | 0.2424 | 1.0 | 0.1379 | 0.5778 | 0.5417 | 0.6190 | 0.3425 | 0.7812 | 0.2193 | 0.1212 | 0.1053 | 0.1429 | 0.1847 | 0.3651 | 0.1237 | 0.7015 | 0.7460 | 0.6620 | 0.2256 | 0.5263 | 0.1435 | 0.4045 | 0.6923 | 0.2857 | 0.7143 | 0.7353 | 0.6944 | 0.3826 | 0.6667 | 0.2683 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.3969 | 0.5513 | 0.3515 |
| 0.0032 | 2.0 | 6028 | 0.0096 | 0.5837 | 0.6370 | 0.9242 | 0.4860 | 0.9639 | 0.9816 | 0.9468 | 0.6190 | 0.8667 | 0.4815 | 0.7553 | 0.9470 | 0.6281 | 0.8386 | 0.8471 | 0.8304 | 0.6419 | 0.9492 | 0.4849 | 0.6152 | 0.9765 | 0.4490 | 0.5576 | 0.5897 | 0.5287 | 0.2517 | 0.6667 | 0.1551 | 0.7988 | 0.8327 | 0.7677 | 0.8088 | 0.9464 | 0.7061 | 0.4808 | 0.8400 | 0.3368 | 0.3381 | 0.996 | 0.2036 | 0.8350 | 0.8993 | 0.7794 | 0.2462 | 0.8030 | 0.1454 | 0.5658 | 0.5436 | 0.5899 | 0.625 | 0.8904 | 0.4815 | 0.6760 | 0.9380 | 0.5284 | 0.2577 | 0.3776 | 0.1956 | 0.6667 | 0.75 | 0.6 | 0.5306 | 0.8125 | 0.3939 | 0.6683 | 0.8232 | 0.5625 | 0.3128 | 0.8425 | 0.1921 | 0.8530 | 0.8947 | 0.8151 | 0.6259 | 0.9644 | 0.4632 | 0.7441 | 0.9658 | 0.6052 | 0.8639 | 0.8902 | 0.8391 | 0.6995 | 0.9846 | 0.5424 | 0.1844 | 0.4836 | 0.1139 | 0.6903 | 0.7358 | 0.65 | 0.3704 | 0.3571 | 0.3846 | 0.3246 | 0.3936 | 0.2761 | 0.6910 | 0.6910 | 0.6910 | 0.3007 | 0.7151 | 0.1904 | 0.8649 | 0.7619 | 1.0 | 0.1047 | 0.6429 | 0.0570 | 0.3158 | 0.375 | 0.2727 | 0.3721 | 0.5 | 0.2963 | 0.8070 | 0.7731 | 0.8440 | 0.2817 | 0.8197 | 0.1701 | 0.3851 | 0.4831 | 0.3202 | 0.0 | 0.0 | 0.0 | 0.7311 | 0.7311 | 0.7311 | 0.4889 | 0.7989 | 0.3522 | 0.3736 | 0.4048 | 0.3469 | 0.3245 | 0.5648 | 0.2276 | 0.7170 | 0.7037 | 0.7308 | 0.0 | 0.0 | 0.0 | 0.5 | 0.8889 | 0.3478 | 0.0 | 0.0 | 0.0 | 0.2021 | 0.6824 | 0.1186 | 0.6538 | 0.6296 | 0.68 | 0.6118 | 0.9811 | 0.4444 | 0.0444 | 0.1667 | 0.0256 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1695 | 0.7895 | 0.0949 | 0.5455 | 0.4615 | 0.6667 | 0.4459 | 0.8049 | 0.3084 | 0.3284 | 0.9167 | 0.2 | 0.4885 | 0.5161 | 0.4638 | 0.3189 | 0.6316 | 0.2133 | 0.5 | 0.4 | 0.6667 | 0.1163 | 0.3846 | 0.0685 | 0.0 | 0.0 | 0.0 | 0.4324 | 0.5714 | 0.3478 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.7059 | 0.6667 | 0.75 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.4000 | 0.4286 | 0.375 | 0.3256 | 1.0 | 0.1944 | 0.6667 | 0.7 | 0.6364 | 0.2222 | 0.875 | 0.1273 | 0.5965 | 0.7083 | 0.5152 | 0.3704 | 0.7812 | 0.2427 | 0.1579 | 0.1579 | 0.1579 | 0.1674 | 0.3175 | 0.1136 | 0.6715 | 0.7302 | 0.6216 | 0.2455 | 0.5965 | 0.1545 | 0.4091 | 0.6923 | 0.2903 | 0.6349 | 0.5882 | 0.6897 | 0.4174 | 0.7273 | 0.2927 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.3969 | 0.5603 | 0.3447 |
| 0.0029 | 3.0 | 9042 | 0.0102 | 0.5857 | 0.6369 | 0.9367 | 0.4824 | 0.9555 | 0.9862 | 0.9266 | 0.5909 | 0.8667 | 0.4483 | 0.7590 | 0.9545 | 0.63 | 0.8374 | 0.8551 | 0.8205 | 0.6347 | 0.9752 | 0.4704 | 0.6097 | 0.9817 | 0.4422 | 0.6286 | 0.7051 | 0.5670 | 0.2581 | 0.6316 | 0.1622 | 0.7871 | 0.8347 | 0.7446 | 0.8664 | 0.9371 | 0.8056 | 0.4789 | 0.8564 | 0.3324 | 0.3383 | 0.996 | 0.2038 | 0.8071 | 0.8929 | 0.7364 | 0.2440 | 0.8106 | 0.1436 | 0.5397 | 0.4738 | 0.6269 | 0.7014 | 0.8767 | 0.5845 | 0.6503 | 0.9225 | 0.5021 | 0.2354 | 0.3306 | 0.1828 | 0.6799 | 0.75 | 0.6217 | 0.592 | 0.7708 | 0.4805 | 0.7163 | 0.7622 | 0.6757 | 0.3133 | 0.8077 | 0.1944 | 0.8278 | 0.8496 | 0.8071 | 0.6187 | 0.9644 | 0.4555 | 0.7749 | 0.9315 | 0.6634 | 0.8606 | 0.8659 | 0.8554 | 0.688 | 0.9923 | 0.5265 | 0.1872 | 0.4918 | 0.1156 | 0.6826 | 0.7170 | 0.6514 | 0.4286 | 0.4286 | 0.4286 | 0.2900 | 0.4149 | 0.2229 | 0.7124 | 0.6910 | 0.7352 | 0.3132 | 0.7093 | 0.2010 | 0.8649 | 0.7619 | 1.0 | 0.0988 | 0.5714 | 0.0541 | 0.1429 | 0.125 | 0.1667 | 0.3889 | 0.4375 | 0.35 | 0.7317 | 0.7563 | 0.7087 | 0.2889 | 0.8525 | 0.1739 | 0.4224 | 0.5763 | 0.3333 | 0.0 | 0.0 | 0.0 | 0.7265 | 0.7143 | 0.7391 | 0.4936 | 0.7598 | 0.3656 | 0.3564 | 0.4286 | 0.3051 | 0.2857 | 0.6944 | 0.1799 | 0.6471 | 0.8148 | 0.5366 | 0.0 | 0.0 | 0.0 | 0.5455 | 1.0 | 0.375 | 0.0 | 0.0 | 0.0 | 0.2392 | 0.5529 | 0.1526 | 0.6415 | 0.6296 | 0.6538 | 0.6 | 0.9057 | 0.4486 | 0.1176 | 0.6667 | 0.0645 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1436 | 0.7368 | 0.0795 | 0.4286 | 0.4615 | 0.4 | 0.4595 | 0.8293 | 0.3178 | 0.3548 | 0.9167 | 0.22 | 0.5197 | 0.5323 | 0.5077 | 0.3300 | 0.6447 | 0.2217 | 0.4444 | 0.4 | 0.5 | 0.0870 | 0.2308 | 0.0536 | 0.0 | 0.0 | 0.0 | 0.3902 | 0.5714 | 0.2963 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.6471 | 0.6111 | 0.6875 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.4000 | 0.4286 | 0.375 | 0.3256 | 1.0 | 0.1944 | 0.6829 | 0.7 | 0.6667 | 0.2258 | 0.875 | 0.1296 | 0.4928 | 0.7083 | 0.3778 | 0.3521 | 0.7812 | 0.2273 | 0.15 | 0.1579 | 0.1429 | 0.2128 | 0.3175 | 0.16 | 0.7059 | 0.7619 | 0.6575 | 0.2581 | 0.7018 | 0.1581 | 0.3956 | 0.6923 | 0.2769 | 0.7143 | 0.7353 | 0.6944 | 0.4915 | 0.8788 | 0.3412 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.3958 | 0.5647 | 0.3398 |
| 0.0026 | 4.0 | 12056 | 0.0105 | 0.5820 | 0.6363 | 0.9236 | 0.4854 | 0.9661 | 0.9839 | 0.9490 | 0.6118 | 0.8667 | 0.4727 | 0.7568 | 0.9545 | 0.6269 | 0.8481 | 0.8370 | 0.8595 | 0.6525 | 0.9587 | 0.4945 | 0.6156 | 0.9844 | 0.4478 | 0.6125 | 0.6282 | 0.5976 | 0.2618 | 0.6316 | 0.1651 | 0.8011 | 0.8448 | 0.7618 | 0.8447 | 0.9254 | 0.7769 | 0.4769 | 0.8303 | 0.3346 | 0.3379 | 0.996 | 0.2034 | 0.8171 | 0.8938 | 0.7525 | 0.2480 | 0.8182 | 0.1461 | 0.5205 | 0.5174 | 0.5235 | 0.6432 | 0.8767 | 0.5079 | 0.6821 | 0.9147 | 0.5438 | 0.2270 | 0.4441 | 0.1525 | 0.6405 | 0.7460 | 0.5612 | 0.6016 | 0.7708 | 0.4933 | 0.7299 | 0.7744 | 0.6902 | 0.3177 | 0.8059 | 0.1978 | 0.8699 | 0.8797 | 0.8603 | 0.6308 | 0.9395 | 0.4748 | 0.7637 | 0.9521 | 0.6376 | 0.8571 | 0.8780 | 0.8372 | 0.688 | 0.9923 | 0.5265 | 0.1887 | 0.4918 | 0.1167 | 0.6879 | 0.7484 | 0.6364 | 0.3333 | 0.3571 | 0.3125 | 0.2439 | 0.3723 | 0.1813 | 0.6925 | 0.6910 | 0.6940 | 0.3186 | 0.7326 | 0.2036 | 0.8333 | 0.7143 | 1.0 | 0.1046 | 0.5714 | 0.0576 | 0.2857 | 0.25 | 0.3333 | 0.3333 | 0.4375 | 0.2692 | 0.7583 | 0.7647 | 0.7521 | 0.2985 | 0.8197 | 0.1825 | 0.3416 | 0.4661 | 0.2696 | 0.0 | 0.0 | 0.0 | 0.7113 | 0.7143 | 0.7083 | 0.4965 | 0.7877 | 0.3625 | 0.3800 | 0.4524 | 0.3276 | 0.3125 | 0.6944 | 0.2016 | 0.6545 | 0.6667 | 0.6429 | 0.0 | 0.0 | 0.0 | 0.5143 | 1.0 | 0.3462 | 0.0 | 0.0 | 0.0 | 0.2338 | 0.5294 | 0.15 | 0.6429 | 0.6667 | 0.6207 | 0.5561 | 0.9811 | 0.3881 | 0.1481 | 0.6667 | 0.0833 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1172 | 0.7368 | 0.0636 | 0.4762 | 0.3846 | 0.625 | 0.4648 | 0.8049 | 0.3267 | 0.2299 | 0.8333 | 0.1333 | 0.4429 | 0.5 | 0.3974 | 0.3009 | 0.6711 | 0.1939 | 0.4444 | 0.4 | 0.5 | 0.0714 | 0.1538 | 0.0465 | 0.0 | 0.0 | 0.0 | 0.4118 | 0.5 | 0.35 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.7568 | 0.7778 | 0.7368 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.3077 | 0.2857 | 0.3333 | 0.3333 | 1.0 | 0.2 | 0.6667 | 0.75 | 0.6 | 0.2222 | 0.875 | 0.1273 | 0.5246 | 0.6667 | 0.4324 | 0.3145 | 0.7812 | 0.1969 | 0.1818 | 0.2105 | 0.16 | 0.1910 | 0.3016 | 0.1397 | 0.6341 | 0.8254 | 0.5149 | 0.2434 | 0.6491 | 0.1498 | 0.3925 | 0.8077 | 0.2593 | 0.7042 | 0.7353 | 0.6757 | 0.4265 | 0.8788 | 0.2816 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.3908 | 0.5625 | 0.3356 |
| 0.0024 | 5.0 | 15070 | 0.0106 | 0.5861 | 0.6371 | 0.9416 | 0.4815 | 0.9805 | 0.9839 | 0.9772 | 0.5909 | 0.8667 | 0.4483 | 0.8039 | 0.9470 | 0.6983 | 0.7907 | 0.7827 | 0.7988 | 0.6354 | 0.9126 | 0.4874 | 0.6076 | 0.9791 | 0.4405 | 0.5590 | 0.5769 | 0.5422 | 0.2528 | 0.5965 | 0.1604 | 0.7992 | 0.8508 | 0.7536 | 0.8109 | 0.9347 | 0.7161 | 0.4823 | 0.8485 | 0.3369 | 0.3381 | 0.996 | 0.2036 | 0.8518 | 0.9103 | 0.8003 | 0.2566 | 0.8106 | 0.1524 | 0.5482 | 0.5203 | 0.5793 | 0.6995 | 0.8767 | 0.5818 | 0.6629 | 0.8992 | 0.5249 | 0.2411 | 0.3355 | 0.1882 | 0.6858 | 0.7103 | 0.6630 | 0.5455 | 0.8125 | 0.4105 | 0.6468 | 0.7927 | 0.5462 | 0.3140 | 0.8498 | 0.1926 | 0.8722 | 0.8722 | 0.8722 | 0.6272 | 0.9431 | 0.4699 | 0.7363 | 0.9658 | 0.5949 | 0.8690 | 0.8902 | 0.8488 | 0.6904 | 0.9692 | 0.5362 | 0.1986 | 0.4590 | 0.1267 | 0.7049 | 0.7736 | 0.6474 | 0.3125 | 0.3571 | 0.2778 | 0.2397 | 0.4043 | 0.1704 | 0.6680 | 0.6953 | 0.6429 | 0.3102 | 0.7267 | 0.1972 | 0.8649 | 0.7619 | 1.0 | 0.0859 | 0.5 | 0.0470 | 0.2667 | 0.25 | 0.2857 | 0.3333 | 0.4375 | 0.2692 | 0.7819 | 0.7983 | 0.7661 | 0.2779 | 0.8361 | 0.1667 | 0.4099 | 0.5593 | 0.3235 | 0.0 | 0.0 | 0.0 | 0.7378 | 0.6975 | 0.7830 | 0.4953 | 0.7318 | 0.3743 | 0.3191 | 0.3571 | 0.2885 | 0.3052 | 0.5185 | 0.2162 | 0.6667 | 0.8519 | 0.5476 | 0.0 | 0.0 | 0.0 | 0.5625 | 1.0 | 0.3913 | 0.0 | 0.0 | 0.0 | 0.25 | 0.5765 | 0.1596 | 0.5926 | 0.5926 | 0.5926 | 0.56 | 0.9245 | 0.4016 | 0.1067 | 0.6667 | 0.0580 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1266 | 0.7895 | 0.0688 | 0.3478 | 0.3077 | 0.4 | 0.4828 | 0.6829 | 0.3733 | 0.2933 | 0.9167 | 0.1746 | 0.4885 | 0.5161 | 0.4638 | 0.3055 | 0.5526 | 0.2111 | 0.4615 | 0.6 | 0.375 | 0.1034 | 0.2308 | 0.0667 | 0.0 | 0.0 | 0.0 | 0.4091 | 0.6429 | 0.3 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.7647 | 0.7222 | 0.8125 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.4000 | 0.4286 | 0.375 | 0.3333 | 1.0 | 0.2 | 0.6341 | 0.65 | 0.6190 | 0.2034 | 0.75 | 0.1176 | 0.3404 | 0.6667 | 0.2286 | 0.3356 | 0.7812 | 0.2137 | 0.2041 | 0.2632 | 0.1667 | 0.3077 | 0.4444 | 0.2353 | 0.6483 | 0.7460 | 0.5732 | 0.2628 | 0.6316 | 0.1659 | 0.4368 | 0.7308 | 0.3115 | 0.7385 | 0.7059 | 0.7742 | 0.448 | 0.8485 | 0.3043 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.3916 | 0.5600 | 0.3347 |
Framework versions
- Transformers 4.35.0
- Pytorch 2.1.0+cu121
- Datasets 2.2.2
- Tokenizers 0.14.1
- Downloads last month
- 499
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for denis-gordeev/rured2-ner-microsoft-mdeberta-v3-base
Base model
microsoft/mdeberta-v3-base