model update
Browse files
README.md
CHANGED
@@ -14,7 +14,7 @@ model-index:
|
|
14 |
metrics:
|
15 |
- name: Accuracy
|
16 |
type: accuracy
|
17 |
-
value: 0.
|
18 |
- task:
|
19 |
name: Analogy Questions (SAT full)
|
20 |
type: multiple-choice-qa
|
@@ -91,10 +91,10 @@ model-index:
|
|
91 |
metrics:
|
92 |
- name: F1
|
93 |
type: f1
|
94 |
-
value:
|
95 |
- name: F1 (macro)
|
96 |
type: f1_macro
|
97 |
-
value:
|
98 |
- task:
|
99 |
name: Lexical Relation Classification (CogALexV)
|
100 |
type: classification
|
@@ -105,10 +105,10 @@ model-index:
|
|
105 |
metrics:
|
106 |
- name: F1
|
107 |
type: f1
|
108 |
-
value:
|
109 |
- name: F1 (macro)
|
110 |
type: f1_macro
|
111 |
-
value:
|
112 |
- task:
|
113 |
name: Lexical Relation Classification (EVALution)
|
114 |
type: classification
|
@@ -119,10 +119,10 @@ model-index:
|
|
119 |
metrics:
|
120 |
- name: F1
|
121 |
type: f1
|
122 |
-
value:
|
123 |
- name: F1 (macro)
|
124 |
type: f1_macro
|
125 |
-
value:
|
126 |
- task:
|
127 |
name: Lexical Relation Classification (K&H+N)
|
128 |
type: classification
|
@@ -133,10 +133,10 @@ model-index:
|
|
133 |
metrics:
|
134 |
- name: F1
|
135 |
type: f1
|
136 |
-
value:
|
137 |
- name: F1 (macro)
|
138 |
type: f1_macro
|
139 |
-
value:
|
140 |
- task:
|
141 |
name: Lexical Relation Classification (ROOT09)
|
142 |
type: classification
|
@@ -147,10 +147,10 @@ model-index:
|
|
147 |
metrics:
|
148 |
- name: F1
|
149 |
type: f1
|
150 |
-
value:
|
151 |
- name: F1 (macro)
|
152 |
type: f1_macro
|
153 |
-
value:
|
154 |
|
155 |
---
|
156 |
# relbert/roberta-large-semeval2012-mask-prompt-b-nce
|
@@ -167,13 +167,13 @@ It achieves the following results on the relation understanding tasks:
|
|
167 |
- Accuracy on U4: 0.5509259259259259
|
168 |
- Accuracy on Google: 0.898
|
169 |
- Lexical Relation Classification ([dataset](https://huggingface.co/datasets/relbert/lexical_relation_classification), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-b-nce/raw/main/classification.json)):
|
170 |
-
- Micro F1 score on BLESS:
|
171 |
-
- Micro F1 score on CogALexV:
|
172 |
-
- Micro F1 score on EVALution:
|
173 |
-
- Micro F1 score on K&H+N:
|
174 |
-
- Micro F1 score on ROOT09:
|
175 |
- Relation Mapping ([dataset](https://huggingface.co/datasets/relbert/relation_mapping), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-b-nce/raw/main/relation_mapping.json)):
|
176 |
-
- Accuracy on Relation Mapping: 0.
|
177 |
|
178 |
|
179 |
### Usage
|
|
|
14 |
metrics:
|
15 |
- name: Accuracy
|
16 |
type: accuracy
|
17 |
+
value: 0.8801984126984127
|
18 |
- task:
|
19 |
name: Analogy Questions (SAT full)
|
20 |
type: multiple-choice-qa
|
|
|
91 |
metrics:
|
92 |
- name: F1
|
93 |
type: f1
|
94 |
+
value: 0.9285821907488323
|
95 |
- name: F1 (macro)
|
96 |
type: f1_macro
|
97 |
+
value: 0.9242931275672687
|
98 |
- task:
|
99 |
name: Lexical Relation Classification (CogALexV)
|
100 |
type: classification
|
|
|
105 |
metrics:
|
106 |
- name: F1
|
107 |
type: f1
|
108 |
+
value: 0.8713615023474178
|
109 |
- name: F1 (macro)
|
110 |
type: f1_macro
|
111 |
+
value: 0.7158151145502465
|
112 |
- task:
|
113 |
name: Lexical Relation Classification (EVALution)
|
114 |
type: classification
|
|
|
119 |
metrics:
|
120 |
- name: F1
|
121 |
type: f1
|
122 |
+
value: 0.7004333694474539
|
123 |
- name: F1 (macro)
|
124 |
type: f1_macro
|
125 |
+
value: 0.6995319467580109
|
126 |
- task:
|
127 |
name: Lexical Relation Classification (K&H+N)
|
128 |
type: classification
|
|
|
133 |
metrics:
|
134 |
- name: F1
|
135 |
type: f1
|
136 |
+
value: 0.9651526744105168
|
137 |
- name: F1 (macro)
|
138 |
type: f1_macro
|
139 |
+
value: 0.8948710758703222
|
140 |
- task:
|
141 |
name: Lexical Relation Classification (ROOT09)
|
142 |
type: classification
|
|
|
147 |
metrics:
|
148 |
- name: F1
|
149 |
type: f1
|
150 |
+
value: 0.9025383892196804
|
151 |
- name: F1 (macro)
|
152 |
type: f1_macro
|
153 |
+
value: 0.901680693253323
|
154 |
|
155 |
---
|
156 |
# relbert/roberta-large-semeval2012-mask-prompt-b-nce
|
|
|
167 |
- Accuracy on U4: 0.5509259259259259
|
168 |
- Accuracy on Google: 0.898
|
169 |
- Lexical Relation Classification ([dataset](https://huggingface.co/datasets/relbert/lexical_relation_classification), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-b-nce/raw/main/classification.json)):
|
170 |
+
- Micro F1 score on BLESS: 0.9285821907488323
|
171 |
+
- Micro F1 score on CogALexV: 0.8713615023474178
|
172 |
+
- Micro F1 score on EVALution: 0.7004333694474539
|
173 |
+
- Micro F1 score on K&H+N: 0.9651526744105168
|
174 |
+
- Micro F1 score on ROOT09: 0.9025383892196804
|
175 |
- Relation Mapping ([dataset](https://huggingface.co/datasets/relbert/relation_mapping), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-b-nce/raw/main/relation_mapping.json)):
|
176 |
+
- Accuracy on Relation Mapping: 0.8801984126984127
|
177 |
|
178 |
|
179 |
### Usage
|