salti commited on
Commit
11d443d
1 Parent(s): 3cfeb44

add zero-shot results

Browse files
Files changed (1) hide show
  1. README.md +30 -3
README.md CHANGED
@@ -4,11 +4,14 @@ language:
4
 
5
  datasets:
6
  - squad
7
-
8
  ---
9
 
10
  # Multilingual BERT fine-tuned on SQuADv1.1
11
 
 
 
 
 
12
  ## Training Arguments
13
 
14
  ```python
@@ -28,6 +31,30 @@ seed = 0
28
 
29
  ## Results
30
 
31
- | EM | F1 |
32
- | :---: | :---: |
33
  | 81.731 | 89.009 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
 
5
  datasets:
6
  - squad
 
7
  ---
8
 
9
  # Multilingual BERT fine-tuned on SQuADv1.1
10
 
11
+ [**WandB run link**](https://wandb.ai/salti/mBERT_QA/runs/wkqzhrp2)
12
+
13
+ **GPU**: Tesla P100-PCIE-16GB
14
+
15
  ## Training Arguments
16
 
17
  ```python
 
31
 
32
  ## Results
33
 
34
+ | EM | F1 |
35
+ | :----: | :----: |
36
  | 81.731 | 89.009 |
37
+
38
+ ## Zero-shot performance
39
+
40
+ ### on ARCD
41
+
42
+ | EM | F1 |
43
+ | :----: | :----: |
44
+ | 20.655 | 48.051 |
45
+
46
+ ### on XQuAD
47
+
48
+ | Language | EM | F1 |
49
+ | :--------: | :----: | :----: |
50
+ | Arabic | 42.185 | 57.803 |
51
+ | English | 73.529 | 85.01 |
52
+ | German | 55.882 | 72.555 |
53
+ | Greek | 45.21 | 62.207 |
54
+ | Spanish | 58.067 | 76.406 |
55
+ | Hindi | 40.588 | 55.29 |
56
+ | Russian | 55.126 | 71.617 |
57
+ | Thai | 26.891 | 39.965 |
58
+ | Turkish | 34.874 | 51.138 |
59
+ | Vietnamese | 47.983 | 68.125 |
60
+ | Chinese | 47.395 | 58.928 |