Update README.md
Browse files
README.md
CHANGED
@@ -1 +1,23 @@
|
|
1 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Multi-lingual Question Generating Model (mt5-small)
|
2 |
+
Give the model a passage and it will generate a question about the passage.
|
3 |
+
|
4 |
+
## Trained on the following datasets:
|
5 |
+
|
6 |
+
- [SQuAD (English)](https://rajpurkar.github.io/SQuAD-explorer/)
|
7 |
+
- [TyDiQA-GoldP (Arabic, Bengali, Finnish, Japanese, Indonesian, Kiswahili, Korean, Russian, Telugu, Thai)](https://github.com/google-research-datasets/tydiqa)
|
8 |
+
- [MLQA (Arabic, Chinese, English, German, Hindi, Spanish, Vietnames)](https://github.com/facebookresearch/MLQA)
|
9 |
+
- [XQuAD (Arabic, Chinese, German, Greek, Hindi, Russian, Spanish, Thai, Turkish Vietnamese)](https://github.com/deepmind/xquad)
|
10 |
+
- [GermanQuAD (German)](https://huggingface.co/datasets/deepset/germanquad)
|
11 |
+
- [Persian QA (Persian)](https://www.kaggle.com/sajjadayobi360/persianqa)
|
12 |
+
- [Bengali QA (Bengali)](https://www.kaggle.com/mayeesha/bengali-question-answering-dataset)
|
13 |
+
- [chaii (Hindi, Tamil)](https://www.kaggle.com/c/chaii-hindi-and-tamil-question-answering/data)
|
14 |
+
|
15 |
+
|
16 |
+
## Training details
|
17 |
+
I used [flax summarization script](https://github.com/huggingface/transformers/tree/master/examples/flax/summarization) and a TPU v3-8. Summarization expects a text column and a summary column. For question generation training, use the context column instead of text column and question instead of summary column.
|
18 |
+
|
19 |
+
|
20 |
+
There is no guarantee that it will produce a question in the language of the passage, but it usually does.
|
21 |
+
|
22 |
+
|
23 |
+
Model trained on Cloud TPUs from Google's TPU Research Cloud (TRC)
|