Addy1008 commited on
Commit
2afca05
1 Parent(s): 5c13219

Upload 2 files

Browse files
Files changed (2) hide show
  1. README.md +109 -0
  2. gitattributes +16 -0
README.md ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - multilingual
4
+ - ar
5
+ - cs
6
+ - de
7
+ - en
8
+ - es
9
+ - et
10
+ - fi
11
+ - fr
12
+ - gu
13
+ - hi
14
+ - it
15
+ - ja
16
+ - kk
17
+ - ko
18
+ - lt
19
+ - lv
20
+ - my
21
+ - ne
22
+ - nl
23
+ - ro
24
+ - ru
25
+ - si
26
+ - tr
27
+ - vi
28
+ - zh
29
+ - af
30
+ - az
31
+ - bn
32
+ - fa
33
+ - he
34
+ - hr
35
+ - id
36
+ - ka
37
+ - km
38
+ - mk
39
+ - ml
40
+ - mn
41
+ - mr
42
+ - pl
43
+ - ps
44
+ - pt
45
+ - sv
46
+ - sw
47
+ - ta
48
+ - te
49
+ - th
50
+ - tl
51
+ - uk
52
+ - ur
53
+ - xh
54
+ - gl
55
+ - sl
56
+ tags:
57
+ - mbart-50
58
+ ---
59
+
60
+ # mBART-50 many to one multilingual machine translation
61
+
62
+
63
+ This model is a fine-tuned checkpoint of [mBART-large-50](https://huggingface.co/facebook/mbart-large-50). `mbart-large-50-many-to-many-mmt` is fine-tuned for multilingual machine translation. It was introduced in [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://arxiv.org/abs/2008.00401) paper.
64
+ The model can translate directly between any pair of 50 languages.
65
+
66
+
67
+ ```python
68
+ from transformers import MBartForConditionalGeneration, MBart50TokenizerFast
69
+
70
+ article_hi = "संयुक्त राष्ट्र के प्रमुख का कहना है कि सीरिया में कोई सैन्य समाधान नहीं है"
71
+ article_ar = "الأمين العام للأمم المتحدة يقول إنه لا يوجد حل عسكري في سوريا."
72
+
73
+ model = MBartForConditionalGeneration.from_pretrained("facebook/mbart-large-50-many-to-one-mmt")
74
+ tokenizer = MBart50TokenizerFast.from_pretrained("facebook/mbart-large-50-many-to-one-mmt")
75
+
76
+ # translate Hindi to English
77
+ tokenizer.src_lang = "hi_IN"
78
+ encoded_hi = tokenizer(article_hi, return_tensors="pt")
79
+ generated_tokens = model.generate(**encoded_hi)
80
+ tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
81
+ # => "The head of the UN says there is no military solution in Syria."
82
+
83
+ # translate Arabic to English
84
+ tokenizer.src_lang = "ar_AR"
85
+ encoded_ar = tokenizer(article_ar, return_tensors="pt")
86
+ generated_tokens = model.generate(**encoded_ar)
87
+ tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
88
+ # => "The Secretary-General of the United Nations says there is no military solution in Syria."
89
+ ```
90
+
91
+
92
+ See the [model hub](https://huggingface.co/models?filter=mbart-50) to look for more fine-tuned versions.
93
+
94
+
95
+ ## Languages covered
96
+ Arabic (ar_AR), Czech (cs_CZ), German (de_DE), English (en_XX), Spanish (es_XX), Estonian (et_EE), Finnish (fi_FI), French (fr_XX), Gujarati (gu_IN), Hindi (hi_IN), Italian (it_IT), Japanese (ja_XX), Kazakh (kk_KZ), Korean (ko_KR), Lithuanian (lt_LT), Latvian (lv_LV), Burmese (my_MM), Nepali (ne_NP), Dutch (nl_XX), Romanian (ro_RO), Russian (ru_RU), Sinhala (si_LK), Turkish (tr_TR), Vietnamese (vi_VN), Chinese (zh_CN), Afrikaans (af_ZA), Azerbaijani (az_AZ), Bengali (bn_IN), Persian (fa_IR), Hebrew (he_IL), Croatian (hr_HR), Indonesian (id_ID), Georgian (ka_GE), Khmer (km_KH), Macedonian (mk_MK), Malayalam (ml_IN), Mongolian (mn_MN), Marathi (mr_IN), Polish (pl_PL), Pashto (ps_AF), Portuguese (pt_XX), Swedish (sv_SE), Swahili (sw_KE), Tamil (ta_IN), Telugu (te_IN), Thai (th_TH), Tagalog (tl_XX), Ukrainian (uk_UA), Urdu (ur_PK), Xhosa (xh_ZA), Galician (gl_ES), Slovene (sl_SI)
97
+
98
+
99
+ ## BibTeX entry and citation info
100
+ ```
101
+ @article{tang2020multilingual,
102
+ title={Multilingual Translation with Extensible Multilingual Pretraining and Finetuning},
103
+ author={Yuqing Tang and Chau Tran and Xian Li and Peng-Jen Chen and Naman Goyal and Vishrav Chaudhary and Jiatao Gu and Angela Fan},
104
+ year={2020},
105
+ eprint={2008.00401},
106
+ archivePrefix={arXiv},
107
+ primaryClass={cs.CL}
108
+ }
109
+ ```
gitattributes ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.bin.* filter=lfs diff=lfs merge=lfs -text
2
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.h5 filter=lfs diff=lfs merge=lfs -text
5
+ *.tflite filter=lfs diff=lfs merge=lfs -text
6
+ *.tar.gz filter=lfs diff=lfs merge=lfs -text
7
+ *.ot filter=lfs diff=lfs merge=lfs -text
8
+ *.onnx filter=lfs diff=lfs merge=lfs -text
9
+ *.arrow filter=lfs diff=lfs merge=lfs -text
10
+ *.ftz filter=lfs diff=lfs merge=lfs -text
11
+ *.joblib filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.pb filter=lfs diff=lfs merge=lfs -text
15
+ *.pt filter=lfs diff=lfs merge=lfs -text
16
+ *.pth filter=lfs diff=lfs merge=lfs -text