patrickvonplaten
commited on
Commit
•
aab999b
1
Parent(s):
9c67023
Update README.md
Browse files
README.md
CHANGED
@@ -2,9 +2,6 @@
|
|
2 |
language: multilingual
|
3 |
datasets:
|
4 |
- mc4
|
5 |
-
tags:
|
6 |
-
- summarization
|
7 |
-
- translation
|
8 |
|
9 |
license: apache-2.0
|
10 |
---
|
@@ -40,10 +37,10 @@ loss = model(input_ids, labels=labels).loss # forward pass
|
|
40 |
For batched inference & training it is however recommended using a tokenizer class for padding:
|
41 |
|
42 |
```python
|
43 |
-
from transformers import T5ForConditionalGeneration,
|
44 |
|
45 |
model = T5ForConditionalGeneration.from_pretrained('google/byt5-large')
|
46 |
-
tokenizer =
|
47 |
|
48 |
model_inputs = tokenizer(["Life is like a box of chocolates.", "Today is Monday."], padding="longest", return_tensors="pt")
|
49 |
labels = tokenizer(["La vie est comme une boîte de chocolat.", "Aujourd'hui c'est lundi."], padding="longest", return_tensors="pt").input_ids
|
|
|
2 |
language: multilingual
|
3 |
datasets:
|
4 |
- mc4
|
|
|
|
|
|
|
5 |
|
6 |
license: apache-2.0
|
7 |
---
|
|
|
37 |
For batched inference & training it is however recommended using a tokenizer class for padding:
|
38 |
|
39 |
```python
|
40 |
+
from transformers import T5ForConditionalGeneration, AutoTokenizer
|
41 |
|
42 |
model = T5ForConditionalGeneration.from_pretrained('google/byt5-large')
|
43 |
+
tokenizer = AutoTokenizer.from_pretrained('google/byt5-large')
|
44 |
|
45 |
model_inputs = tokenizer(["Life is like a box of chocolates.", "Today is Monday."], padding="longest", return_tensors="pt")
|
46 |
labels = tokenizer(["La vie est comme une boîte de chocolat.", "Aujourd'hui c'est lundi."], padding="longest", return_tensors="pt").input_ids
|