update readme with more examples

README.md

@@ -124,6 +124,8 @@ The model is mostly meant to be fine-tuned on a supervised dataset. See the [mod
 
 ### How to use
 
+The following shows how one can extract the last hidden representation for the model.
+
 ```python
 from transformers import T5Tokenizer, LongT5Model
 
@@ -136,6 +138,70 @@ outputs = model(**inputs)
 last_hidden_states = outputs.last_hidden_state
 ```
 
+The following shows how one can predict masked passages using the different denoising strategies.
+
+### S-Denoising
+
+For *S-Denoising*, please make sure to prompt the text with the prefix `[S2S]` as shown below.
+
+```python
+from transformers import LongT5ForConditionalGeneration, T5Tokenizer
+import torch
+
+model = LongT5ForConditionalGeneration.from_pretrained("agemagician/mlong-t5-tglobal-xl", low_cpu_mem_usage=True, torch_dtype=torch.bfloat16).to("cuda")                                                                                                   
+tokenizer = T5Tokenizer.from_pretrained("agemagician/mlong-t5-tglobal-xl")
+
+input_string = "[S2S] Mr. Dursley was the director of a firm called Grunnings, which made drills. He was a big, solid man with a bald head. Mrs. Dursley was thin and blonde and more than the usual amount of neck, which came in very useful as she spent so much of her time craning over garden fences, spying on the neighbours. The Dursleys had a small son called Dudley and in their opinion there was no finer boy anywhere <extra_id_0>"                                               
+
+inputs = tokenizer(input_string, return_tensors="pt").input_ids.to("cuda")
+
+outputs = model.generate(inputs, max_length=200)
+
+print(tokenizer.decode(outputs[0]))
+```
+
+### R-Denoising
+
+For *R-Denoising*, please make sure to prompt the text with the prefix `[NLU]` as shown below.
+
+```python
+from transformers import LongT5ForConditionalGeneration, T5Tokenizer
+import torch
+
+model = LongT5ForConditionalGeneration.from_pretrained("agemagician/mlong-t5-tglobal-xl", low_cpu_mem_usage=True, torch_dtype=torch.bfloat16).to("cuda")                                                                                                   
+tokenizer = T5Tokenizer.from_pretrained("agemagician/mlong-t5-tglobal-xl")
+
+input_string = "[NLU] Mr. Dursley was the director of a firm called <extra_id_0>, which made <extra_id_1>. He was a big, solid man with a bald head. Mrs. Dursley was thin and <extra_id_2> of neck, which came in very useful as she spent so much of her time <extra_id_3>. The Dursleys had a small son called Dudley and <extra_id_4>"                                               
+
+inputs = tokenizer(input_string, return_tensors="pt", add_special_tokens=False).input_ids.to("cuda")
+
+outputs = model.generate(inputs, max_length=200)
+
+print(tokenizer.decode(outputs[0]))
+```
+
+### X-Denoising
+
+For *X-Denoising*, please make sure to prompt the text with the prefix `[NLG]` as shown below.
+
+```python
+from transformers import LongT5ForConditionalGeneration, T5Tokenizer
+import torch
+
+model = LongT5ForConditionalGeneration.from_pretrained("agemagician/mlong-t5-tglobal-xl", low_cpu_mem_usage=True, torch_dtype=torch.bfloat16).to("cuda")                                                                                                   
+tokenizer = T5Tokenizer.from_pretrained("agemagician/mlong-t5-tglobal-xl")
+
+input_string = "[NLG] Mr. Dursley was the director of a firm called Grunnings, which made drills. He was a big, solid man wiht a bald head. Mrs. Dursley was thin and blonde and more than the usual amount of neck, which came in very useful as she
+spent so much of her time craning over garden fences, spying on the neighbours. The Dursleys had a small son called Dudley and in their opinion there was no finer boy anywhere. <extra_id_0>"                                               
+
+model.cuda()
+inputs = tokenizer(input_string, return_tensors="pt", add_special_tokens=False).input_ids.to("cuda")
+
+outputs = model.generate(inputs, max_length=200)
+
+print(tokenizer.decode(outputs[0]))
+```
+
 ### BibTeX entry and citation info
 
 ```bibtex