Create README.md

README.md

@@ -0,0 +1,61 @@
+---
+language:
+- 'no'
+- nb
+- nn
+inference: false
+tags:
+- T5
+- NorT5
+- Norwegian
+- encoder-decoder
+license: cc-by-4.0
+---
+
+# NorT5 x-small
+
+
+## Other sizes:
+- [NorBERT 3 xs (15M)](https://huggingface.co/ltg/nort5-xs)
+- [NorBERT 3 small (40M)](https://huggingface.co/ltg/nort5-small)
+- [NorBERT 3 base (123M)](https://huggingface.co/ltg/nort5-base)
+- [NorBERT 3 large (323M)](https://huggingface.co/ltg/nort5-large)
+
+
+## Example usage
+
+This model currently needs a custom wrapper from `modeling_nort5.py`. Then you can use it like this:
+
+```python
+import torch
+from transformers import AutoTokenizer
+from modeling_norbert import NorT5ForConditionalGeneration
+
+tokenizer = AutoTokenizer.from_pretrained("path/to/folder")
+t5 = NorT5ForConditionalGeneration.from_pretrained("path/to/folder")
+
+
+# MASKED LANGUAGE MODELING
+
+sentence = "Brukseksempel: Elektrisk oppvarming. Definisjonen på ordet oppvarming er[MASK_0]."
+encoding = tokenizer(sentence)
+
+input_tensor = torch.tensor([encoding.input_ids])
+output_tensor = model.generate(input_tensor, decoder_start_token_id=7, eos_token_id=8)
+tokenizer.decode(output_tensor.squeeze(), skip_special_tokens=True)
+
+# should output:  å varme opp
+
+
+# PREFIX LANGUAGE MODELING
+# you need to finetune this model or use `nort5-{size}-lm` model, which is finetuned on prefix language modeling
+
+sentence = "Brukseksempel: Elektrisk oppvarming. Definisjonen på ordet oppvarming er (Wikipedia) "
+encoding = tokenizer(sentence)
+
+input_tensor = torch.tensor([encoding.input_ids])
+output_tensor = model.generate(input_tensor, max_new_tokens=50, num_beams=4, do_sample=False)
+tokenizer.decode(output_tensor.squeeze())
+
+# should output: [BOS]ˈoppvarming, det vil si at det skjer en endring i temperaturen i et medium, f.eks. en ovn eller en radiator, slik at den blir varmere eller kaldere, eller at den blir varmere eller kaldere, eller at den blir
+```