Wrong generations on empty strings
#3
by
Alealejandrooo
- opened
The model will output:
["I'm sorry. I'm sorry. I'm sorry."]
when prompted on an empty string:
Code to reproduce the error:
'''from transformers import MarianTokenizer, AutoModelForSeq2SeqLM
name = "Helsinki-NLP/opus-mt-sv-en"
model = AutoModelForSeq2SeqLM.from_pretrained(name)
tokenizer = MarianTokenizer.from_pretrained(name)
inputs = tokenizer(" ", padding=True, truncation=True, return_tensors="pt")
translated_outputs = model.generate(**inputs)
translations = [tokenizer.decode(output, skip_special_tokens=True) for output in translated_outputs]'''
Anyone having the same issue and possibly a solution to fix it?
Thanks!