wrong padding token
#2
by
aliencaocao
- opened
per the author's training code, unk is also used as padding token. However, this repo uses a new , untrained one.
Thanks for reporting, will fix in https://github.com/huggingface/transformers/pull/29586
Not sure using <unk> is a good practice, but whatever the content if the model outputs 0 for the pad_token_idx does it make a difference? Appart from unkown tokens being able to pop randomly in a text prompt, which does not sound like padding no?