Tokenizer for GPT-J-6B fails when trying to fine-tune for GLUE tasks
#24
by
Jojimon
- opened
To profile and optimize performance, I am trying to fine-tune GLUE tasks. The tasks fails as tokenizer does not have pad_token. Tried adding pad_token, it still fails due to differences in dimension. Any help appreciated.
Pad token failure.
Dimension issue
Tried using GPT-2 tokenizer, that works with pad_token fix. It would be ideal to use GPT-J-6B tokenizer itself.
Jojimon
changed discussion title from
Tokenizer for GPT-J-6B fails with trying to fine-tune for GLUE tasks
to Tokenizer for GPT-J-6B fails when trying to fine-tune for GLUE tasks