model_max_length vs max_position_embeddings?
#2
by
arclight1981
- opened
Hey there!
So when I work with LLMs, I sometimes have a pretty long context I try to feed into my models. Now, what confuses me about this model is (and mind you, I am still learning all of this): the model_max_length is 512 tokens, while the max_position_embeddings are 2048 tokens.
Does that mean the model itself supports 2048 tokens but the tokenizer can only encode 512 tokens max? Would there be a way to switch out the tokenizer so that it can feed the full 2048 tokens the model supports? Or am I missing something?