32k or 128k?
#3
by
ChuckMcSneed
- opened
config.json says "max_position_embeddings": 32768, and readme says 128k
+1
To run the model at 128k, should we extend the max_position_embeddings or is there some sort of RoPE Scaling configuration that we should apply?
+1
Currently only support 32k with vllm deployment test.
Dear developers. Please clarify. In your blog you write about 128k, but the model says “max_position_embeddings”: 32768.
What to believe?
Hi! Should be fixed soon: https://huggingface.co/mistralai/Mistral-Large-Instruct-2407/discussions/11
Its indeed 128k!! 🔥
ChuckMcSneed
changed discussion status to
closed