What is the length of the prompt considered by BLOOM?
#236
by
akratz
- opened
See subject line. How long can a prompt be to be considered in its entirety? What happens if it exceeds some length, is the beginning cut off?
BLOOM was trained with sequences of length 2048, but uses ALiBi position embeddings (https://arxiv.org/abs/2108.12409) -- meaning it can be used with longer sequences.
So ALIBI allows input of what length to be considered? You write that it allows for “longer sequences” but this way I only know it is more than 2048…
Any update on this? @cakiki
I'd refer you to both the paper I linked to and empirical experimentation to answer that question. The practical limit will likely sooner be your hardware than any theoretical one.
christopher
changed discussion status to
closed