Smaller UL2 models

#8
by leshanbog - opened

Hi!
Thanks for sharing this 20B model with the community! From your paper it seems that smaller model also benefit from this kind of pretraining. Do you have any plans to release a UL2-base or something of that scale?

Sign up or log in to comment