"Unknown model size" when submitting model

#945
by alvations - opened

When trying to submit the google/mt5-large model for LLM leaderboard, it throws a "Unknown model size" error.

image.png

This error did not happen when submitting google/mt5-base or google/mt5-xl model.

Any way to resolve this?

It also happened for umt5-small

image.png

Open LLM Leaderboard org

Hi! We extract model size from the safetensors weights (or the name in last resort), so you would need to provide models stored as safetensors - it's also a safer format for model storing than pickles for example.

For these older models that are not in safetensors, can there be (i) exceptions made or (ii) some way to rewrite the models artefacts to add the safetensors to the model files?

Open LLM Leaderboard org

There are spaces which can port the model weights to safetensors iirc - the whole documentation is here if you want to do it for these models

Thank you @clefourrier for the pointers! I'll see how to work it out for these older models =)

Open LLM Leaderboard org

I'm closing this discussion, please feel free to reopen it in case of any questions or open a new one!

alozowski changed discussion status to closed

Sign up or log in to comment