Hugging Face's logo

zirui3
/
llm-multilingual-tokenizer

Model card Files Community
Edit model card
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

summary

multilingual tokenizer trained on multilingual data by using the SentencePiece library and the BPE algorithm.

  • vocab size: 100k
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model's library. Check the docs .