Edit Models filters

Inference status

Misc

Inference Endpoints

AutoTrain Compatible

Misc with no match

text-generation-inference

4-bit precision

text-embeddings-inference

8-bit precision

Carbon Emissions

Mixture of Experts

Models

28

Full-text search

Active filters: tokenizer

novateur/WavTokenizer

Text-to-Speech • Updated Sep 27 • 43

ybelkada/japanese-dummy-tokenizer

Updated Jul 11, 2022 • 6 • 1

Dimitre/bert_en_cased_preprocess

Updated Aug 18, 2022 • 1

dn-gh/gpt2_pubmed_tokenizer

Updated Nov 11, 2022 • 1

jeromeku/llama-7b-converted-tokenizer

Updated Apr 27, 2023

balochiml/balochi-tokenizer

Updated Jun 3, 2023 • 1

allenai/eleuther-ai-gpt-neox-20b-pii-special

Updated Jul 13, 2023

mkshing/novelai-tokenizer-v1

Updated Aug 14, 2023

NovelAI/nerdstash-tokenizer-v1

Updated Aug 2, 2023 • 6

NovelAI/nerdstash-tokenizer-v2

Updated Aug 2, 2023 • 7

radlab/polish-fast-tokenizer

Updated Dec 31, 2023

if001/sentencepiece_ja

Updated Jan 23 • 1 • 3

allenai/meta-llama-pii-special

Updated Oct 29, 2023 • 1

ShirinYamani/NLTK-tokenizer

Updated Jan 3 • 1

dustalov/wikitext-wordlevel

Updated Jan 22 • 1

nilq/baby-tokenizer

Updated Feb 22 • 1

ldilov/llama2-bg-tokenizer

Updated Feb 15 • 1

nilq/baby-tokenizer-uncased

pszemraj/claude-tokenizer-mlm

DOSaAI/VALa1Tokenizer

Saxo/llama2_Linkbricks_korean_tokenzier_stem1

samim2024/text-to-image

Text-to-Image • Updated Jun 2 • 26

Saxo/llama3_Linkbricks_korean_tokenzier_stem1

BEE-spoke-data/claude-tokenizer-forT5

riotu-lab/Aranizer-PBE-32k

Updated Aug 25 • 1

ibm/DAC.speech.v1.0

Updated Sep 19 • 9 • 1

pszemraj/sail-slimpajama-6B-32768-BPE-tokenizer

gpwr/bark-it-tokenizer

Updated about 7 hours ago