Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Spaces:

xu-song
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / tokenizer

2 contributors

History: 7 commits

xu-song's picture

add compress rate

814ee6b 7 months ago

chinese_sptokenizer_patch.py

27 Bytes

add compress rate 7 months ago
sptokenizer_patch.py

2.82 kB

add compress rate 7 months ago
tiktoken_patch.py

2.97 kB

add compress rate 7 months ago
tokenizer_patcher.py

211 Bytes

add compress rate 7 months ago