Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

XiaHan19
/

shami-token

Model card Files Files and versions Community

2 contributors

History: 6 commits

jstzwj

update tokenizer

6283007 over 1 year ago

.gitattributes

1.48 kB

initial commit over 1 year ago
README.md

88 Bytes

init repo over 1 year ago
merges.txt

1.27 MB

update tokenizer over 1 year ago
requirements.txt

55 Bytes

init repo over 1 year ago
special_tokens_map.json

99 Bytes

init repo over 1 year ago
tokenization_shami.py

10.3 kB

add code over 1 year ago
tokenization_shami_fast.py

3.5 kB

add code over 1 year ago
tokenizer.json

4.99 MB

update tokenizer over 1 year ago
tokenizer_config.json

234 Bytes

update tokenizer over 1 year ago
train_tokenizer.py

764 Bytes

update tokenizer over 1 year ago
vocab.json

1.98 MB

update tokenizer over 1 year ago