HamdanXI's picture
Upload tokenizer
1eb869d verified
raw
history blame
113 Bytes
{
"0": 1,
"1": 2,
"4": 3,
"5": 4,
"6": 5,
"9": 6,
"[PAD]": 9,
"[UNK]": 8,
"|": 0,
"’": 7
}