Isaak Carter Augustus
Upload tokenizer
73a3132 verified
|
raw
history blame contribute delete
No virus
880 Bytes
metadata
library_name: transformers
base_model: google/gemma-2b-it

Model card of JOSIExgemma_2b_Chat

This is my Token customized google/gemma-2b-it model

[Origional Model][https://huggingface.co/google/gemma-2b-it]

This is based on google/gemma-2b-it model with added custom special Tokens.


New added Special Tokens

'<|functions|>',
'<|system|>',
'<|gökdeniz|>',
'<|user|>',
'<|josie|>',
'<|assistant|>',
'<|function_call|>',
'<|function_response|>',
'<|image|>',
'<|long_term_memory|>',
'<|short_term_memory|>',
'<|home_state|>',
'<|current_states|>',
'<|context|>'

New BOS and EOS Tokens

BOS = '<|startoftext|>'
EOS = '<|endoftext|>'

New added Normal Tokens

['Gökdeniz Gülmez', 'Gökdeniz', 'Gülmez', 'JOSIE', 'J.O.S.I.E.', 'Josie', 'josie', 'Just an Outstandingly Smart and Intelligent Entity']