Isaak Carter Augustus
Upload tokenizer
73a3132 verified
|
raw
history blame
880 Bytes
---
library_name: transformers
base_model: google/gemma-2b-it
---
# Model card of JOSIExgemma_2b_Chat
## This is my Token customized google/gemma-2b-it model
[Origional Model][https://huggingface.co/google/gemma-2b-it]
This is based on google/gemma-2b-it model with added custom special Tokens.
<br>
### New added Special Tokens
```text
'<|functions|>',
'<|system|>',
'<|gökdeniz|>',
'<|user|>',
'<|josie|>',
'<|assistant|>',
'<|function_call|>',
'<|function_response|>',
'<|image|>',
'<|long_term_memory|>',
'<|short_term_memory|>',
'<|home_state|>',
'<|current_states|>',
'<|context|>'
```
<br>
### New BOS and EOS Tokens
```text
BOS = '<|startoftext|>'
EOS = '<|endoftext|>'
```
<br>
### New added Normal Tokens
```text
['Gökdeniz Gülmez', 'Gökdeniz', 'Gülmez', 'JOSIE', 'J.O.S.I.E.', 'Josie', 'josie', 'Just an Outstandingly Smart and Intelligent Entity']
```