Tokenizer.model
#53
by
itcleaf
- opened
Hi everyone,
I'm currently facing an issue with loading the tokenizer for the LLaMA model. I have downloaded the model and followed the instructions, but I'm encountering a problem where the tokenizer is recognized as a 'bool' object instead of the expected class. This results in an error when trying to tokenize input text.
I've checked that I have the tokenizer.model file in the correct directory, but it appears to be a plain ASCII text file.
Could anyone provide guidance on how to properly set up the tokenizer for the LLaMA model or suggest any troubleshooting steps? Any help would be greatly appreciated!
Thank you!