Update to tokenizers > 0.13
#4
by
gregoriopellegrino
- opened
Hi,
thanks for this great work. Unfortunately tokenizers 0.12.1 cannot be installed on mac with M1 or M2 chips. Would it be possible to update tokenizers dependencies to at least version 0.13?
Thank you,
Gregory