Qwen
/

Qwen-VL

Text Generation

Model card Files Files and versions Community

Qwen-VL / tokenizer_config.json

simonJJJ's picture

add tokenizer config

0dfece6 about 1 year ago

173 Bytes

	{
	"model_max_length": 8192,
	"tokenizer_class": "QWenTokenizer",
	"auto_map": {
	"AutoTokenizer": [
	"tokenization_qwen.QWenTokenizer",
	null
	]
	}
	}