mistralai

/

Mistral-7B-Instruct-v0.3

like 975

I am initiating my 0.3 Instruct connecting with langchain and notized that it uses by default a LlamaTokenizer. I would assume that the performance of MistralTokenizer is better compared to LlamaTokenizer, but I have no idea how to combine these two.

This is my basic statement:

llm = HuggingFacePipeline.from_model_id(
           model_id=checkpoint,
           task="text-generation",
           device=None,
           pipeline_kwargs={
               "max_new_tokens": 2048,  # changed from 2048
               "top_p": 0.15,  # changed from 0.15
               "do_sample": True,  # changed from true
               "torch_dtype": torch.float64,  # bfloat16
               "use_fast": True,
               },
           model_kwargs=model_kwargs
       )

When I add the tokenizer to the pipeline_kwarks ("tokenizer":tokenizer,)
I get "TypeError: transformers.pipelines.pipeline() got multiple values for keyword argument 'tokenizer'"

When I add it to the from_modelid() as tokenizer=tokenizer,
then the app crashes with pydantic.v1.error_wrappers.ValidationError: 1 validation error for HuggingFacePipeline tokenizer extra fields not permitted (type=value_error.extra)

Any ideas how to fix this?