Use with Private-GPT
Hello there. I would like to use this LLM with private-GPT. Would you able to provide me with soem guidance? I'm able to download the model and start it on my local Ubuntu server. However, when I try to chat with it, I'm noticing unexpected behaviour such as the LLM responding to itself and so on. Any help would be greatly appreciated.
Hello there. I would like to use this LLM with private-GPT. Would you able to provide me with soem guidance? I'm able to download the model and start it on my local Ubuntu server. However, when I try to chat with it, I'm noticing unexpected behaviour such as the LLM responding to itself and so on. Any help would be greatly appreciated.
Hey - usually when this happens, it is because you need the right stop sequence (use Llama 3 as reference).
The stop sequences I am using:
<|eot_id|><|start_header_id|>, <|end_of_text|>, USER: