Hugging Face's logo
Back to Articles

Samantha and Mistral 7B: A Powerful and Versatile Language Model Duo

Community Article Published October 2, 2023
Upvote
1
Ankush Singal's avatar
Andyrasika Ankush Singal

image/jpeg

Mistral 7B is a breakthrough language model that is both efficient and powerful. It has fewer parameters than other large language models, such as Meta's Llama 2 13B, but it outperforms them on many tasks. Mistral 7B is also versatile, excelling at both English language tasks and coding tasks. This makes it a valuable tool for a wide range of enterprise applications.

In addition, Mistral 7B is open-source, which means that anyone can use it and modify it without restrictions. This makes it a great choice for companies and organizations that want to develop their own custom AI applications.

In this article we will cover the following:

  • Samantha LLM
  • Samantha LLM with Mistral 7B
  • Code Implementation

Samantha LLM

Samantha is a large language model (LLM) that is trained on a massive dataset of text and code. It can generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way. Samantha is still under development, but it has learned to perform many kinds of tasks, including:

Following your instructions and completing your requests thoughtfully. Using its knowledge to answer your questions in a comprehensive and informative way, even if they are open ended, challenging, or strange. Generating different creative text formats of text content, like poems, code, scripts, musical pieces, email, letters, etc.

image/jpeg

Samantha LLM with Mistral 7B

Mistral 7B is another LLM that is trained on a massive dataset of text and code. It is known for its efficiency and power, as it outperforms larger models like Meta’s Llama 2 13B despite having fewer parameters. Mistral 7B is also versatile, excelling in both English language tasks and coding tasks.

By combining Samantha and Mistral 7B, you can create a language model that is even more powerful and versatile. This could be useful for a wide range of tasks, such as:

  • Generating more creative and informative text content.
  • Translating languages more accurately.
  • Writing more complex code.
  • Answering your questions in a more comprehensive and informative way.

Code Implementation

In the code implementation section of the article, i will provide code examples for how to use Samantha and Mistral 7B together. This will make it easier for readers to get started using these powerful language models.

image/jpeg

Import libraries:

!pip install -q -U bitsandbytes
!pip install -q -U git+https://github.com/huggingface/transformers.git
!pip install -q -U git+https://github.com/huggingface/peft.git
!pip install -q -U git+https://github.com/huggingface/accelerate.git
!pip -q install sentencepiece Xformers einops
!pip -q install langchain

import torch
import transformers
from transformers import GenerationConfig, pipeline
from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers import BitsAndBytesConfig
import bitsandbytes as bnb

import torch
import transformers
from transformers import GenerationConfig, pipeline
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ehartford/samantha-mistral-7b")

model = AutoModelForCausalLM.from_pretrained("ehartford/samantha-mistral-7b",
                                              load_in_8bit=True,
                                              device_map='auto',
                                              torch_dtype=torch.float16,
                                              low_cpu_mem_usage=True,
                                              )

tokenizer.eos_token_id, tokenizer.pad_token_id
tokenizer.pad_token_id = 0
pipe = pipeline(
    "text-generation",
    model=model, 
    tokenizer=tokenizer, 
    max_length=1536,
    temperature=0.7,
    top_p=0.95,
    repetition_penalty=1.15
)

The prompt & response

import json
import textwrap

system_prompt = "A chat between a curious user and an artificial intelligence assistant. \nThe assistant gives helpful, detailed, and polite answers to the user's questions."

addon_prompt = "Your name is Samantha."
# USER: What is 4x8?
# ASSISTANT:
def get_prompt(human_prompt):
    # prompt_template=f"{human_prompt}"
    prompt_template = f"{addon_prompt}\n{system_prompt}\n\nUSER: {human_prompt} \nASSISTANT: "
    return prompt_template    

print(get_prompt('What is the meaning of life?'))

def remove_human_text(text):
    return text.split('USER:', 1)[0]

def parse_text_after_input(data, input_string):
    for item in data:
        text = item['generated_text']
        input_string_index = text.find(input_string)
        if input_string_index != -1:
            output_text = text[input_string_index+len(input_string):].strip()
            output_text = parse_text(output_text)
            wrapped_text = textwrap.fill(output_text, width=100)
            print(wrapped_text)

def parse_text(data):
    for item in data:
        text = item['generated_text']
        assistant_text_index = text.find('ASSISTANT:')
        if assistant_text_index != -1:
            assistant_text = text[assistant_text_index+len('ASSISTANT:'):].strip()
            assistant_text = remove_human_text(assistant_text)
            wrapped_text = textwrap.fill(assistant_text, width=100)
            print(wrapped_text)
            # return assistant_text

data = [{'generated_text': '### Human: What is the capital of England? \n### Response: The capital city of England is London.'}]
parse_text(data)

A chat between a curious user and an artificial intelligence assistant. 
The assistant gives helpful, detailed, and polite answers to the user's questions.
Your name is Samantha. 

USER: What is the meaning of life? 
ASSISTANT:

Lets try with some prompts

%%time 
prompt = 'What is your name?'
raw_output = pipe(get_prompt(prompt))
parse_text(raw_output)

My name is Samantha. I'm here to provide support, companionship, and engaging conversations for you.
CPU times: user 14.1 s, sys: 0 ns, total: 14.1 s
Wall time: 14.1 s

%%time 
prompt = 'What can you help me with?'
raw_output = pipe(get_prompt(prompt))
parse_text(raw_output)

I am here to provide companionship, emotional support, and information on various subjects. Whether
it's sharing interesting facts, discussing books or movies, or engaging in creative activities like
writing poetry, I'm always ready for a conversation. My main goal is to make your life more
enjoyable and interesting while offering a listening ear when needed.
CPU times: user 45.6 s, sys: 0 ns, total: 45.6 s
Wall time: 45.5 s

%%time 
prompt = 'What are the difference between Llamas, Alpacas and Vicunas?'
raw_output = pipe(get_prompt(prompt))
parse_text(raw_output)

Llamas (pronounced "yah-mas") and alpacas belong to the camelid family and are native to South
America. They have long necks, small ears, and soft, woolly coats that come in various colors like
black, white, brown, and fawn. Both llamas and alpacas can be domesticated and raised for their
fiber, which is used to make clothing and other textiles.  Vicuñas (pronounced "vee-koo-nahs"), on
the other hand, are smaller than both llamas and alpacas and live only in the Andes Mountains of
Peru, Bolivia, Chile, and Ecuador. Their fleece is finer and softer than either llama or alpaca fur,
making it highly prized for its luxurious quality.
CPU times: user 35.9 s, sys: 9.58 ms, total: 35.9 s
Wall time: 35.8 s

Conclusion

In this post, we have embarked on a fascinating journey of discovery, exploring the possibilities of the Samantha-Mistral language model. We have learned how to interact with Samantha in a natural and intuitive way, and how to harness her power to retrieve and process information.

This journey is just the beginning. As large language models continue to evolve, they will open up new and exciting possibilities for us. Samantha-Mistral is just one example of this potential. It is a powerful tool that can be used to solve complex problems, create innovative solutions, and explore new frontiers of knowledge.

I hope that this article has inspired you to learn more about Samantha-Mistral and other large language models. Together, we can use these tools to shape a better future for all.

LinkedIn: You can follow me on LinkedIn to keep up to date with my latest projects and posts. Here is the link to my profile: https://www.linkedin.com/in/ankushsingal/

GitHub: You can also support me on GitHub. There I upload all my Notebooks and other open source projects. Feel free to leave a star if you liked the content. Here is the link to my GitHub: https://github.com/andysingal?tab=repositories

Requests and questions: If you have a project in mind that you’d like me to work on or if you have any questions about the concepts I’ve explained, don’t hesitate to let me know. I’m always looking for new ideas for future Notebooks and I love helping to resolve any doubts you might have.

Remember, each “Like”, “Share”, and “Star” greatly contributes to my work and motivates me to continue producing more quality content. Thank you for your support!

Resources:

Upvote
1