title: WSB-GPT-7B Quantized in GGUF
tags:
- GGUF
language: en
Tsunemoto GGUF's of WSB-GPT-7B
This is a GGUF quantization of WSB-GPT-7B.
Original Repo Link:
Original Model Card:
Model Card for WSB-GPT-7B
This is a Llama 2 7B Chat model fine-tuned with QLoRA on 2017-2018ish /r/wallstreetbets subreddit comments and responses, with the hopes of learning more about QLoRA and creating models with a little more character.
Model Description
Developed by: Sentdex
Shared by: Sentdex
GPU Compute provided by: Lambda Labs
Model type: Instruct/Chat
Language(s) (NLP): Multilingual from Llama 2, but not sure what the fine-tune did to it, or if the fine-tuned behavior translates well to other languages. Let me know!
License: Apache 2.0
Finetuned from Llama 2 7B Chat
Demo [optional]: [More Information Needed]
Uses
This model's primary purpose is to be a fun chatbot and to learn more about QLoRA. It is not intended to be used for any other purpose and some people may find it abrasive/offensive.
Bias, Risks, and Limitations
This model is prone to using at least 3 words that were popularly used in the WSB subreddit in that era that are much more frowned-upon. As time goes on, I may wind up pruning or find-replacing these words in the training data, or leaving it.
Just be advised this model can be offensive and is not intended for all audiences!
How to Get Started with the Model
Prompt Format:
### Comment:
[parent comment text]
### REPLY:
[bot's reply]
### END.
Use the code below to get started with the model.
from transformers import pipeline
# Initialize the pipeline for text generation using the Sentdex/WSB-GPT-7B model
pipe = pipeline("text-generation", model="Sentdex/WSB-GPT-7B")
# Define your prompt
prompt = """### Comment:
How does the stock market actually work?
### REPLY:
"""
# Generate text based on the prompt
generated_text = pipe(prompt, max_length=128, num_return_sequences=1)
# Extract and print the generated text
print(generated_text[0]['generated_text'].split("### END.")[0])
Example continued generation from above:
### Comment:
How does the stock market actually work?
### REPLY:
You sell when you are up and buy when you are down.
Despite </s> being the typical Llama stop token, I was never able to get this token to be generated in training/testing so the model would just never stop generating. I wound up testing with ### END. and that worked, but obviously isn't ideal. Will fix this in the future maybe(tm).
Hardware
This QLoRA was trained on a Lambda Labs 1x H100 80GB GPU instance.
Citation
- Llama 2 (Meta AI) for the base model.
- Farouk E / Far El: https://twitter.com/far__el for helping with all my silly questions about QLoRA
- Lambda Labs for the compute. The model itself only took a few hours to train, but it took me days to learn how to tie everything together.
- Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer for QLoRA + implementation on github: https://github.com/artidoro/qlora/
- @eugene-yh and @jinyongyoo on Github + @ChrisHayduk for the QLoRA merge: https://gist.github.com/ChrisHayduk/1a53463331f52dca205e55982baf9930