KoAlpaca-RealQA-Solar-Ko-Recovery-11B (QLoRA with Unsloth)
Model Description
- Developed by: Lee Junbum (Beomi)
- Model type: Instruction Tuned, with beomi/KoAlpaca-RealQA dataset
- Language(s) (NLP): Korean Mainly, partially English
- License: Apache 2.0
- Finetuned from model: beomi/Solar-Ko-Recovery-11B
Model Sources
- Training Code (Google Colab, Pro+ A100 40G): https://colab.research.google.com/drive/11Ni8rOBmV1Qh15i7gMWncKjYBEdrJLBt
- Inference Code (Google Colab): https://colab.research.google.com/drive/1hEPSHI4aGOn29Y21c6SWJc-y2ECVx3Bz?usp=sharing
Direct Use with Unsloth
# pip install -U hf_transfer unsloth
import os
os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1" # download speed upto 1000MB/s
import torch
from unsloth import FastLanguageModel
from transformers import TextStreamer
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "beomi/KoAlpaca-RealQA-Solar-Ko-Recovery-11B", # YOUR MODEL YOU USED FOR TRAINING
max_seq_length = 2048,
dtype = torch.bfloat16,
load_in_4bit = True,
)
FastLanguageModel.for_inference(model) # Enable native 2x faster inference
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{}
### Response:
{}"""
def gen(x):
inputs = tokenizer(
[
alpaca_prompt.format(
x.strip(), # instruction
"", # output - leave this blank for generation!
)
], return_tensors = "pt").to("cuda")
text_streamer = TextStreamer(tokenizer)
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 512)
Generation Example
Sample 01
gen("μλ
νμΈμ")
<s> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
μλ
νμΈμ
### Response:
μλ
νμΈμ! μ΄λ»κ² λμλ릴κΉμ?</s>
Sample 02
gen("""μλ κΈμ νκ΅μ΄λ‘ λ²μν΄μ€.
Dataset Summary
The KoAlpaca-RealQA dataset is a unique Korean instruction dataset designed to closely reflect real user interactions in the Korean language. Unlike conventional Korean instruction datasets that rely heavily on translated prompts, this dataset is composed of authentic Korean instructions derived from real-world use cases. Specifically, the dataset has been curated from user interactions with the ChatKoAlpaca service, which is based on the KoAlpaca model trained between 2023 and 2024.
This dataset provides a more accurate portrayal of typical Korean user behaviors, questions, and language structures, making it highly relevant for developing language models aimed at understanding and responding to Korean speakers. By leveraging GPT4o to generate high-quality answers, KoAlpaca-RealQA aims to offer a robust resource for training models that need to engage with Korean users in a natural and meaningful way.
""")
<s> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
μλ κΈμ νκ΅μ΄λ‘ λ²μν΄μ€.
Dataset Summary
The KoAlpaca-RealQA dataset is a unique Korean instruction dataset designed to closely reflect real user interactions in the Korean language. Unlike conventional Korean instruction datasets that rely heavily on translated prompts, this dataset is composed of authentic Korean instructions derived from real-world use cases. Specifically, the dataset has been curated from user interactions with the ChatKoAlpaca service, which is based on the KoAlpaca model trained between 2023 and 2024.
This dataset provides a more accurate portrayal of typical Korean user behaviors, questions, and language structures, making it highly relevant for developing language models aimed at understanding and responding to Korean speakers. By leveraging GPT4o to generate high-quality answers, KoAlpaca-RealQA aims to offer a robust resource for training models that need to engage with Korean users in a natural and meaningful way.
### Response:
KoAlpaca-RealQA λ°μ΄ν°μ
μ νκ΅μ΄ μ¬μ©μλ€μ μ€μ μνΈμμ©μ λ§€μ° μ λ°μνλλ‘ μ€κ³λ λ
νΉν νκ΅μ΄ μ§μ λ°μ΄ν°μ
μ
λλ€. λ²μλ ν둬ννΈμ ν¬κ² μμ‘΄νλ κΈ°μ‘΄μ νκ΅μ΄ μ§μ λ°μ΄ν°μ
κ³Ό λ¬λ¦¬, μ΄ λ°μ΄ν°μ
μ μ€μ μ¬μ© μ¬λ‘μμ μ λλ μ§μ ν νκ΅μ΄ μ§μλ‘ κ΅¬μ±λμ΄ μμ΅λλ€. νΉν, μ΄ λ°μ΄ν°μ
μ 2023λ
κ³Ό 2024λ
μ¬μ΄μ νλ ¨λ KoAlpaca λͺ¨λΈμ κΈ°λ°μΌλ‘ ν ChatKoAlpaca μλΉμ€μμ μ¬μ©μ μνΈμμ©μμ μμ§λμμ΅λλ€.
μ΄ λ°μ΄ν°μ
μ μΌλ°μ μΈ νκ΅μ΄ μ¬μ©μμ νλ, μ§λ¬Έ λ° μΈμ΄ ꡬ쑰λ₯Ό λ μ ννκ² λ¬μ¬νμ¬, νκ΅μ΄ μ¬μ©μλ₯Ό μ΄ν΄νκ³ μλ΅νλ μΈμ΄ λͺ¨λΈμ κ°λ°νλ λ° λ§€μ° μ μ©ν©λλ€. GPT4oλ₯Ό νμ©νμ¬ κ³ νμ§μ λ΅λ³μ μμ±ν¨μΌλ‘μ¨, KoAlpaca-RealQAλ μμ°μ€λ½κ³ μλ―Έ μλ λ°©μμΌλ‘ νκ΅μ΄ μ¬μ©μμ μνΈμμ©ν΄μΌ νλ λͺ¨λΈμ νλ ¨μν€κΈ° μν κ°λ ₯ν μμμ λͺ©νλ‘ νκ³ μμ΅λλ€.</s>
Sample 03
gen("""KoAlpacaμ λν΄ μ€λͺ
ν΄μ€.""")
<s> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
KoAlpacaμ λν΄ μ€λͺ
ν΄μ€.
### Response:
KoAlpacaλ νκ΅μ΄ μμ°μ΄ μ²λ¦¬(NLP) λͺ¨λΈλ‘, OpenAIμ GPT-3μ κ°μ λν μΈμ΄ λͺ¨λΈ(LLM)μ κΈ°λ°μΌλ‘ ν©λλ€. KoAlpacaλ νκ΅μ΄ λ°μ΄ν°λ₯Ό μ£Όλ‘ μ¬μ©νμ¬ νμ΅λμκΈ° λλ¬Έμ νκ΅μ΄ ν
μ€νΈλ₯Ό μ΄ν΄νκ³ μμ±νλ λ° νΉνλμ΄ μμ΅λλ€. μ΄ λͺ¨λΈμ λ€μν νκ΅μ΄ μμ© νλ‘κ·Έλ¨μμ νμ©λ μ μμΌλ©°, μλ₯Ό λ€μ΄ λνν AI, λ²μ, μμ½, μ§λ¬Έ λ΅λ³ λ± μ¬λ¬ λΆμΌμμ μ¬μ©λ μ μμ΅λλ€.
KoAlpacaλ νκ΅μ΄ μ¬μ©μμκ² λ³΄λ€ μμ°μ€λ½κ³ μ μ°½ν μΈμ΄ μνΈμμ©μ μ 곡νλ©°, νκ΅μ΄ λ¬Έλ§₯μ μ μ΄ν΄νκ³ μ²λ¦¬ν μ μλλ‘ μ€κ³λμμ΅λλ€. μ΄λ¬ν λͺ¨λΈμ νκ΅μ΄ NLP μ°κ΅¬μ μ°μ
μμ μ€μν λκ΅¬λ‘ μ¬μ©λ μ μμ΅λλ€.</s>
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.