open-llama-2-ko based model with modified DPO dataset
This is an Korean Model based on
- [beomi/open-llama-2-ko-7b]
Dataset is modified from
- [SJ-Donald/orca-dpo-pairs-ko]
Parameters
learning_rate: float = 3e-4
lr_scheduler: str = "cosine"
warmup_ratio: float = 0.1
lora_r: int = 16
lora_alpha: int = 16
lora_dropout: float = 0.05
optim='paged_adamw_32bit'
bf16=True
- Downloads last month
- 1,856
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.