This is a quick experiment with only 150 orpo steps from a german dataset.

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Inference Examples

Inference API (serverless) is not available, repository is disabled.

Model tree for cstr/phi-3-orpo-v8_16

Developed by: cstr
License: apache-2.0
Finetuned from model: vonjack/Phi-3-mini-4k-instruct-LLaMAfied

Base model

Finetuned

this model

Finetunes

Quantizations