Model Details

Model Description

This is a fine-tuned version of the Llama 3.1 8B model, adapted for generating expert sommelier responses in Russian. The model was trained on a custom Russian-language dataset consisting of user query - sommelier response pairs.

Developed by: [Your Name or Organization]
Model type: Language Model
Language(s) (NLP): Russian
License: [Specify the license, e.g., MIT, Apache 2.0, etc.]
Finetuned from model: Llama 3.1 8B

Model Sources [optional]

Repository: [Your repository link if available]

Uses

Direct Use

This model can be used to generate sommelier-like responses to user queries about wine in Russian. It's designed to provide expert-level information and recommendations related to wine.

Out-of-Scope Use

This model should not be used for generating responses outside the domain of wine expertise or in languages other than Russian. It is not designed for general-purpose language tasks.

Bias, Risks, and Limitations

The model's knowledge is limited to the training data provided, which focuses on wine expertise. It may not have up-to-date information on recent wine releases or changes in the wine industry. The model may also reflect biases present in the training data.

Recommendations

Users should verify important information provided by the model, especially for critical decisions related to wine selection or pairing. The model should be used as a supportive tool rather than a definitive source of wine expertise.

Training Details

Training Data

The model was fine-tuned on a custom Russian-language dataset consisting of user query - sommelier response pairs. The exact size and composition of the dataset are not specified.

Training Procedure

Training Hyperparameters

Training regime: Mixed precision (specific type not provided)
Number of epochs: 1
Batch size per device: 2
Gradient Accumulation steps: 4
Total batch size: 8
Total steps: 60
Number of trainable parameters: 83,886,080

Evaluation

Metrics

The primary metric used for evaluation was Training Loss. The loss decreased from an initial value of 2.6349 to 0.5164 over 60 training steps, indicating significant improvement in the model's performance.

Results

The training loss showed a consistent decrease over the 60 training steps, suggesting that the model successfully adapted to the task of generating sommelier-like responses.

Environmental Impact

Hardware Type: 1 GPU (specific type not provided)
Hours used: [Information needed]
Cloud Provider: [Information needed]
Compute Region: [Information needed]
Carbon Emitted: [Information needed]

Technical Specifications [optional]

Compute Infrastructure

Hardware

1 GPU (specific details not provided)

Software

Unsloth for faster finetuning
Transformers library

Model Card Contact

[https://t.me/grozaae]

Uploaded model

Developed by: mcgrozov
License: apache-2.0
Finetuned from model : unsloth/meta-llama-3.1-8b-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.