--- base_model: unsloth/meta-llama-3.1-8b-bnb-4bit language: - ru license: apache-2.0 tags: - text-generation-inference - transformers - unsloth - llama - trl --- # Model Details ### Model Description This is a fine-tuned version of the Llama 3.1 8B model, adapted for generating expert sommelier responses in Russian. The model was trained on a custom Russian-language dataset consisting of user query - sommelier response pairs. - **Developed by:** [Your Name or Organization] - **Model type:** Language Model - **Language(s) (NLP):** Russian - **License:** [Specify the license, e.g., MIT, Apache 2.0, etc.] - **Finetuned from model:** Llama 3.1 8B ### Model Sources [optional] - **Repository:** [Your repository link if available] ## Uses ### Direct Use This model can be used to generate sommelier-like responses to user queries about wine in Russian. It's designed to provide expert-level information and recommendations related to wine. ### Out-of-Scope Use This model should not be used for generating responses outside the domain of wine expertise or in languages other than Russian. It is not designed for general-purpose language tasks. ## Bias, Risks, and Limitations The model's knowledge is limited to the training data provided, which focuses on wine expertise. It may not have up-to-date information on recent wine releases or changes in the wine industry. The model may also reflect biases present in the training data. ### Recommendations Users should verify important information provided by the model, especially for critical decisions related to wine selection or pairing. The model should be used as a supportive tool rather than a definitive source of wine expertise. ## Training Details ### Training Data The model was fine-tuned on a custom Russian-language dataset consisting of user query - sommelier response pairs. The exact size and composition of the dataset are not specified. ### Training Procedure #### Training Hyperparameters - **Training regime:** Mixed precision (specific type not provided) - **Number of epochs:** 1 - **Batch size per device:** 2 - **Gradient Accumulation steps:** 4 - **Total batch size:** 8 - **Total steps:** 60 - **Number of trainable parameters:** 83,886,080 ### Evaluation #### Metrics The primary metric used for evaluation was Training Loss. The loss decreased from an initial value of 2.6349 to 0.5164 over 60 training steps, indicating significant improvement in the model's performance. ### Results The training loss showed a consistent decrease over the 60 training steps, suggesting that the model successfully adapted to the task of generating sommelier-like responses. ## Environmental Impact - **Hardware Type:** 1 GPU (specific type not provided) - **Hours used:** [Information needed] - **Cloud Provider:** [Information needed] - **Compute Region:** [Information needed] - **Carbon Emitted:** [Information needed] ## Technical Specifications [optional] ### Compute Infrastructure #### Hardware 1 GPU (specific details not provided) #### Software - Unsloth for faster finetuning - Transformers library ## Model Card Contact [https://t.me/grozaae] # Uploaded model - **Developed by:** mcgrozov - **License:** apache-2.0 - **Finetuned from model :** unsloth/meta-llama-3.1-8b-bnb-4bit This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library. [](https://github.com/unslothai/unsloth)