Qwen2-0.5B-Instruct-SQL-query-generator

This model is a fine-tuned version of Qwen/Qwen2-0.5B-Instruct on the motherduckdb/duckdb-text2sql-25k dataset (first 10k rows).

Model Description

The Qwen2-0.5B-Instruct-SQL-query-generator is a specialized model fine-tuned to generate SQL queries from natural language text prompts. This fine-tuning allows the model to better understand and convert text inputs into corresponding SQL queries, facilitating tasks such as data retrieval and database querying through natural language interfaces.

Intended Uses & Limitations

Intended Uses

• Convert natural language questions to SQL queries.
• Facilitate data retrieval from databases using natural language.
• Assist in building natural language interfaces for databases.

Limitations

• The model is fine-tuned on a specific subset of data and may not generalize well to all SQL query formats or databases.
• It is recommended to review the generated SQL queries for accuracy and security, especially before executing them on live databases.

Training and Evaluation Data

Training Data

The model was fine-tuned on the motherduckdb/duckdb-text2sql-25k dataset, specifically using the first 10,000 rows. This dataset includes natural language questions and their corresponding SQL queries, providing a robust foundation for training a text-to-SQL model.

Evaluation Data

The evaluation data used for fine-tuning was a subset of the same dataset, ensuring consistency in training and evaluation metrics.

Training Procedure

Github Code: https://github.com/omaratef3221/SQL_Query_Generator_llm/

Training Hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-4
• train_batch_size: 8
• save_steps: 1
• logging_steps: 500
• num_epochs: 5

Training Frameworks

• Transformers: 4.39.0
• PyTorch: 2.2.0
• Datasets: 2.20.0
• Tokenizers: 0.15.2

Training Results

During the training process, the model was periodically evaluated to ensure it was learning effectively. The specific training metrics and results were logged for further analysis.

Model Performance

Evaluation Metrics

• Evaluation metrics such as accuracy, precision, recall, and F1-score were used to assess the model's performance. (Specific values can be added here if available.)

Usage

To use this model, simply load it from the Hugging Face Model Hub and provide natural language text prompts. The model will generate the corresponding SQL queries.

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("omaratef3221/Qwen2-0.5B-Instruct-SQL-query-generator")
model = AutoModelForSeq2SeqLM.from_pretrained("omaratef3221/Qwen2-0.5B-Instruct-SQL-query-generator")

inputs = tokenizer("Show me all employees with a salary greater than $100,000", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))