Update README.md

README.md

@@ -1,172 +1,83 @@
 ---
 library_name: transformers
-tags: []
+tags: [LLM, fine-tuning, SQL, Llama 2, PEFT]
 ---
 
 # Model Card for Model ID
 
-<!-- Provide a quick summary of what the model is/does. -->
-
+This model card presents the enhanced Llama 2 model, fine-tuned for SQL programming and deployed using the Yale High Performance Computing (HPC) platform. The project focuses on leveraging the computational power of Yale HPC to push the boundaries of what Large Language Models (LLMs) can achieve, specifically in the context of SQL programming.
 
 
 ## Model Details
 
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+### Model Description
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+This model aimed at advancing the capabilities of LLMs in programming languages, with a focus on SQL. The Llama 2 model, developed by Meta AI, serves as the foundation for this project. It has been fine-tuned using a Parameter Efficient Fine-Tuning (PEFT) approach, integrating techniques such as Low-Rank Adaptation (LoRA) and Retrieval Augmented Generation (RAG) to enhance its SQL programming assistance capabilities.
 
-### Model Sources [optional]
+- **Developed by:** Kaifeng Gao, Jiayi Chen, Yuntian Liu, Yixiao Chen
+- **Model type:** Large Language Model (Llama 2)
+- **Language(s) (NLP):** English
+- **License:** TBD
+- **Finetuned from model:** Llama 2
 
-<!-- Provide the basic links for the model. -->
+### Model Sources
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Repository:** https://github.com/Kaifeng-Gao/Llama-7b-HPC
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+The model is designed to assist developers in writing efficient and accurate SQL queries by providing contextually relevant suggestions and explanations. It can be directly used by SQL programmers of all skill levels to improve their query writing process.
 
-[More Information Needed]
 
-### Downstream Use [optional]
+### Downstream Use
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+The model can serve as a backend for educational tools, IDE plugins, or other applications that require SQL query generation or optimization.
 
-[More Information Needed]
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+Uses that involve tasks far removed from SQL programming or those that require real-time interaction with live databases may not be suitable.
 
-[More Information Needed]
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+The model's performance and output quality are directly tied to the training dataset. As such, any biases or inaccuracies in the dataset could be reflected in the model's suggestions.
 
-[More Information Needed]
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+Users should verify the model's suggestions against best practices and the latest SQL standards. Ongoing evaluation and refinement with updated datasets are recommended to mitigate biases and improve performance.
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
-[More Information Needed]
+Refer to the project's GitHub repository for detailed instructions on deploying and interacting with the model through the Streamlit web application.
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+The model was fine-tuned using the "b-mc2/sql-create-context" dataset, enriched with SQL questions and their corresponding answers covering a broad range of concepts.
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
+#### Preprocessing
+The dataset was structured into a format conducive to efficient learning by employing templates that transformed raw data into a series of instructions and answers.
 
 #### Training Hyperparameters
+Training regime: The model leveraged LoRA for efficient adaptation, alongside model quantization for reduced memory footprint.
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
+Post fine-tuning and RAG integration, the model showed significant improvement in generating correct SQL syntax and providing comprehensive information from SQL tutorial websites. Some responses may contain minor syntax errors, attributed to the initial training dataset quality.
 
 ## Environmental Impact
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+The use of Yale HPC resources aimed at efficient computational usage, though specific metrics on carbon emissions and electricity usage are pending further analysis.
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
 
 ## Citation [optional]