Upload model

README.md

@@ -1,231 +1,208 @@
 ---
-license: llama2
 library_name: peft
-tags:
-- llama-2
-datasets:
-- ehartford/dolphin
-inference: false
-pipeline_tag: text-generation
 base_model: meta-llama/Llama-2-13b-hf
 ---
 
-<div align="center">
+# Model Card for Model ID
 
-<img src="./assets/llama.png" width="150px">
+<!-- Provide a quick summary of what the model is/does. -->
 
-</div>
 
-# Llama-2-13B-Instruct-v0.1
 
-This instruction model was built via parameter-efficient QLoRA finetuning of [llama-2-13b](https://huggingface.co/meta-llama/Llama-2-13b-hf) on the first 100k rows of [ehartford/dolphin](https://huggingface.co/datasets/ehartford/dolphin) (an open-source implementation of [Microsoft's Orca](https://www.microsoft.com/en-us/research/publication/orca-progressive-learning-from-complex-explanation-traces-of-gpt-4/)). Finetuning was executed on a single A6000 (48 GB) for roughly 18 hours on the [Lambda Labs](https://cloud.lambdalabs.com/instances) platform.
+## Model Details
 
-## Benchmark metrics
+### Model Description
 
-| Metric                | Value |
-|-----------------------|-------|
-| MMLU (5-shot)         | 55.76 |
-| ARC (25-shot)         | 59.98 |
-| HellaSwag (10-shot)   | 82.43 |
-| TruthfulQA (0-shot)   | 42.59 |
-| Avg.                  | 60.19 |
+<!-- Provide a longer summary of what this model is. -->
 
-We use state-of-the-art [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) to run the benchmark tests above, using the same version as Hugging Face's [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
 
-## Helpful Links
 
-* Model license: Llama 2 Community License Agreement
-* Basic usage: [notebook](assets/basic_inference_llama_2_13b_dolphin.ipynb)
-* Finetuning script: [script](https://github.com/daniel-furman/sft-demos/blob/main/src/sft/one_gpu/llama-2/dolphin/sft-Llama-2-13B-Instruct-v0.1-peft.py)
-* Loss curves: [plot](https://huggingface.co/dfurman/Llama-2-13B-Instruct-v0.1-peft#finetuning-description)
-* Runtime stats: [table](https://huggingface.co/dfurman/Llama-2-13B-Instruct-v0.1-peft#runtime-tests)
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
 
-## Example prompts and responses
+### Model Sources [optional]
 
-Example 1: 
+<!-- Provide the basic links for the model. -->
 
-**User**:
->You are a helpful assistant. Write me a numbered list of things to do in New York City.\n
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
 
-**Llama-2-13B-Instruct-v0.1-peft**:
+## Uses
 
-> 1. Visit the iconic Statue of Liberty and Ellis Island.
-> 2. Take a stroll through Central Park and enjoy its many attractions.
-> 3. Explore the world-renowned museums, such as the Metropolitan Museum of Art and the Museum of Modern Art.
-> 4. Experience the vibrant energy of Times Square and take in the bright lights and billboards.
-> 5. Visit the 9/11 Memorial and Museum to pay tribute to those who lost their lives in the attacks.
-> 6. Enjoy a Broadway show or a concert at one of the many theaters and venues in the city.
-> 7. Take a ride on the Staten Island Ferry for a free view of the Statue of Liberty and the New York City skyline.
-> 8. Shop at the famous Fifth Avenue stores and explore the high-end boutiques.
-> 9. Indulge in a variety of cuisines at one of the many restaurants in the city.
-> 10. Visit the Empire State Building and enjoy the panoramic views of the city from the observation deck.
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
-<br>
+### Direct Use
 
-Example 2: 
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
-**User**:
->You are a helpful assistant. Write a short email inviting my friends to a dinner party on Friday. Respond succinctly.\n"
+[More Information Needed]
 
-**Llama-2-13B-Instruct-v0.1-peft**:
+### Downstream Use [optional]
 
-> Subject: Friday Dinner Party Invitation
-> 
-> Dear Friends,
-> 
-> I hope this email finds you well. I'm excited to invite you all to a dinner party on Friday, March 10th, at 7:00 PM. The address is 123 Main Street, Anytown, USA.
-> 
-> Please RSVP by Wednesday, March 8th, so I can plan accordingly. I look forward to seeing you all and sharing a delicious meal together!
-> 
-> Best,
-> Your Friendly Assistant
-> 
-> P.S. If you have any dietary restrictions or allergies, please let me know in your RSVP. Thank you!
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
 
-<br>
+[More Information Needed]
 
-## Model Description
+### Out-of-Scope Use
 
-The architecture is a modification of a standard decoder-only transformer.
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
 
-The llama-2-13b models have been modified from a standard transformer in the following ways:
-* It uses the [SwiGLU activation function](https://arxiv.org/abs/2002.05202)
-* It uses [rotary positional embeddings](https://arxiv.org/abs/2104.09864) (RoPE)
+[More Information Needed]
 
-| Hyperparameter | Value |
-|----------------|-------|
-| n_parameters | 13B |
-| tokens | 2.0T |
-| vocab size | 32000 |
-| sequence length | 4096 |
+## Bias, Risks, and Limitations
 
-## Finetuning Description
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
-This model was trained on a single A6000 (48 GB) for about 18 hours using the [Lambda Labs](https://cloud.lambdalabs.com/instances) platform.
+[More Information Needed]
 
-![loss curves](https://raw.githubusercontent.com/daniel-furman/sft-demos/main/assets/jul_24_23_1_13_00_log_loss_curves_Llama-2-13B-Instruct-v0.1.png)
+### Recommendations
 
-The above loss curve was generated from the run's private wandb.ai log.  
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
 
-## PreTraining Data
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
 
-For more details on the pretraining process, see [Llama-2-13b-hf](https://huggingface.co/meta-llama/Llama-2-13b-hf).
+## How to Get Started with the Model
 
-The data was tokenized using the [Llama-2-13b-hf](https://huggingface.co/meta-llama/Llama-2-13b-hf) tokenizer.
+Use the code below to get started with the model.
 
-## Limitations and Biases
+[More Information Needed]
 
-_The following language is modified from [EleutherAI's GPT-NeoX-20B](https://huggingface.co/EleutherAI/gpt-neox-20b)_
+## Training Details
 
-This model can produce factually incorrect output, and should not be relied on to produce factually accurate information.
-This model was trained on various public datasets.
-While great efforts have been taken to clean the pretraining data, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
+### Training Data
 
-## How to Use
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-* [notebook](assets/basic_inference_llama_2_dolphin.ipynb)
+[More Information Needed]
 
-```python
-!pip install -q -U huggingface_hub peft transformers torch accelerate
-```
+### Training Procedure 
 
-```python
-from huggingface_hub import notebook_login
-import torch
-from peft import PeftModel, PeftConfig
-from transformers import (
-    AutoModelForCausalLM,
-    AutoTokenizer,
-    BitsAndBytesConfig,
-    pipeline,
-)
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-notebook_login()
-```
+#### Preprocessing [optional]
 
-```python
-peft_model_id = "dfurman/Llama-2-13B-Instruct-v0.1-peft"
-config = PeftConfig.from_pretrained(peft_model_id)
+[More Information Needed]
 
-bnb_config = BitsAndBytesConfig(
-    load_in_4bit=True,
-    bnb_4bit_quant_type="nf4",
-    bnb_4bit_compute_dtype=torch.bfloat16,
-)
 
-model = AutoModelForCausalLM.from_pretrained(
-    config.base_model_name_or_path,
-    quantization_config=bnb_config,
-    use_auth_token=True,
-    device_map="auto",
-)
+#### Training Hyperparameters
 
-tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path, use_fast=True)
-tokenizer.pad_token = tokenizer.eos_token
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 
-model = PeftModel.from_pretrained(model, peft_model_id)
+#### Speeds, Sizes, Times [optional]
 
-format_template = "You are a helpful assistant. {query}\n"
-```
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
 
-```python
-# First, format the prompt
-query = "Tell me a recipe for vegan banana bread."
-prompt = format_template.format(query=query)
+[More Information Needed]
 
-# Inference can be done using model.generate
-print("\n\n*** Generate:")
+## Evaluation
 
-input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()
-with torch.autocast("cuda", dtype=torch.bfloat16):
-    output = model.generate(
-        input_ids=input_ids,
-        max_new_tokens=512,
-        do_sample=True,
-        temperature=0.7,
-        return_dict_in_generate=True,
-        eos_token_id=tokenizer.eos_token_id,
-        pad_token_id=tokenizer.pad_token_id,
-        repetition_penalty=1.2,
-    )
+<!-- This section describes the evaluation protocols and provides the results. -->
 
-print(tokenizer.decode(output["sequences"][0], skip_special_tokens=True))
-```
+### Testing Data, Factors & Metrics
 
-## Runtime tests
+#### Testing Data
 
+<!-- This should link to a Dataset Card if possible. -->
 
-| runtime / 50 tokens (sec) | GPU             | attn | torch dtype | VRAM (GB) |
-|:-----------------------------:|:----------------------:|:---------------------:|:-------------:|:-----------------------:|
-| 2.93                        | 1x A100 (40 GB SXM)  | torch               | bfloat16    | 25                    |
-| 3.24                        | 1x A6000 (48 GB)  | torch               | bfloat16    | 25                    |
+[More Information Needed]
 
-The above runtime stats were generated from this [notebook](https://github.com/daniel-furman/sft-demos/blob/main/src/sft/one_gpu/llama-2/dolphin/postprocessing-Llama-2-13B-Instruct-v0.1-peft.ipynb). 
+#### Factors
 
-## Acknowledgements
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
 
-This model was finetuned by Daniel Furman on July 22, 2023 and is intended primarily for research purposes.
+[More Information Needed]
 
-## Disclaimer
+#### Metrics
 
-The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please cosult an attorney before using this model for commercial purposes.
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
 
-## Meta citation for llama-2 blog
+[More Information Needed]
 
-```
-@online{Meta2023Introducing,
-    author    = {Meta AI},
-    title     = {Meta and Microsoft Introduce the Next Generation of Llama},
-    year      = {2023},
-    url       = {https://about.fb.com/news/2023/07/llama-2/},
-    note      = {Accessed: 2023-07-24},
-    urldate   = {2023-07-24}
-}
-```
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+## Training procedure
 
----
 
-## Framework versions
+### Framework versions
 
 
-- PEFT 0.5.0.dev0
+- PEFT 0.6.3.dev0

adapter_config.json

@@ -1,4 +1,5 @@
 {
+  "alpha_pattern": {},
   "auto_mapping": null,
   "base_model_name_or_path": "meta-llama/Llama-2-13b-hf",
   "bias": "none",
@@ -12,12 +13,13 @@
   "modules_to_save": null,
   "peft_type": "LORA",
   "r": 64,
+  "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
     "k_proj",
-    "v_proj",
-    "o_proj"
+    "q_proj",
+    "o_proj",
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM"
 }

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9f70dc35db3f30fd244fcbb6c1e7ba4488e89dda8c1b0033bacf8f0dc2b74dbe
-size 419474032
+oid sha256:809dc32759bbcaeae54c5098d86a1b5aded6b46875245adbf85f8c59a7cecbea
+size 209758976