Update README.md

README.md

@@ -8,6 +8,12 @@ metrics:
 model-index:
 - name: Llama-2-7b-hf-IDMGSP
   results: []
+license: mit
+datasets:
+- tum-nlp/IDMGSP
+language:
+- da
+library_name: transformers
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -15,19 +21,70 @@ should probably proofread and complete it, then remove this comment. -->
 
 # Llama-2-7b-hf-IDMGSP
 
-This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on the None dataset.
-It achieves the following results on the evaluation set:
+This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on the [tum-nlp/IDMGSP](https://huggingface.co/datasets/tum-nlp/IDMGSP) dataset.
+It achieves the following results on the evaluation split:
 - Loss: 0.1450
 - Accuracy: {'accuracy': 0.9759036144578314}
 - F1: {'f1': 0.9758125472411187}
 
 ## Model description
 
-More information needed
+Model loaded fine-tuned in 4bit quantization mode using LoRA.
 
 ## Intended uses & limitations
+Labels: `0` non-AI generated, `1` AI generated.
+
+For classifying AI generated text. Code to run the inference
+
+```
+import transformers
+import torch
+import datasets
+import numpy as np
+import torch
+from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training, PeftModel, AutoPeftModelForCausalLM, TaskType
+import bitsandbytes as bnb
+
+class Model():
+    def __init__(self, name) -> None:
+        # Hyperparams 
+        self.lr = 1e-4
+        self.epochs = 5
+        self.weight_decay = 0.01
+        self.dropout = 0.1
+        self.sequence_length = 512
+        self.batch_size = 32
+
+        # Tokenizer
+        self.tokenizer = transformers.LlamaTokenizer.from_pretrained(self.name)
+        self.tokenizer.pad_token = self.tokenizer.eos_token
+        print(f"Tokenizer: {self.tokenizer.eos_token}; Pad {self.tokenizer.pad_token}")
+
+        # Model
+        bnb_config = transformers.BitsAndBytesConfig(
+            load_in_4bit = True,
+            bnb_4bit_use_double_quant = True,
+            bnb_4bit_quant_type = "nf4",
+            bnb_4bit_compute_dtype = "bfloat16",
+        )
+        self.peft_config = LoraConfig(
+            task_type=TaskType.SEQ_CLS, r=8, lora_alpha=16, lora_dropout=0.05, bias="none"
+        )
+        self.model = transformers.LlamaForSequenceClassification.from_pretrained(self.name, 
+            num_labels=2,
+            quantization_config = bnb_config,
+            device_map = "auto"
+            )
+        self.model.config.pad_token_id = self.model.config.eos_token_id
+
+    def predict(self, text):
+        inputs = self.tokenize(text)
+        outputs = self.model(**inputs)
+        logits = outputs.logits
+        predictions = torch.argmax(logits, dim=-1)
+        return id2label[predictions.item()]
+```
 
-More information needed
 
 ## Training and evaluation data
 
@@ -37,6 +94,12 @@ More information needed
 
 ### Training hyperparameters
 
+BitsAndBytes and LoRA config parameters:
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/638f0f9ab0525fa370479467/XI1imFyXmzFjCGCkBYClc.png)
+
+GPU Consumption during training:
+
 The following hyperparameters were used during training:
 - learning_rate: 0.0001
 - train_batch_size: 32
@@ -65,4 +128,4 @@ The following hyperparameters were used during training:
 - Transformers 4.35.0
 - Pytorch 2.0.1
 - Datasets 2.14.6
-- Tokenizers 0.14.1
+- Tokenizers 0.14.1