Create README.md

**Model Name: Instruct-Kiswallama7b**

*Instruct-Kiswallama7b* is a language model that builds upon the foundation of *Jacaranda/kiswallama-pretrained* which is a swahili continual pre-trained version of Meta/Llama2 -7b. This particular model represents the 7B variant of Llama 2, meticulously converted to the Hugging Face Transformers format for accessibility.

**Model Details**
*Instruct-Kiswallama7b* was created by adapting the *Jacaranda/kiswallama-pretrained* model with an instruction dataset in Swahili and English comprising 140, 389 prompt-response pairs.The training took the shape of aligning with conventional methodologies; integrating task-specific heads into the model and delicately adjusting the neural network's weights through backpropagation, all while being intricately attuned to the nuanced requirements of the task at hand. Recognizing the merits of LoRA, two smaller matrices that approximate the larger matrix of the *Jacaranda/kiswallama-pretrained* were fine-tuned. These matrices constituted the Low Rank Adapter(LoRa) for the instruction finetuning. At the end of training the trained LoRa was extracted and we used the merge and unload () function provided by hugging face to merge the adapter weights (for the trained Instruct-Kiswallama7b) with the base model (*Jacaranda/kiswallama-pretrained*) which will allow us to effectively use our newly merged model as a standalone model for inference.

**Intended uses**
*Instruct-Kiswallama7b* is mostly intended to be further fine-tuned on different downstream tasks especially the ones which involve instruction datasets requiring swahili, english or a mix of both. The model can be further fine tuned for In-domain question-answering/ assistant-like chat functionalities - e.g healthcare use cases, agricultural use cases, government services, communication, customer support, retail/trade e.t.c
The continually pretrained version- *Jacaranda/kiswallama-pretrained* can be further used for functions like:
Text Summarization for Swahili/English
Text Completion and Autoregression for Swahili/English
Content Creation for Swahili/English
Paraphrasing, Grammar Correction and Editing for Swahili/English

*This is a static model trained on an offline dataset*

**Ethical Considerations and Limitations**

Introducing *Instruct-Kiswallama7b* a cutting-edge technology brimming with possibilities, yet not without its inherent risks. The extensive testing conducted thus far has been predominantly in Swahili, English and , leaving an expansive terrain of uncharted scenarios. Consequently, like its LLM counterparts, *Instruct-Kiswallama7b* outcome predictability remains elusive, and there's the potential for it to occasionally generate responses that are either inaccurate, biased, or otherwise objectionable in nature when prompted by users.
With this in mind, the responsible course of action dictates that, prior to deploying *Instruct-Kiswallama7b* in any applications, developers must embark on a diligent journey of safety testing and meticulous fine-tuning, customized to the unique demands of their specific use cases.

**How to further fine-tune with an instruction dataset**

README.md

@@ -0,0 +1,15 @@
+---
+license: mit
+datasets:
+- swahili
+- mc4
+- swahili_news
+language:
+- sw
+- en
+metrics:
+- perplexity
+- bleu
+library_name: peft
+pipeline_tag: question-answering
+---