Panda-Coder πΌ
Panda Coder-13B vLLM Inference:
Panda Coder is a state-of-the-art LLM capable of generating code on the NLP based Instructions
Model description
π€ Model Description: Panda-Coder is a state-of-the-art LLM, a fine-tuned model, specifically designed to generate code based on natural language instructions. It's the result of relentless innovation and meticulous fine-tuning, all to make coding easier and more accessible for everyone.
Inference
Hardware requirements:
30GB VRAM - A100 Preferred
vLLM - For Faster Inference
Installation
!pip install vllm
Implementation:
from vllm import LLM, SamplingParams
llm = LLM(model='aiplanet/panda-coder-13B',gpu_memory_utilization=0.95,max_model_len=4096)
prompts = [""" ### Instruction: Write a Java code to add 15 numbers randomly generated.
### Input: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
### Response:
""",
"### Instruction: write a neural network complete code in Keras ### Input: Use cifar dataset ### Response:"
]
sampling_params = SamplingParams(temperature=0.1, top_p=0.95,repetition_penalty = 1.1,max_tokens=1000)
outputs = llm.generate(prompts, sampling_params)
for output in outputs:
prompt = output.prompt
generated_text = output.outputs[0].text
print(generated_text)
print("\n\n")
Transformers - Basic Implementation
import torch
import transformers
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments,BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
model = "aiplanet/panda-coder-13B"
base_model = AutoModelForCausalLM.from_pretrained(model, quantization_config=bnb_config, device_map="cuda")
tokenizer = AutoTokenizer.from_pretrained(model, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"
prompt = f"""### Instruction:
Below is an instruction that describes a task. Write a response that appropriately completes the request.
Write a Python quickstart script to get started with TensorFlow
### Input:
### Response:
"""
input_ids = tokenizer(prompt, return_tensors="pt", truncation=True).input_ids.cuda()
outputs = base_model.generate(input_ids=input_ids, max_new_tokens=512, do_sample=True, top_p=0.9,temperature=0.1,repetition_penalty=1.1)
print(f"Output:\n{tokenizer.batch_decode(outputs.detach().cpu().numpy(), skip_special_tokens=True)[0][len(prompt):]}")
Output
Output:
import tensorflow as tf
# Create a constant tensor
hello_constant = tf.constant('Hello, World!')
# Print the value of the constant
print(hello_constant)
Prompt Template for Panda Coder 13B
### Instruction:
{<add your instruction here>}
### Input:
{<can be empty>}
### Response:
π Key Features:
π NLP-Based Coding: With Panda-Coder, you can transform your plain text instructions into functional code effortlessly. No need to grapple with syntax and semantics - it understands your language.
π― Precision and Efficiency: The model is tailored for accuracy, ensuring your code is not just functional but also efficient.
β¨ Unleash Creativity: Whether you're a novice or an expert coder, Panda-Coder is here to support your coding journey, offering creative solutions to your programming challenges.
π Evol Instruct Code: It's built on the robust Evol Instruct Code 80k-v1 dataset, guaranteeing top-notch code generation.
π’ What's Next?: We believe in continuous improvement and are excited to announce that in our next release, Panda-Coder will be enhanced with a custom dataset. This dataset will not only expand the language support but also include hardware programming languages like MATLAB, Embedded C, and Verilog. π§°π‘
Get in Touch
You can schedule 1:1 meeting with our DevRel & Community Team to get started with AI Planet Open Source LLMs and GenAI Stack. Schedule the call here: https://calendly.com/jaintarun
Stay tuned for more updates and be a part of the coding evolution. Join us on this exciting journey as we make AI accessible to all at AI Planet!
Framework versions
- Transformers 4.33.3
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.13.3
Citation
@misc {lucifertrj,
author = { {Tarun Jain} },
title = { Panda Coder-13B by AI Planet},
year = 2023,
url = { https://huggingface.co/aiplanet/panda-coder-13B },
publisher = { Hugging Face }
}
- Downloads last month
- 39
Model tree for aiplanet/panda-coder-13B
Base model
codellama/CodeLlama-13b-Instruct-hf