support vllm
#10
by
CarrotAI
- opened
hello upstage
Thank you for sharing the model.
To make it easier for me to use it, I asked vllm for PR by referring to the given code. Can you explain the difference with Llama in more detail? And will the structure of the Pro model be changed in the future?
Yes please!
@CarrotAI
Thank you for your interest!
The architectural difference from Llama is the presence of the BSKCN (Block level SKip CoNnection). The rope scaling mentioned in the PR is not different from Llama. We will officially review your PR and offer assistance in case any challenges or difficulties arise.
Thank you. It's been merged.
CarrotAI
changed discussion status to
closed