Can not host as vLLM ?
#17
by
tommywu052
- opened
when I try to host that as API endpoint, vllm serve "nvidia/NVLM-D-72B" --trust-remote-code
it will throw the error as
Model architectures ['NVLM_D'] are not supported for now
We currently do not support vLLM but are actively working on integrating NVLM with vLLM. Our team is committed to delivering this support as soon as possible.
Thanks,
Boxin
Seems supported in this pr https://github.com/vllm-project/vllm/pull/9045, but not yet released.
Installing vllm from the latest code might work.
checking out from latest main and building docker image worked for me