Recommended inference devices

#18

by AdrienVeepee - opened Oct 9

Oct 9

Hello, thanks for this model and work done on this project !
I'm designing a solution with NVLM-D-72B and I was looking at how much should we provision in terms of GPU ?
For testing puropose, will the H100 will be sufficient ?
How much RAM and VRAM should we target ? Thanks !

boxin-wbx

NVIDIA org Oct 9

Hi @AdrienVeepee ,

At least 2 GPUs (A100 / H100) would be needed for inference.

Thanks,
Boxin

AdrienVeepee changed discussion status to closed Oct 10

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment