Cannot deploy Phi-3-mini-4k-instruct model on AWS Sagemaker

#51

by MajTad - opened May 2

May 2

I try to run a sagemaker endpoint with the Phi-3-mini-4k-instruct using HuggingFaceModel to load it but I got from Sagemaker logs the following error:

ValueError: Unsupported model type phi3.

Does anyone face the same problem.

amgadsahl

May 2

It's only made to run on Azure
/s

Seriously, not all models are supported by the HuggingFaceModel feature on SageMaker. You can just deploy it on a regular gpu-powered vm using vLLM or TGI

MajTad

May 2

Thanks @amgadsahl for your answer.
I'll manage to deploy based on your recommandations.

nguyenbh changed discussion status to closed May 18

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment