Not able to load locally downloaded model using SentenceTransformer

#16
by umesh-c - opened

My simple code to load the model using locally downloaded path:

from sentence_transformers import SentenceTransformer
self.model = SentenceTransformer(self.model_path)

model_path is : models/downloads/mxbai-embed-large-v1

ll models/downloads/mxbai-embed-large-v1                                                                                      
total 7248200
drwxr-xr-x  3 umesh  staff    96B Aug 28 15:43 1_Pooling
-rw-r--r--  1 umesh  staff    11K Aug 28 15:43 LICENSE
-rw-r--r--  1 umesh  staff   111K Aug 28 15:43 README.md
-rw-r--r--  1 umesh  staff   677B Aug 28 15:43 config.json
-rw-r--r--  1 umesh  staff   171B Aug 28 15:43 config_sentence_transformers.json
drwxr-xr-x  3 umesh  staff    96B Aug 28 15:43 gguf
-rw-r--r--  1 umesh  staff   1.2G Aug 28 15:45 model.onnx
-rw-r--r--  1 umesh  staff   639M Aug 28 15:44 model.safetensors
-rw-r--r--  1 umesh  staff   638M Aug 28 15:46 model_fp16.onnx
-rw-r--r--  1 umesh  staff   321M Aug 28 15:46 model_quantized.onnx
-rw-r--r--  1 umesh  staff   229B Aug 28 15:43 modules.json
-rw-r--r--  1 umesh  staff   639M Aug 28 15:44 mxbai-embed-large-v1-f16.gguf
drwxr-xr-x  5 umesh  staff   160B Aug 28 15:43 onnx
-rw-r--r--  1 umesh  staff    53B Aug 28 15:43 sentence_bert_config.json
-rw-r--r--  1 umesh  staff   695B Aug 28 15:43 special_tokens_map.json
-rw-r--r--  1 umesh  staff   695K Aug 28 15:43 tokenizer.json
-rw-r--r--  1 umesh  staff   1.2K Aug 28 15:43 tokenizer_config.json
-rw-r--r--  1 umesh  staff   226K Aug 28 15:43 vocab.txt

Lib versions are as follows :

Python 3.11.9
sentence-transformers                   3.0.1
torch                                   2.4.0

The error which I am getting with below thread dump:

Fatal Python error: Aborted

Current thread 0x00000001f59e2500 (most recent call first):
  File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1160 in convert
  File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 805 in _apply
  File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780 in _apply
  File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780 in _apply
  File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780 in _apply
  File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780 in _apply
  File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1174 in to
  File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 316 in __init__
  File "/Users/umesh/git-repos/genai_search/models/impl/mxbai_embed_large_v1.py", line 13 in __init__

Clueless what might be wrong as I am doing same as mentioned in the README. Can someone point the possible issue?
BTW, I am able to load the bge-large-en-v1.5 model just fine using SentenceTransformer.

Thanks!

Facing same issue, were you able to find the solution?

Try to use SentenceTransformer like this:

        # setting device as GPU if available, else CPU
        device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
        self.model = SentenceTransformer(self.model_path, device=device)

Thanks for reply, I am using machine with no internet access; I think it is trying to download something.
Getting this error.
Cannot find an appropriate cached snapshot folder for the specified revision on the local disk and outgoing traffic has been disabled. To enable repo look-ups and downloads online

Sign up or log in to comment