Troubleshooting Model Conversion with DJL Library
Encountering a problem while converting this model using the DJL library. The traceback indicates an error: 'NoneType' object has no attribute 'dim'. Here is Exception and Traceback
Failed to trace model: nomic-ai/nomic-embed-text-v1.
'NoneType' object has no attribute 'dim'
Traceback (most recent call last):
File "/python/djl-git/djl/extensions/tokenizers/src/main/python/huggingface_converter.py", line 118, in jit_trace_model
script_module = torch.jit.trace(hf_pipeline.model,
File "/python/djl/lib/python3.10/site-packages/torch/jit/_trace.py", line 806, in trace
return trace_module(
File "/python/djl/lib/python3.10/site-packages/torch/jit/_trace.py", line 1074, in trace_module
module._c._create_method_from_trace(
File "/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _slow_forward
result = self.forward(*input, **kwargs)
File "/Users/surender/.cache/huggingface/modules/transformers_modules/nomic-ai/nomic-embed-text-v1/c9cf92b91ed6986f593ba836da7afdb8ed1ca85c/modeling_hf_nomic_bert.py", line 1069, in forward
attention_mask = self.get_extended_attention_mask(attention_mask, input_ids.shape)
File "/python/djl/lib/python3.10/site-packages/transformers/modeling_utils.py", line 959, in get_extended_attention_mask
if not (attention_mask.dim() == 2 and self.config.is_decoder):
Seeking insights, suggestions, or guidance on resolving this issue.
Grateful for any help or advice offered. Thank you!
can you post the code snippet you used to produce this? i'd have to guess that your attention mask is None which is causing this
Here are the steps to convert the model -
Step 1 Install the required packages listed in the requirements.txt file:
python3 -m pip install -r src/main/python/requirements.txt
Step 2 Run the model_zoo_importer.py script
python3 model_zoo_importer.py -m nomic-ai/nomic-embed-text-v1
This script will download and import the nomic-embed-text-v1 model from the Deep Java Library (DJL) model zoo
The requirements.txt file and the model_zoo_importer.py script can be accessed from the following GitHub repository:
https://github.com/deepjavalibrary/djl/blob/master/extensions/tokenizers/src/main/python/requirements.txt
https://github.com/deepjavalibrary/djl/blob/master/extensions/tokenizers/src/main/python/model_zoo_importer.py
README.md for Model Conversion
https://github.com/deepjavalibrary/djl/blob/master/extensions/tokenizers/README.md#usage
Can you retry now? i updated the code, it looks like the model is expecting the inputs in a specific order and they aren't passed in with keyword arguments: https://github.com/deepjavalibrary/djl/blob/master/extensions/tokenizers/src/main/python/huggingface_converter.py#L115
Hi Thanks for quick response
now I am getting this exception, if you can help
ERROR: Tensor-valued Constant nodes differed in value across invocations. This often indicates that the tracer has encountered untraceable code.
Node:
%70 : Tensor = prim::Constantvalue={2}, scope: __module.encoder/__module.encoder.layers.0/__module.encoder.layers.0.attn/__module.encoder.layers.0.attn.rotary_emb # /.cache/huggingface/modules/transformers_modules/nomic-ai/nomic-embed-text-v1/289f532e14dbbbd5a04753fa58739e9ba766f3c7/modeling_hf_nomic_bert.py:497:0
Source Location:
/.cache/huggingface/modules/transformers_modules/nomic-ai/nomic-embed-text-v1/289f532e14dbbbd5a04753fa58739e9ba766f3c7/modeling_hf_nomic_bert.py(497): apply_rotary_emb
/.cache/huggingface/modules/transformers_modules/nomic-ai/nomic-embed-text-v1/289f532e14dbbbd5a04753fa58739e9ba766f3c7/modeling_hf_nomic_bert.py(619): forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1501): _slow_forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1520): _call_impl
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl
/.cache/huggingface/modules/transformers_modules/nomic-ai/nomic-embed-text-v1/289f532e14dbbbd5a04753fa58739e9ba766f3c7/modeling_hf_nomic_bert.py(782): forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1501): _slow_forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1520): _call_impl
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl
/.cache/huggingface/modules/transformers_modules/nomic-ai/nomic-embed-text-v1/289f532e14dbbbd5a04753fa58739e9ba766f3c7/modeling_hf_nomic_bert.py(890): forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1501): _slow_forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1520): _call_impl
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl
/.cache/huggingface/modules/transformers_modules/nomic-ai/nomic-embed-text-v1/289f532e14dbbbd5a04753fa58739e9ba766f3c7/modeling_hf_nomic_bert.py(957): forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1501): _slow_forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1520): _call_impl
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl
/.cache/huggingface/modules/transformers_modules/nomic-ai/nomic-embed-text-v1/289f532e14dbbbd5a04753fa58739e9ba766f3c7/modeling_hf_nomic_bert.py(1070): forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1501): _slow_forward
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1520): _call_impl
/python/djl/lib/python3.10/site-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl
/python/djl/lib/python3.10/site-packages/torch/jit/_trace.py(1074): trace_module
/python/djl/lib/python3.10/site-packages/torch/jit/_trace.py(806): trace
/python/djl-git/djl/extensions/tokenizers/src/main/python/huggingface_converter.py(118): jit_trace_model
/python/djl-git/djl/extensions/tokenizers/src/main/python/huggingface_converter.py(65): save_model
/python/djl-git/djl/extensions/tokenizers/src/main/python/model_zoo_importer.py(53): main
/python/djl-git/djl/extensions/tokenizers/src/main/python/model_zoo_importer.py(72):
Comparison exception: The values for attribute 'shape' do not match: torch.Size([]) != torch.Size([7, 32]).
there's a decent chance this code is not jit traceable. if you want there are onnx weights however which should be serveable. you can also use our api in case that doesn't work