NameError: name 'torch' is not defined
My computer does not have a GPU, so the CPU should be used for calculations. But it will report an error every time I run it.
Below is the output error:
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
bin F:\anaconda\envs\godel-env\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so
F:\anaconda\envs\godel-env\lib\site-packages\bitsandbytes\cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
warn("The installed version of bitsandbytes was compiled without GPU support. "
'NoneType' object has no attribute 'cadam32bit_grad_fp32'
CUDA SETUP: Loading binary F:\anaconda\envs\godel-env\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so...
argument of type 'WindowsPath' is not iterable
Detected the presence of a quantization_config
attribute in the model's configuration but you don't have the correct bitsandbytes
version to support int8 serialization. Please install the latest version of bitsandbytes
with pip install --upgrade bitsandbytes
.
Loading checkpoint shards: 0%| | 0/2 [00:07<?, ?it/s]
Traceback (most recent call last):
File "jais8bit.py", line 18, in
model = AutoModelForCausalLM.from_pretrained("F:\code\chat-py\jais-13b-chat-8bit", device_map="auto", offload_folder="offload", trust_remote_code=True)
File "F:\anaconda\envs\godel-env\lib\site-packages\transformers\models\auto\auto_factory.py", line 560, in from_pretrained
return model_class.from_pretrained(
File "F:\anaconda\envs\godel-env\lib\site-packages\transformers\modeling_utils.py", line 3307, in from_pretrained
) = cls._load_pretrained_model(
File "F:\anaconda\envs\godel-env\lib\site-packages\transformers\modeling_utils.py", line 3695, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
File "F:\anaconda\envs\godel-env\lib\site-packages\transformers\modeling_utils.py", line 749, in _load_state_dict_into_meta_model
set_module_quantized_tensor_to_device(
File "F:\anaconda\envs\godel-env\lib\site-packages\transformers\integrations\bitsandbytes.py", line 58, in set_module_quantized_tensor_to_device
if old_value.device == torch.device("meta") and device not in ["meta", torch.device("meta")] and value is None:
NameError: name 'torch' is not defined
Have you tried to install torch
(pip install torch
)? If you have it installed already, try to import it as a package in your code (import torch
).