The model parameters have been renamed and a config file has been provided to enable its usage with the CLIPModel in the transformers library.
#2
by
zooblastlbz
- opened
The model parameters have been renamed and a config file has been provided to enable its usage with the CLIPModel in the transformers library.
zooblastlbz
changed pull request title from
Upload 3 files
to The model parameters have been renamed and a config file has been provided to enable its usage with the CLIPModel in the transformers library.
I tested this PR and encountered RuntimeErrors for many mismatched sizes for various layers.
>>> from transformers import CLIPModel
>>> CLIPModel.from_pretrained('.')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/workCLIP-ViT-B-16-laion2B-s34B-b88K/venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3941, in from_pretrained
) = cls._load_pretrained_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/work/CLIP-ViT-B-16-laion2B-s34B-b88K/venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4473, in _load_pretrained_model
raise RuntimeError(f"Error(s) in loading state_dict for {model.__class__.__name__}:\n\t{error_msg}")
RuntimeError: Error(s) in loading state_dict for CLIPModel:
size mismatch for text_model.encoder.layers.0.self_attn.k_proj.weight: copying a param with shape torch.Size([768, 512]) from checkpoint, the shape in current model is torch.Size([512, 512]).
size mismatch for text_model.encoder.layers.0.self_attn.k_proj.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([512]).
The names for all errors follow the pattern text_model.encoder.layers.*.self_attn.*.