openbmb

/

MiniCPM-V-2

like 423

Follow

OpenBMB 341

It should be that the input exceeds the maximum length ~ you can try some cases with only one image input. Or try MiniCPM-V 2.6

@akm78
This might be related to the generation code at "/root/.cache/huggingface/modules/transformers_modules/openbmb/MiniCPM-V-2/.../modeling_minicpm.py", line 1489, in prepare_inputs_for_generation

if inputs_embeds is not None and past_key_values is None:

Here the model needs to take the context embeddings as input at the 1st generation step. However, the "past_key_values" is empty but not "None", so at the 1st step, "model_inputs" consists neither "inputs_embeds" nor "input_ids" (presisely, "input_ids" is empty). It means that the q,k,v are empty, which leads to empty "cos", "sin" calculated by rotary embedding. So at "/root/.cache/huggingface/modules/transformers_modules/openbmb/MiniCPM-V-2/.../modeling_minicpm.py", line 284, in apply_rotary_pos_emb

cos = cos[position_ids].unsqueeze(unsqueeze_dim)  # [bs, 1, seq_len, dim]

the "cos" is empty while the "position_ids" is not, which leads to IndexError.

Since the code of "prepare_inputs_for_generation" function for MiniCPM-V-2 is a little bit different with that for MiniCPM-V-2.6, I tried to change it like this:

if inputs_embeds is not None and past_length==0:

Then the generation process seems to be fine.

@Leafnwind your change fixed it for me as well, thanks

改为"if inputs_embeds is not None and past_length==0:"成功运行

@akm78
This might be related to the generation code at "/root/.cache/huggingface/modules/transformers_modules/openbmb/MiniCPM-V-2/.../modeling_minicpm.py", line 1489, in prepare_inputs_for_generation

if inputs_embeds is not None and past_key_values is None:

Here the model needs to take the context embeddings as input at the 1st generation step. However, the "past_key_values" is empty but not "None", so at the 1st step, "model_inputs" consists neither "inputs_embeds" nor "input_ids" (presisely, "input_ids" is empty). It means that the q,k,v are empty, which leads to empty "cos", "sin" calculated by rotary embedding. So at "/root/.cache/huggingface/modules/transformers_modules/openbmb/MiniCPM-V-2/.../modeling_minicpm.py", line 284, in apply_rotary_pos_emb

cos = cos[position_ids].unsqueeze(unsqueeze_dim)  # [bs, 1, seq_len, dim]

the "cos" is empty while the "position_ids" is not, which leads to IndexError.

Since the code of "prepare_inputs_for_generation" function for MiniCPM-V-2 is a little bit different with that for MiniCPM-V-2.6, I tried to change it like this:

if inputs_embeds is not None and past_length==0:

Then the generation process seems to be fine.