THUDM

/

chatglm-6b

like 2.81k

https://huggingface.co/THUDM/chatglm-6b/commit/fdb7a601d8f8279806124542e11549bdd76f62f6

这段代码：

tokens = tokenizer(text, return_tensors='pt')['input_ids'].cuda()
gen_kwargs = {"max_length": 512, "num_beams": 1, "do_sample": True, "top_p": 0.7,
                "temperature": 0.95}
outputs = model.generate(input_ids=tokens, **gen_kwargs)
preds = tokenizer.batch_decode(outputs, skip_special_tokens=True)

用最新的版本decode会报这个错误：

Traceback (most recent call last):                                                                            
  File "inference_adgen.py", line 28, in <module>                                                             
    preds = tokenizer.batch_decode(outputs, skip_special_tokens=True)                                         
  File "/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py", line 3437, in batch_d
ecode                                                                                                         
    return [                                                                                                  
  File "/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py", line 3438, in <listco
mp>                                                                                                           
    self.decode(                                                                                              
  File "/root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/fdb7a601d8f8279806124542e11549b
dd76f62f6/tokenization_chatglm.py", line 276, in decode                                                       
    if self.pad_token_id in token_ids:  # remove pad                                                          
RuntimeError: Boolean value of Tensor with more than one value is ambiguous

断点打印对应的变量如下：

> /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/fdb7a601d8f8279806124542e11549bdd76f6
2f6/tokenization_chatglm.py(277)decode()
-> if self.pad_token_id in token_ids:  # remove pad
(Pdb) p self.pad_token_id
20003
(Pdb) p token_ids
[tensor([ 20005,  85421,  20061,  95898,  20032,  88554,  20061,  97257,  84555,
         20032,  85107,  20061,  86268,  20032,  85347,  20061,  91689,  20032,
         89768,  20061, 105428,  20032,  85173,  93942,  20061,  90984,  20032,
         85173,  90936,  20061,  84703,  85509, 150001, 150004,  20005,  84703,
         85509,  94681,  83829,  85388,  85023,  92871, 106418,  83825,  94707,
         20006,  83827,  94681,  83839,  84690, 105428,  89768,  20006,  85109,
         94705, 107834,  20006,  83887,  98530,  85425, 116447,  20006,  97284,
         83842, 101233, 109473, 109473,  83825, 139235,  83823,  92194,  91231,
         20006,  87122,  91689,  20006,  86955,  86800,  83823, 150005],
       device='cuda:0')]
(Pdb)