THUDM

/

chatglm2-6b

like 2.03k

Follow

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University 1,162

Hi~ Can anyone know the multi_query_group_num? Please help to understand the meaning of it.
"multi_query_attention": true,
"multi_query_group_num": 2,
"num_attention_heads": 32,
"num_layers": 28,