num_key_value_heads bug?
#4
by
cckm
- opened
num_key_value_heads seems to be unreferenced in https://huggingface.co/MBZUAI/MobiLlama-05B/blob/main/modelling_mobillama.py?
num_key_value_heads seems to be unreferenced in https://huggingface.co/MBZUAI/MobiLlama-05B/blob/main/modelling_mobillama.py?