8-kv-heads
#14
by
ArthurZ
HF staff
- opened
No description provided.
Hi there! I noticed that the 8 kv heads PR was merged in for the other 405b checkpoints, is there an ETA on landing this one? Thanks for the help!
Merging now!
ArthurZ
changed pull request status to
open
Does num_key_value_heads in config.json need to be updated as well after this PR is merged?
TYSM
@ArthurZ
!! just a heads up tho that this is probably an upload issue, but it appears that model parts update: those 4 files are not affected by the 16 -> 8 kv head change. { 002, [ 107 - 109 ] } were missed from the list / diff above
Looks like this is not yet merged?
Let me update the value in the config to merge!
(I don't have rights yet 😿)
Looking forward to trying!
osanseviero
changed pull request status to
merged