Hugging Face's logo

TomGrc
/
FusionNet_7Bx2_MoE_14B

Text Generation
Transformers
Safetensors
English
mixtral
Mixture of Experts
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
11

GGUF Version

#1
by SimSim93 - opened
Discussion
SimSim93
•
edited Jan 14

First of all, thanks a lot for your mixture of expert model!
Just out of curiosity, which models did you merge?

@TheBloke could you please provide us with a gguf quant version? =)

Thank you all for the awesome work you do for the community!

SimSim93 changed discussion title from GGML Version to GGUF Version
DKRacingFan

@TheBloke this would be great!

testerav

yeah waiting

Pumba2
•
edited Jan 15

Leaderboard shows some impressive results...

pramjan

@TheBloke would really appreciate a GGUF version for this model, thank you!

johnnnna

@TheBloke please consider making a GGUF version of this. Just look at its scores

Pumba2
•
edited Jan 16

Strange @TheBloke is not doing gguf for this model.
EDIT: i meant it doesnt seem so as its 16th and he must have received the notifications regarding this post.

testerav

Strange @TheBloke is not doing gguf for this model.

did you ask him?

FlorianJc

TheBloke is not your slave. Is it too hard to call a Python script to do it yourself ?

johnnnna

TheBloke is not your slave. Is it too hard to call a Python script to do it yourself ?

I said "please consider". Maybe you need to learn to read better

testerav
•
edited Jan 21

TheBloke is not your slave. Is it too hard to call a Python script to do it yourself ?

among the tons of models he compiles , what's wrong with doing one on the leaderboard.
Also isn't that the reason people know him?

Nan-Do

I have generated the gguf quantized version of the model.
The files can be found at https://huggingface.co/Nan-Do/FusionNet_7Bx2_MoE_14B-GGUF

johnnnna
•
edited Jan 19

I have generated the gguf quantized version of the model.
The files can be found at https://huggingface.co/Nan-Do/FusionNet_7Bx2_MoE_14B-GGUF

Thx @Nan-Do , I will try it asap

testerav

@johnnnna pls do give feeback , wonder if its actually good or hoax.

Rybens

@Nan-Do 's quants does not work for me (it generates random tokens) so I made my own basic quants of this model.

https://huggingface.co/Rybens/FusionNet_7Bx2_MoE_14B_gguf

Nan-Do

@Rybens there was indeed a problem with the uploaded files, thanks for letting me know. I should have checked the uploaded files.

I have redone the repository (and re-checked the files) and it should be fine now, also I have added a model's card.

Rybens
•
edited Jan 20

Thanks @Nan-Do
But I'll leave my repository with quants in case anyone needs it

Nan-Do
•
edited Jan 20

@Rybens sure, that's good.

DKRacingFan
•
edited Jan 20

Thanks @Nan-Do
But I'll leave my repository with quants in case anyone needs it

Just tried out the model in LM Studio. Works very well! I'm impressed!

TomGrc changed discussion status to closed

Sign up or log in to comment