Check new, much better, version of this model

#5
by DreamGenX - opened
DreamGen org

This model has issues (trained without BOS token), please use the following preview models instead:

DreamGenX pinned discussion

But no quants :|

Yea, I'm waiting on quants as well. I can just BARELY not run the full model on my VRAM haha.

Well, I was watching this drama and wanted to wait till a more "final" version appears. But I've put both in the queue and a full set of static quants should be available in a few hours.

Spoke too soon:

NotImplementedError: Unknown rope scaling type: dynamic

the models are not supported by llama.cpp at the moment it seems. Not without disabling rope scaling at least.

DreamGen org

You can remove that from the config and use llama.cpp's own rope scaling.
Though I am surprised it throws an error like this.

llama.cpp can throw a lot of interesting errors, even with old models that did convert fine at the time :)

https://huggingface.co/mradermacher/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5-GGUF

BTW, anybody can request quants from me at https://huggingface.co/mradermacher/model_requests in cases I overlooked it. Can save the model creators a lot of time, too :)

DreamGen org

Awesome, thank you @mradermacher !

@DreamGenX , thanks for you work on these Llama3-8B finetunes. Any plans for a 70B finetune?

DreamGen org
edited Jun 10

@dobs I finished a train of L3 70B DreamGen model few weeks ago. I changed the template to take full advantage of the built-in tokens, so first I need to update the documentation.
Based on user feedback it performs better than any other DreamGen model.

Sign up or log in to comment