Check new, much better, version of this model
This model has issues (trained without BOS token), please use the following preview models instead:
But no quants :|
Yea, I'm waiting on quants as well. I can just BARELY not run the full model on my VRAM haha.
I just spotted GGUF for one of them:
https://huggingface.co/localfultonextractor/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5-Q8_0-GGUF
Well, I was watching this drama and wanted to wait till a more "final" version appears. But I've put both in the queue and a full set of static quants should be available in a few hours.
Spoke too soon:
NotImplementedError: Unknown rope scaling type: dynamic
the models are not supported by llama.cpp at the moment it seems. Not without disabling rope scaling at least.
You can remove that from the config and use llama.cpp's own rope scaling.
Though I am surprised it throws an error like this.
llama.cpp can throw a lot of interesting errors, even with old models that did convert fine at the time :)
https://huggingface.co/mradermacher/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5-GGUF
BTW, anybody can request quants from me at https://huggingface.co/mradermacher/model_requests in cases I overlooked it. Can save the model creators a lot of time, too :)
Awesome, thank you @mradermacher !
@DreamGenX , thanks for you work on these Llama3-8B finetunes. Any plans for a 70B finetune?
@dobs
I finished a train of L3 70B DreamGen model few weeks ago. I changed the template to take full advantage of the built-in tokens, so first I need to update the documentation.
Based on user feedback it performs better than any other DreamGen model.