gabrielmbmb
/

Upcycled-Qwen1.5-MoE2.7B-LoRA-merged

Text Generation

Inference Endpoints

Model card Files Files and versions Community

No model card

New: Create and edit this model card directly on the website!

Contribute a Model Card

Downloads last month: 6

Safetensors

Model size

14B params

Tensor type

BF16

·

Inference Examples

Text Generation

Inference API (serverless) is not available, repository is disabled.

Collection including gabrielmbmb/Upcycled-Qwen1.5-MoE2.7B-LoRA-merged

Upcycling Experiments

Models I pre-trained initialising SMoE models using dense model weights and the upcycling process used for Qwen1.5-MoE2.7BA (or something similar) • 6 items • Updated Apr 1