Baidu releases ERNIE 5.5 Flash Base weights, a 280B MoE trained on 33.5T tokens

Baidu released ERNIE 5.5 Flash Base

Baidu has published ERNIE-5.5-Flash-Base, a sparse Mixture-of-Experts model with 280B total parameters and 14B activated per token, pre-trained from scratch on 33.5T tokens with a 128K training context and support for 1M-token fine-tuning and inference. The checkpoint is the raw end-of-pre-training weights, with no mid- or post-training, meant as a starting point for domain adaptation, instruction tuning, and reinforcement-learning research. Efficiency drove the design: hybrid sparse attention, latent MoE experts, and a MultiMax output head all target cheaper long-context training and inference.