Baidu's ERNIE 5.5 Flash Base: 280B open weights, 14B active

What Baidu actually released

ERNIE 5.5 Flash Base is not a chatbot. It is the raw checkpoint at the end of pretraining, before any instruction tuning or reinforcement learning.

Baidu's release is a sparse Mixture-of-Experts language model with 280 billion total parameters, of which 14 billion are activated per token. It was trained from scratch on 33.5 trillion tokens with a training context of up to 128K, and it supports a 1M-token context for fine-tuning and inference.

The key detail is what is missing: no mid-training, no post-training, no instruction tuning. Developers get the general capabilities that pretraining builds and nothing else. That makes it a starting point for domain adaptation, instruction tuning and RL research rather than a ready-to-use assistant.

Baidu frames the release as a way to spare developers the cost of large-scale pretraining. Instead of spending millions on a base model, teams can fine-tune ERNIE 5.5 Flash Base on their own data and put their resources into the tasks they actually care about.

ERNIE 5.5 Flash Base: parameters, context and architecture

The architecture choices that keep it affordable

Baidu says efficiency was a design goal from the outset, and the model's attention stack reflects that.

The model has 43 Transformer layers, a hidden size of 4096 and 64 attention heads. Most layers use Heavily Compressed Attention, which compresses every 128 tokens into a single cached entry. Only one layer in seven keeps token-level access to the full context using Multi-head Latent Attention. The result is that most layers still see the whole sequence while caching only a fraction of what a full-context layer needs.

Baidu also uses layer-specific head dimensions: cheap HCA layers get wide 512-dim heads, while expensive MLA layers use narrower 256-dim heads. The MLA layers use sparse attention (DSA) so their per-query cost stays constant as context grows. Low-Rank Gated Attention and Virtual Head Attention round out the changes, cutting gate projection weights by 75% and adding a lightweight mixing step across heads.

The company says it designed the model and the training system together, and that the corpus emphasizes capabilities that are hard to add later, such as repository-scale code and long-context understanding.

How it compares to other open-weight models

ERNIE 5.5 Flash Base enters a crowded field of open-weight models, but its combination of size and active parameters is unusual.

At 280B total parameters and 14B active, ERNIE 5.5 Flash Base sits in the same weight class as some of the largest open models, but its sparse activation keeps inference costs closer to a 14B model. That is the same trade-off that has made Mixture-of-Experts models popular for serving.

The 1M-token context for fine-tuning is notable. Most open-weight models top out at 128K or 256K, so developers working with long documents or codebases get more room without custom context-extension work.

The catch is that a base model requires real fine-tuning expertise to be useful. Teams expecting a drop-in chatbot will be disappointed. Those with the data and the compute to adapt it may find it a cheaper starting point than training from scratch.

Availability and pricing

Baidu has not announced pricing for hosted access to ERNIE 5.5 Flash Base. The weights are released openly, so developers in the US, UK and EU can download and fine-tune them subject to the license terms, which Baidu has not detailed in the release notes.

Running a 280B-parameter model locally requires substantial GPU memory, even with sparse activation. Most teams will use cloud instances or multi-GPU setups, and costs will vary by provider.

Baidu's ERNIE 5.5 Flash Base is a bet that open weights and efficient architecture can attract developers who want control over their models. It is not a consumer product, and it will not replace a fine-tuned assistant out of the box. But for teams with the data and the compute, it is a credible new starting point.