Release v5.17.0 New Model additions HYV4 Hy4-Preview is a 780B-parameter mixture-of-experts language model that activates 49B parameters per token. Each MoE layer holds 256 routed experts plus one always-active shared expert and routes every token to 8 of them. The context window is 1M tokens. The architecture combines four features: Multi-head Latent Attention (MLA) compresses keys and values into a low-rank latent (`kv_lora_rank`) that `kv_b_proj` expands back to one key/value per query head. DeepSeek Sparse Attention (DSA) selects `index_topk` keys per query with a lightweight indexer. Following IndexShare, only the layers marked `"full"` in `indexer_types` run an indexer; `"shared"` layers reuse the previous full layer's selection.

Gated MLA with learnable attention sinks, where each head owns a sink logit that participates in the softmax and contributes no value, as in GPT-OSS. Independent Hyper-Connections (iHC) replace the plain residual path with `hc_mult` parallel residual streams that are collapsed before, and redistributed after, every sublayer. The implementation does not execute the multi-token prediction (MTP) layers. Released checkpoints keep those weights so that…