shipfeedAI news, curated daily

16:08:19 CET
26 AUG16:08:19shipfeed
pull to refreshlast sync
Just in — 30 new
§ tools · storyline

Speeds up DSpark TTFT ~60% with adaptive token budget

vLLM v0.28.0 releases with 584 commits, delivering ~60% faster DSpark TTFT via an adaptive speculative token budget and broad Kimi-K3 and DeepSeek V4 optimisations.

today · · primary fetch1 sourceupdated today ·

v0.28.0 Highlights This release features 584 commits from 270 contributors (76 new)! Kimi-K3 performance push: a major optimization effort for Kimi-K3 across the stack — Decode Context Parallel (DCP) support (#50484), fused FlashKDA decode and prefill kernels (#50654, #51311, #52458), SiTU activation support for MegaMoE (#50510), GEMM-RS for sequence parallelism (#52079), combined all-gathers with 1.5~3x kernel-level speedup (#51070), an adaptive speculative token budget delivering ~60% better DSpark TTFT (#51725), and optional shared-expert sharding saving ~17 GiB of memory per GPU (#50912).

Kimi-K3 also now runs on ROCm with the V2 model runner (#51653). DeepSeek V4: sparse MLA now works end-to-end for plain decode, MTP, and DSpark speculative decoding (#51538), joined by AMD Quark NVFP4 support (#47972), reasoning-effort prompts and mappings (#50580), sparse top-k metadata kernel optimizations (#52084, #51967), narrowed eager CUDA graph regions (#51430, #52401), and ROCm enablement on gfx11 and gfx950 (#47017, #52212). Speculative decoding advances: DFlash2 with local convolution and a candidate selector (#52816), DSpark confidence-scheduled verification (#47808), and async…

read full article on github.com
§ sources1 publication · timeline below
  1. github.comvLLM v0.28.0primary