shipfeedAI news, curated daily

01:22:09 CET
21 MAY01:22:09shipfeed
pull to refreshlast sync
Just in — 30 new
§ tools · storyline

vLLM v0.16.0

vLLM v0.16.0 releases with async scheduling and pipeline parallelism support, a WebSocket-based realtime API, RLHF workflow improvements, and expanded model and XPU platform support across 440 commits.

Feb 25 · · primary fetch1 sourceupdated Feb 25 ·

vLLM v0.16.0 Please note that this release was branch cut on Feb 8, so any features added to vLLM after that date is not included. Highlights This release features 440 commits from 203 contributors (7 new)! Async scheduling + Pipeline Parallelism is now fully supported, delivering 30.8% E2E throughput improvement and 31.8% TPOT improvement (#32618). Realtime API: A new WebSocket-based Realtime API enables streaming audio interactions (#33187), building on the Voxtral realtime infrastructure. RLHF workflow improvements: Native NCCL-based weight syncing API (#31943), layerwise weight reloading for QeRL (#32133), and engine pause/resume with request preservation (#32351).

Unified Parallel Drafting for speculative decoding (#32887), plus spec decode now works with structured outputs (#33374) and penalty application in Model Runner V2 (#33251). Major XPU platform overhaul: Deprecated IPEX in favor of vllm-xpu-kernels (#33379), adding MoE (#33659), MXFP4 MoE (#33679), WNA16 (#33973), scaled_mm (#34117), and FP8 MoE (#34202) support. Model Support New architectures: GLM-OCR with MTP (#33005), Qwen3-ASR (#33312), DeepSeek-OCR-2 (#33165), Intern-S1-Pro (#33636), MiniCPM-o 4.5 (#33431)…

read full article on github.com
§ sources1 publication · timeline below
  1. github.comvllm v0.16.0primary