shipfeedAI news, curated daily

23:04:38 CET
20 MAY23:04:38shipfeed
pull to refreshlast sync
Just in — 30 new
§ tools · storyline

vLLM v0.19.0

vLLM releases v0.19.0 with Gemma 4 support, zero-bubble async speculative decoding, CPU KV cache offloading, and NVIDIA B300/GB300 compatibility across 448 commits from 197 contributors.

Apr 3 · · primary fetch1 sourceupdated Apr 3 ·

vLLM v0.19.0 Highlights This release features 448 commits from 197 contributors (54 new)! Gemma 4 support: Full Google Gemma 4 architecture support including MoE, multimodal, reasoning, and tool-use capabilities (#38826, #38847). Requires `transformers>=5.5.0`. We recommend using pre-built docker image `vllm/vllm-openai:gemma4` for out of box usage. Zero-bubble async scheduling + speculative decoding: Async scheduling now supports speculative decoding with zero-bubble overlap, significantly improving throughput (#32951). Model Runner V2 maturation: MRV2 gains piecewise CUDA graphs for pipeline parallelism (#35162), spec decode rejection sampler with greedy/logprobs support (#37238, #37237), multi-modal embeddings for spec decode (#36097), streaming inputs (#37028), and EPLB support (#37488).

ViT Full CUDA Graphs: Vision encoders (ViT) now support full CUDA graph capture for reduced overhead (#35963). General CPU KV cache offloading: A simple yet general CPU KV cache offloading mechanism for V1, with pluggable cache policy and block-level preemption handling (#37160, #37874, #34805, #36642, #37853). DBO (Dual-Batch Overlap) generalization: The microbatch optimization (DBO) now…

read full article on github.com
§ sources1 publication · timeline below
  1. github.comvllm v0.19.0primary