shipfeedAI news, curated daily

10:48:42 CET
11 JUL10:48:42shipfeed
pull to refreshlast sync
Just in — 30 new
§ local-llm · storyline

Adds Mellum v2 support, zentorch-accelerated quantized inference

vLLM v0.22.1 adds support for JetBrains' Mellum v2 code-generation model, zentorch-accelerated quantized inference on AMD Zen CPUs, and fixes for multi-node Ray serving and DeepSeek-V4 initialization.

Jun 5 · · primary fetch1 sourceupdated Jun 5 ·

Highlights This release features 8 commits from 6 contributors (1 new)! v0.22.1 is a patch release on top of v0.22.0 with targeted bug fixes plus a couple of additions: new model support for JetBrains' Mellum v2, zentorch-accelerated quantized linear inference on AMD Zen CPUs, and fixes for multi-node Ray data-parallel serving, DeepSeek-V4 initialization, and a few model-loading regressions. Model Support New model: JetBrains' Mellum v2, an open-weights Mixture-of-Experts code-generation model (#43992). DeepSeek-V4: resolve a CUTLASS `fmin` compatibility issue that broke initialization (0decac0d).

Fix `OlmoHybridForCausalLM` failing to initialise after the checkpoint changed `rope_parameters` from `None` to `{"rope_type": None}` (#43846). Fix HyperCLOVAX loading after the upstream HuggingFace repo removed its remote code (now native in `transformers >= 5.9.0`): register the `hyperclovax` model_type so vLLM uses its vendored config instead of the stale `auto_map` (#43860). Hardware & Performance AMD Zen CPUs: route W8A8 (int8 dynamic-symmetric) and W4A16 (GPTQ) linear inference through zentorch kernels, registered ahead of the generic oneDNN CPU kernels, with transparent fallback…

read full article on github.com
§ sources1 publication · timeline below
  1. github.comvLLM v0.22.1primary