shipfeedAI news, curated daily

02:53:25 CET
12 OCT02:53:25shipfeed⋯
pull to refreshlast sync
Just in — 30 new
§ local-llm · storyline

Adds llama_batch_ext API for mixed token/embedding inputs

llama.cpp v0.6.0 releases the llama_batch_ext API for mixed token/embedding inputs, adds GLM-5.3-Flash and Clef model support, and ships Metal flash attention and Vulkan sparse attention improvements.

Oct 5 · · primary fetch1 sourceupdated Oct 5 ·

Overview llama.cpp v0.6.0 introduces the new `llama_batch_ext` extended batch API (with `llama_process`) for mixed token/embedding inputs and MTP/deepstack state embeddings, adds support for the GLM-5.3-Flash (GLM5-Next) 320B hybrid model, the Clef decision model (text and vision) and MTP speculative decoding for Qwen4Exp, ships a new `/v1/systemone` server API for decision models (laya, julia-1, lev, openjev, kev, nimble), overhauls the Web UI with a Hugging Face Hub data layer and model download pipeline, adds a Metal tensor-API flash attention kernel for F16 KV, sparse flash attention for quantized K/V on Vulkan, and updates ggml to v0.26.0.

Highlights New `llama_batch_ext` extended batch API with `llama_process()`, supporting mixed token/embedding batches and per-token "state" embeddings for MTP and deepstack models #24669 New models: GLM-5.3-Flash (GLM5-Next), a 320B text+vision hybrid model #27773, and the Clef decision model, fully supported with both text and vision #29831 #29969 Qwen4Exp: high-quality support is now available, with MTP speculative decoding (~1.5x decode speedup on DGX Spark) and various correctness fixes #29761 #29751 llama and server: new…

read full article on github.com ↗
§ sources1 publication · timeline below
  1. github.comllama.cpp v0.6.0primary