Adds llama_batch_ext API for mixed token/embedding inputs
llama.cpp v0.6.0 releases the llama_batch_ext API for mixed token/embedding inputs, adds GLM-5.3-Flash and Clef model support, and ships Metal flash attention and Vulkan sparse attention improvements.
Overview llama.cpp v0.6.0 introduces the new `llama_batch_ext` extended batch API (with `llama_process`) for mixed token/embedding inputs and MTP/deepstack state embeddings, adds support for the GLM-5.3-Flash (GLM5-Next) 320B hybrid model, the Clef decision model (text and vision) and MTP speculative decoding for Qwen4Exp, ships a new `/v1/systemone` server API for decision models (laya, julia-1, lev, openjev, kev, nimble), overhauls the Web UI with a Hugging Face Hub data layer and model download pipeline, adds a Metal tensor-API flash attention kernel for F16 KV, sparse flash attention for quantized K/V on Vulkan, and updates ggml to v0.26.0.
Highlights New `llama_batch_ext` extended batch API with `llama_process()`, supporting mixed token/embedding batches and per-token "state" embeddings for MTP and deepstack models #24669 New models: GLM-5.3-Flash (GLM5-Next), a 320B text+vision hybrid model #27773, and the Clef decision model, fully supported with both text and vision #29831 #29969 Qwen4Exp: high-quality support is now available, with MTP speculative decoding (~1.5x decode speedup on DGX Spark) and various correctness fixes #29761 #29751 llama and server: new…
- github.comllama.cpp v0.6.0primary