Adds lazy tensor reading
llama.cpp 0.4.0 adds lazy tensor reading, Qwen3.8-Flash-Next and Nemotron-3-Puzzle support, and performance optimizations.
Overview llama.cpp 0.4.0 adds initial Qwen3.8-Flash-Next and Nemotron-3-Puzzle support, on-demand tensor reading, per-slot server context limits, video input options, and a ggml update to 0.23.0 with major sparse flash attention and RDMA work. API changes Added `llama_lazy_mode` and `lazy_mode` (#27794). Added `max_buf_size` to quantize params (#27795). Bumped session/state versions for KV-cell token tracking (#27762). Added `mtmd_input_part` and `mtmd_tokenize_from_parts()` (#28250). Made `mtmd_tokenize` const-correct (#28310). Made `mtmd_bitmap_init_lazy` const-correct (#28307). Added mtmd helper video/init options (#24318).
Server multimodal helpers accept `mtmd_helper_init_opt` (#24318). New models Added initial Qwen3.8-Flash-Next (`qwen4exp`) architecture support; optimization improvements are still pending (#27742). Added NVIDIA Nemotron-3-Puzzle-75B-A9B support (#25444). Added DSpark support for Nemotron 3.5 (#27804). Added support for nanbeige4.2-3B (#27730). Core changes Added lazy tensor reading (#27794). Renamed option to `--lazy-mode` (#27969). Added `--n-cpu-ffn` (#26622). Added per-layer expert routing/FFN (#28323). Added KV-cell token tracking (#27762). Added…
- github.comllama.cpp v0.4.0primary