▶ llama.cpp·
llama.cpp b9999
kleidiai : add SME2 f32 kernel (#24414) kleidiai : add SME2 f32 kernel enable dynamic scheduling for SME2 f32 kernel macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
items50 latest
▶ llama.cpp·
llama.cpp b9994
metal : add Q2_0 support (#25419) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
▶ llama.cpp·
llama.cpp b9993
model: add Hy3 (hy_v3) support with MTP speculative decoding (#25395) model: add Hy3 (hy_v3) architecture support Adds Tencent Hunyuan 3 (HF architecture HYV3ForCausalLM, GGUF arch hy_v3): a MoE decoder stack with…
▶ llama.cpp·
llama.cpp b9992
CUDA: refactor MMQ kernel configuration (#24127) CUDA: refactor MMQ kernel configuration fix Blackwell config remove legacy code macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
▶ llama.cpp·
llama.cpp b9990
spec: add Minimax2 eagle3 support Fix nullptr in minimax2 EAGLE3 minor : add newline --------- Co-authored-by: Georgi Gerganov macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
▶ llama.cpp·
llama.cpp b9988
tests: Harmonize header use (#25616) tests: Harmonize the use of private ggml includes tests: In test-backend-ops, use quoted includes As with all other tests. This is to ensure that the build uses shipped headers over…
▶ llama.cpp·
llama.cpp b9987
gguf : add tensor shape accessor (#24405) gguf : add tensor shape accessors gguf : return tensor shape as const int64_t gguf : remove n_dims accessor, keep only gguf_get_tensor_ne macOS/iOS: macOS Apple Silicon (arm64)…
▶ llama.cpp·
llama.cpp b9986
chat : fix reasoning leak with force-opened bare templates (#24674) chat : fix reasoning leak with force-opened bare templates The reasoning start tag inferred from prior turns can carry trailing whitespace (e.g. \n)…
▶ llama.cpp·
llama.cpp b9985
sycl: add fused top-k MoE (#25217) sycl: add fused top-k MoE sycl: address review: GGML_SYCL_ENABLE_FUSION env, move fusion dispatch to topk-moe sycl: print GGML_SYCL_ENABLE_FUSION at startup like other env vars…
▶ llama.cpp·
llama.cpp b9984
sycl: add Q2_K to DMMV reorder path (#25064) Signed-off-by: Todd Malsbary macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
▶ llama.cpp·
llama.cpp b9982
server: honour per-request reasoning_budget_tokens in chat completions (#23116) server: honour per-request reasoning_budget_tokens in chat completions The reasoning-budget block in oaicompat_chat_params_parse read only…
▶ llama.cpp·
llama.cpp b9981
vendor : update cpp-httplib to 0.50.1 (#25576) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
▶ llama.cpp·
llama.cpp b9980
server: Don't consider models with --no-mmproj-auto as multimodal (#25590) If mmproj is explicitly disabled via the model preset or command-line parameters then the model won't be able to handle image/audio inputs and…
▶ llama.cpp·
llama.cpp b9979
mtmd: fix silent prompt truncation on embedded NUL (#25548) mtmd: fix silent prompt truncation on embedded NUL mtmd_input_text carried the prompt as a bare const char* with no length, so a NUL byte in message content…
▶ llama.cpp·
llama.cpp b9978
server : evict checkpoints within min-step of each other (#25472) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
▶ llama.cpp·
llama.cpp b9977
server : fix image blocks in tool_result being dropped during Anthropic OpenAI conversion (#22536) server : fix image blocks in tool_result being dropped during Anthropic→OpenAI conversion…
▶ llama.cpp·
llama.cpp b9976
Fix conditional to display 'LLAMA_SPLIT_MODE_TENSOR not implemented for architecture' message (#24926) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
▶ llama.cpp·
llama.cpp b9975
gguf : reject empty metadata keys (#24917) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
▶ llama.cpp·
llama.cpp b9974
cuda: Don't crash when querying memory on device with no free memory. (#25157) If a Cuda device has no or limited available memory, the actual call to cudaMemGetInfo() itself can cause a fatal crash due to a cuda out…
▶ llama.cpp·
llama.cpp b9973
DeepseekV4: clear cache only for seq rather than full (#25521) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
▶ llama.cpp·
llama.cpp b9972
server: allow stream for exec_shell_command (#25526) init stream add stream for shell tool add test nits update docs macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
▶ llama.cpp·
llama.cpp b9971
server: refactor server_stream (#25541) server: refactoring, remove spipe from server_http_res wip remove non-thread-safe rd.stop() call move server_res_spipe nits improve server_stream_create_spipe server-stream…
▶ llama.cpp·
llama.cpp b9969
Vulkan: route large matmuls to medium tile on Adreno (#24877) [Vulkan] Fixes llama-cli breaking over longer promts sizes The llama-cli was breaking for longer promts sizes for q4_0 quantized networks. Causing due to…
▶ llama.cpp·
llama.cpp b9968
opencl: add int8 dp4 dense and MoE prefill optimization for Adreno GPUs (#25537) opencl: add int8 dp4 dense and moe GEMM opencl: refactor --------- Co-authored-by: Li He macOS/iOS: macOS Apple Silicon (arm64) macOS…
▶ llama.cpp·
llama.cpp b9967
server: accept null sampling params (#25538) server: accept null sampling params Extend the schema validation to treat a null value as absent, so clients can send null on nullable params (temperature, top_p, ...) to…
▶ llama.cpp·
llama.cpp b9966
llama : make tensor-split regex patterns static (#24710) llama_meta_device_get_split_state() recompiled 29 std::regex on every call. In -sm tensor mode the callback runs once per tensor per token, so this dominated the…
▶ llama.cpp·
llama.cpp b9965
hexagon: improve ARGSORT performance for small tensors (#25512) hex-sort: add efficient bitomic sort in hvx regs up to 1024 elements hex-sort: fix inverted vrors hex-sort: specialize sort functions for the common cases…
▶ llama.cpp·
llama.cpp b9964
arg: prevent duplicate spec model downloads (#25527) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
▶ llama.cpp·
llama.cpp b9963
mtmd: deepseek-ocr v1 multi-tile (#24717) mtmd: deepseek-ocr v1 multi-tile dynamic resolution + unified image-preprocessors for both versions (ds-ocr v1 and v2) remove hacky API fuse row into a long image almost…
▶ llama.cpp·
llama.cpp b9960
server: remove loading.html (#25500) server: remove loading.html apply ui changes macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux…
▶ llama.cpp·
llama.cpp b9959
sync : ggml macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
▶ llama.cpp·
llama.cpp b9957
server: improve tools, remove apply_diff (#25498) server: improve tools, remove apply_diff improve edit tool add tools_io abstraction add tools_io_basic fix build move utils to class member add const macOS/iOS: macOS…
▶ llama.cpp·
llama.cpp b9956
cli: fix crash on wrong server base url (#25497) llama-cli: fix crash on wrong server base url by catching exceptions and graceful exit review: leaner catch group: json error and standard exception macOS/iOS: macOS…
▶ llama.cpp·
llama.cpp b9952
llama : make all KQ masks f16 if FA is used, remove zero attention bias, remove raw_k repeats in DeepSeek V4 (#25370) llama : make all KQ masks (except the lightning indexer one) f16 if FA is used and remove zero…
▶ llama.cpp·
llama.cpp b9951
ggml-et: Initial ET backend (#24179) ggml-et: Add performance logging ggml-et: Quants helpers ggml-et: Add MUL_MAT kernel ggml-et: Add ROPE kernel ggml-et: Add RMS_NORM kernel ggml-et: Add GLU kernel ggml-et: Add…
▶ llama.cpp·
llama.cpp b9950
llama-batch: add unit test (#25471) llama-batch: add unit test fix win32 builds add not implemented assertion in unused methods remove unreachable code macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64…
▶ llama.cpp·
llama.cpp b9949
opencl: cluster-parallel decode FA for Adreno (#25473) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu…
▶ llama.cpp·
llama.cpp b9948
ggml : process data in smaller chunks in CUDA ggml_top_k() and ggml_argsort() to reduce temporary buffers memory usage (#24776) ggml : process data in smaller chunks in CUDA ggml_top_k() implementation to reduce…
▶ llama.cpp·
llama.cpp b9947
cli: add --output option (#25484) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
▶ llama.cpp·
llama.cpp b9946
hexagon: tiling, tracing and optimizations for unary ops (#25474) hexagon: tile wide rows in pointwise unary ops to avoid VTCM overflow unary: reject permuted tensors for now (not used by models) hex-unary: replace…
▶ llama.cpp·
llama.cpp b9945
server : move chat-template thinking probe inside the init try/catch (#24093) A model whose chat template parses at init but fails parser generation at apply time (e.g. uses {% call %}) throws std::invalid_argument…
▶ llama.cpp·
llama.cpp b9941
Only index by compile times + always multiply/add (#25445) The first one avoids relying on compile to optimize local memory away, and the second is cheaper than issuing control flow statements macOS/iOS: macOS Apple…
▶ llama.cpp·
llama.cpp b9940
llama-bench : init params.offline (#25476) Signed-off-by: Adrien Gallouët macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
▶ llama.cpp·
llama.cpp b9939
metal : add CONV_2D_DW (depthwise convolution) support (#21565) metal : add CONV_2D_DW (depthwise 2D convolution) support test : add perf cases for CONV_2D_DW metal : use 3D dispatch for CONV_2D_DW kernel metal : add…
▶ llama.cpp·
llama.cpp b9938
ggml-hip: enable -funsafe-math-optimizations (#24668) CUDA is compiled with fast math and AMD/HIP is not — this flag lets AMD use fast math too. We can't use -ffast-math: it implies -ffinite-math-only, which won't…
▶ llama.cpp·
llama.cpp b9937
cuda: align snake fusion matcher with the other backends (#25460) cuda: fix snake fusion type predicate, a and inv_b are F32 The matcher required a->type == x->type while launch_snake reads both as const float *…
▶ llama.cpp·
llama.cpp b9936
server : respect min-step when splitting prompt batches (#25420) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
▶ llama.cpp·
llama.cpp b9935
hexagon: add VISION RoPE support (#25216) hexagon: add VISION RoPE support hexagon: support RoPE on strided half-dim views for all modes hex-rope: decouple src0 DMA copy size from row stride hex-rope: support…
▶ llama.cpp·
llama.cpp b9934
ggml-webgpu: tune subgroup split (d_split) in flash_attn_vec (#25418) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
▶ llama.cpp·
llama.cpp b9933
opencl: Q6_K GEMM/GEMV fix for ne01 of weights that are not multiples of 128. (#25464) opencl: fix garbled output for Q6_K weights with ne01 % 128 != 0 on Adreno Observed with granite-3.1-3b-a800m-instruct, whose vocab…