▶ llama.cpp·
llama.cpp b10415
spec : auto-detect mtp draft model type (#27005) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu…
items50 latest
▶ llama.cpp·
llama.cpp b10410
sycl: remove separate fp32 type promotion in gemm non-oneDNN path (#26372) sycl: use automatic fp16 promotion in gemm sycl: remove redundant comment Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
▶ llama.cpp·
llama.cpp b10408
sycl : Add DMMV ESIMD Q3_K kernel (#26251) Add DMMV Q4_K and Q6_K ESIMD kernels Configure cmake build with -DGGML_SYCL_ESIMD=ON to enable. Signed-off-by: Todd Malsbary Refactor ESIMD kernels to share common code…
▶ llama.cpp·
llama.cpp b10405
ggml-hip : remove -funsafe-math-optimizations (#26696) It enables -fassociative-math, which reassociates FP reductions and can flip greedy argmax on RDNA3.5 (e.g. MTP speculative decode diverging from the…
▶ llama.cpp·
llama.cpp b10400
ggml : fix arm builds, unused var (#26991) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
▶ llama.cpp·
llama.cpp b10375
chat : tighten bare function parsing for Qwen models (#26793) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
▶ llama.cpp·
llama.cpp b10373
imatrix.cpp: Move finite check and only check touched experts (#26861) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux…
▶ llama.cpp·
llama.cpp b10369
mtmd: support pocket-tts (#26871) adapt the api text model ok working impl, need verify and clean up mtmd: build the pocket-tts transposed convolutions as GEMM + col2im ggml_conv_transpose_1d has no grouped mode, so…
▶ llama.cpp·
llama.cpp b10362
tests : disable backend sampler hip multi output (#26878) test-backend-sampler: skip multi_output_sampling_chain on HIP The new multi_output_sampling_chain test uses top_k, whose backend probs path needs CUB…
▶ llama.cpp·
llama.cpp b10361
model : fix SWA not being enabled for EXAONE 4.5 (#26848) model : fix SWA not being enabled for EXAONE 4.5 load_arch_hparams tests `hparams.n_layer() == 64` before LLM_KV_NEXTN_PREDICT_LAYERS has been read. n_layer()…
▶ llama.cpp·
llama.cpp b10360
common/peg : suppress incomplete escape sequences (#26780) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
▶ llama.cpp·
llama.cpp b10359
ggml-webgpu: fix CI errors from #25025 and #25262 (#26566) test new flash_attn test rebase and fix to disable subgrou matrices when max_kv_tile == 0 delete log output Add i32 support to cpy and enables the all ops test…
▶ llama.cpp·
llama.cpp b10358
Address review comment of PR 25532 (#26852) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
▶ llama.cpp·
llama.cpp b10357
opencl: transpose the K tile in local memory for FA prefill kernels (#26428) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
▶ llama.cpp·
llama.cpp b10356
ci : target ROCm 7.14 for build and release (#25775) Switch ROCm from 7.2.1 to 7.14 ROCm 7.14 is the first production release using TheRock build system. It can be installed using multi-arch deliverables from wheels…
▶ llama.cpp·
llama.cpp b10355
llama : support multi-output backend sampling (#25532) Enable backend sampling with token speculation Clamp the mask sum before converting it into the sampled index Add a numeric context parameter declaring the maximum…
▶ llama.cpp·
llama.cpp b10354
ggml-cpu : fix CPU affinity mask being ignored on Android (#26838) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
▶ llama.cpp·
llama.cpp b10353
ggml : require contiguous src for ROLL on CUDA and Metal (#25928) ggml_roll only asserts nb[0] == ggml_type_size, so a permuted src is a valid input, but the CUDA and Metal roll kernels index by ne alone and never read…
▶ llama.cpp·
llama.cpp b10344
model: add MTP support for Nemotron model (#26725) model: add MTP support for Nemotron Nano model model: add mtp_flags for nemotron model address review comments Website: macOS/iOS: macOS Apple Silicon (arm64) macOS…
▶ llama.cpp·
llama.cpp b10343
vendor : update cpp-httplib to 0.53.0 (#26821) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu…
▶ llama.cpp·
llama.cpp b10342
model : Granite-Switch Architecture (#25107) granite-switch: add llama.cpp backend (POC, CPU) New "granite-switch" architecture: a dense, all-attention Granite-4.1 model with N embedded LoRA adapters selected per-token…
▶ llama.cpp·
llama.cpp b10338
model-saver : fix expert shared/chunk FFN length key clobber (#26693) The saver called add_kv with LLM_KV_EXPERT_SHARED_FEED_FORWARD_LENGTH twice, the second time passing n_ff_chexp. gguf_set_val_u32…
▶ llama.cpp·
llama.cpp b10336
ggml-webgpu : refactor several wgsl files and simplify flash_attn wgsl. (#26134) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
▶ llama.cpp·
llama.cpp b10333
ggml-cpu : fix missing Q5_0 dispatch in SpaceMiT backend (#26792) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
▶ llama.cpp·
llama.cpp b10332
ci: rm `GGML_HIP_ROCWMMA_FATTN` (#26760) Signed-off-by: Aaron Teo Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
▶ llama.cpp·
llama.cpp b10331
server: report the isolate working directory from get_info (#26773) server: report the isolate working directory from get_info Without an explicit cwd, get_info fell back to the server process working directory even…
▶ llama.cpp·
llama.cpp b10330
CUDA: fuse rms_norm + mul + rope (+ view + set_rows) (#26767) CUDA: fuse rms_norm + mul + rope (+ view + set_rows) tests: add broadcast weight case to rms_norm_mul_rope CUDA: check memory ranges before rms_norm rope…
▶ llama.cpp·
llama.cpp b10329
server, ui: only offer a working directory when a tool reads it (#26762) The working directory chip showed up as soon as the server exposed any builtin tool, so a server started with just get_datetime, or a user who…
▶ llama.cpp·
llama.cpp b10328
server: add initial tool isolation support (via docker) (#26507) server: add initial tool isolation support (via docker) add docs adapt get_info py: fix type check cont separate tools_io_sandbox / tools_io_docker…
▶ llama.cpp·
llama.cpp b10327
CUDA: fix thread/block count in quantized cpy kernel launches (#26731) CUDA: fix thread/block count in quantized cpy kernel launches tests: add uneven block count cpy case Website: macOS/iOS: macOS Apple Silicon…
▶ llama.cpp·
llama.cpp b10326
tts: account for the vocoder pass in the timings line (#26733) get_output runs the waveform work the pipeline defers to it, from a single trailing window to a full pass depending on the model. Measuring it keeps the…
▶ llama.cpp·
llama.cpp b10322
sycl: coalesce the ssm_conv window loads (#26612) test-backend-ops perf -o SSM_CONV on an Arc Pro B70, interleaved A/B against master, 6 reps, us/run: ne_a=[515,3328,1,1] ne_b=[4,3328,1,1] n_t=512 97.68 -> 52.95 1.85x…
▶ llama.cpp·
llama.cpp b10321
metal : fix NORM/RMS_NORM for row lengths that leave a partial simdgroup (#26708) ggml_metal_op_norm sized the threadgroup with `nth = std::min(nth, args.ne00_t)`, which can leave nth not a multiple of the simdgroup…
▶ llama.cpp·
llama.cpp b10319
mtmd: fix longest_edge ignoring min/max pixels (#26638) mtmd: fix longest_edge ignoring min/max pixels nits Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
▶ llama.cpp·
llama.cpp b10318
sync : ggml Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
▶ llama.cpp·
llama.cpp b10313
server: (router) add LRU scheduler (#26572) add lru_sched handle coalescing (req leaves waiting queue) add tests fix stream case address review comments Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
▶ llama.cpp·
llama.cpp b10312
server: (router) do not evict busy models (#26567) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu…
▶ llama.cpp·
llama.cpp b10311
mtmd: stop feeding the text stream again during Qwen3-TTS generation (#26706) The reference implementation has two mutually exclusive prompt layouts. In non streaming mode the prefill carries the whole utterance text…
▶ llama.cpp·
llama.cpp b10310
ggml : add aarch64 HWCAP fallbacks and fix fp16 variant detection (#25554) ggml : add fallback definitions for missing aarch64 HWCAP bits ggml : require HWCAP_ASIMDHP for the aarch64 fp16 cpu variants Also rename…
▶ llama.cpp·
llama.cpp b10308
Mitigate crashing issue on Windows MSYS2 UCRT64 environment (GCC 16.1.0) (#26555) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
▶ llama.cpp·
llama.cpp b10307
sycl: fix UE4M3 parsing (#25608) The NVFP4 quantization format stores a scaling factor for every group of 16 weights, packed into a single UE4M3 byte. The SYCL GPU code was converting these scale values using the E4M3…
▶ llama.cpp·
llama.cpp b10306
sycl: *glu flat path (#26354) tests: add SWIGLU perf cases perf mode had no GLU coverage. Adds SWIGLU at 17408 columns, 512 and 2048 tokens, f16 and f32, with the operands both fused and split. sycl: consolidate…
▶ llama.cpp·
llama.cpp b10305
sycl : Support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PRE (#26568) support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PREwq update ops.md fix format issue Website: macOS/iOS…
▶ llama.cpp·
llama.cpp b10303
sycl : fix error Error OP FLASH_ATTN_EXT on arc770 (#26441) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
▶ llama.cpp·
llama.cpp b10301
cuda: fix warnings for unused variable/function (#26688) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
▶ llama.cpp·
llama.cpp b10299
metal : avoid `threadgroup` matrix array instantiation in kernel_lightning_indexer (#26646) In MSL, declaring an array of matrix types like `threadgroup half4x4` causes a 'no matching constructor' compilation error…
▶ llama.cpp·
llama.cpp b10298
mtmd: add chunk save/load function (#26645) mtmd: add chunk save/load function nits add tests rn _MAX --> _COUNT Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
▶ llama.cpp·
llama.cpp b10297
server: fix empty response for /cors-proxy (#26656) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
▶ llama.cpp·
llama.cpp b10295
model-loader : fix quantized reshaped tensor strides (#26672) Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
▶ llama.cpp·
llama.cpp b10293
ci : onboard AMD ROCm CI with gfx1151 fixes (#26544) ci: prepare for amd rocm ci Signed-off-by: Aaron Teo ci: fix editorconfig-checker Signed-off-by: Aaron Teo ci: fix device not recognised Signed-off-by: Aaron Teo ci…