Speeds up SYCL SSM_CONV by up to 1.85x
Llama.cpp increases SYCL SSM_CONV performance by up to 1.85x via optimized window loads.
sycl: coalesce the ssm_conv window loads (#26612) test-backend-ops perf -o SSM_CONV on an Arc Pro B70, interleaved A/B against master, 6 reps, us/run: ne_a=[515,3328,1,1] ne_b=[4,3328,1,1] n_t=512 97.68 -> 52.95 1.85x ne_a=[937,8192,1,1] ne_b=[4,8192,1,1] n_t=934 516.16 -> 276.13 1.87x ne_a=[4,3328,1,1] ne_b=[4,3328,1,1] n_t=1 2.73 -> 2.71 flat llama-bench on qwen35 27B Q4_K - Medium (48 of its 64 blocks run ssm_conv), -ngl 99 -fa 1 -ctk f16 -ctv f16, interleaved passes of r=3: -b 2048 -ub 2048 pp2048 1045.1 / 1043.5 / 1043.7 -> 1069.5 / 1066.3 / 1065.9 +2.2% -b 2048 -ub 512 pp2048 771.8 / 772.7 -> 785.5 / 786.6 +1.8% -b 2048 -ub 512 tg128 23.81 / 23.88 -> 23.87 / 23.86 flat Website: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (ROCm 7.2) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Android: Android arm64 (CPU) Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.3 DLLs…
- github.comllama.cpp b10322primary