Fixes decreasing pos in llama-batch sequences
ggml : process data in smaller chunks in CUDA ggml_top_k() and ggml_argsort() to reduce temporary buffers memory usage (#24776) ggml : process data in smaller chunks in CUDA ggml_top_k() implementation to reduce temporary buffers memory usage ggml : allocate tmp_dst only only once before the loop chore : whitespaces Co-authored-by: Georgi Gerganov ggml : use chunked processing in both CUDA CUB top-k and argsort implementations chore : separate argsort_f32_i32_cuda_bitonic() call from return statement Co-authored-by: Johannes Gäßler chore : replace ternary operators with min/max --------- Co-authored-by: Stanisław Szymczyk Co-authored-by: Georgi Gerganov Co-authored-by: Johannes Gäßler macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (ROCm 7.2) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Android: Android arm64 (CPU) Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.3…
- github.comllama.cpp b9948primary
- github.comllama.cpp b9949
- github.comllama.cpp b9930
§ how this story moved
- primary — llama.cpp — Releases publishes the launch post.
- llama.cpp — Releases picks up coverage.
- llama.cpp — Releases picks up coverage.