shipfeedAI news, curated daily

09:51:16 CET
28 JUL09:51:16shipfeed
pull to refreshlast sync
Just in — 30 new
§ local-llm · storyline

Fixes decreasing pos in llama-batch sequences

Jul 10 · · primary fetch1 sourceupdated Jul 10 ·

ggml : process data in smaller chunks in CUDA ggml_top_k() and ggml_argsort() to reduce temporary buffers memory usage (#24776) ggml : process data in smaller chunks in CUDA ggml_top_k() implementation to reduce temporary buffers memory usage ggml : allocate tmp_dst only only once before the loop chore : whitespaces Co-authored-by: Georgi Gerganov ggml : use chunked processing in both CUDA CUB top-k and argsort implementations chore : separate argsort_f32_i32_cuda_bitonic() call from return statement Co-authored-by: Johannes Gäßler chore : replace ternary operators with min/max --------- Co-authored-by: Stanisław Szymczyk Co-authored-by: Georgi Gerganov Co-authored-by: Johannes Gäßler macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (ROCm 7.2) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Android: Android arm64 (CPU) Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.3…

read full article on github.com
§ sources3 publications · timeline below
  1. github.comllama.cpp b9948primary
  2. github.comllama.cpp b9949
  3. github.comllama.cpp b9930

§ how this story moved

  1. primaryllama.cpp — Releases publishes the launch post.
  2. llama.cpp — Releases picks up coverage.
  3. llama.cpp — Releases picks up coverage.