shipfeedAI news, curated daily

01:00:17 CET
28 SEPT01:00:17shipfeed⋯
pull to refreshlast sync
Just in — 30 new
§ local-llm · storyline

Extends MOE fusion to speculative decoding

CUDA extends MOE fusion to speculative decoding, enabling multi-token support previously limited to single tokens.

Aug 31 · · primary fetch1 sourceupdated Aug 31 ·

CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token (#27621) CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were resticted to 1 token Signed-off-by: ynankani Address review comments Signed-off-by: ynankani Add SWIGLU_CLAMP case to multi-token moe fusion Signed-off-by: ynankani --------- Signed-off-by: ynankani Website: Attestations: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (ROCm 7.14) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Android: Android arm64 (CPU) Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.3 DLLs Windows arm64 (CUDA 13) (preview) - CUDA 13.4 DLLs Windows x64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (ROCm 7.14) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL…

read full article on github.com ↗
§ sources2 publications · timeline below
  1. github.comllama.cpp b10718primary
  2. github.comllama.cpp b10704

§ how this story moved

  1. primary — llama.cpp — Releases publishes the launch post.
  2. llama.cpp — Releases picks up coverage.