shipfeedAI news, curated daily

15:51:47 CET
20 SEPT15:51:47shipfeed
pull to refreshlast sync
Just in — 30 new
§ local-llm · storyline

Adds F16 input to Metal FWHT kernel

Metal FWHT kernel adds native F16 input support, eliminating F32 conversion overhead.

today · · primary fetch1 sourceupdated today ·

metal: add F16 input to the FWHT (#29094) metal: add F16 input to the FWHT The Metal FWHT kernel accepts F32 input only. This change makes the source type a template parameter, so the kernel reads an F16 source directly instead of requiring a converted copy. The F32 instantiations are unchanged. The pipeline name now carries the source type, and supports_op accepts an F16 src1 for the Hadamard hint at the four sizes the kernels cover. Every other F16 src1 path still goes through ggml_metal_supports_mul_mat_op. These are the test cases mentioned in #27779. test-backend-ops on M5 Pro: MUL_MAT_HADAMARD 16/16, MUL_MAT 1265/1265.

metal: ask the same FWHT question in supports_op and the dispatch supports_op admitted an F16 src1 on the type, the hint and the width alone, but the dispatch also requires src1 and dst to be contiguous and the same shape. A Hadamard hinted MUL_MAT that passed the first and failed the second reached the generic path, which has no F32 src0 by F16 src1 kernel, and aborted on a nil pipeline: kernel not found in any metal library: base = 'kernel_mul_mv_f32_f16_4' ggml_metal_encoder_set_pipeline: nil Metal pipeline ggml_metal_use_fwht now holds the whole condition…

read full article on github.com
§ sources1 publication · timeline below
  1. github.comllama.cpp b11059primary