Speeds up CUDA FWHT by accepting F16 input directly
CUDA adds direct F16 input support to FWHT, removing type conversion overhead.
cuda: add F16 input to the FWHT (#29096) cuda: add F16 input to the FWHT The CUDA FWHT accepts F32 input only. This makes the source type a template parameter, so the kernel reads an F16 source directly instead of requiring a converted copy. The F32 path is unchanged. supports_op accepts an F16 src1 against an F32 src0 for the Hadamard hint. Every other F16 src1 against a non-F16 src0 is still refused. ggml_cuda_op_mul_mat_use_fwht is the single predicate both supports_op and the dispatch call now share, checking contiguity and same-shape(src1, dst) in addition to the type/hint conditions above.
Without a shared predicate, supports_op could admit an op that ggml_cuda_op_fwht then rejects only after the unconditional same-shape assert has already fired; that gap predates this change (it applies to the existing F32 path too) but this PR is what touches supports_op, so it closes it here. test-backend-ops on an A10 (lambdalabs): MUL_MAT 1297/1297, including all 24 Hadamard cases (18 existing F32, 6 new F16). cuda: use ggml_cuda_cast in the FWHT load, drop the comment Website: Attestations: macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
- github.comllama.cpp b11203primary