shipfeedAI news, curated daily

01:00:42 CET
28 SEPT01:00:42shipfeed⋯
pull to refreshlast sync
Just in — 30 new
§ local-llm · storyline

Adds NVIDIA Nemotron-3-Puzzle-75B-A9B support

llama.cpp adds NVIDIA Nemotron-3-Puzzle-75B-A9B support with variable per-layer expert configuration.

Sep 3 · · primary fetch1 sourceupdated Sep 3 ·

model: add NVIDIA Nemotron-3-Puzzle-75B-A9B (NemotronHPuzzle) support (#25444) hparams: add per-layer n_ff_exp/n_expert_used arrays with scalar-or-array loading G1/G2 infrastructure for variable-per-layer expert FFN size and top-k routing (required for Puzzle-75B which has 5 distinct n_ff_exp values and 7 top-k values across its 40 MoE layers). Design: rename scalar members to _impl suffix (following existing convention), add LLAMA_MAX_LAYERS arrays, add n_ff_exp(il)/n_expert_used(il) accessors with scalar fallback. No new GGUF keys: reuses existing expert_feed_forward_length and expert_used_count keys via get_key_or_arr (scalar -> broadcast, array -> per-layer).

llama-hparams.h: n_ff_exp -> n_ff_exp_impl, n_expert_used -> n_expert_used_impl; add n_ff_exp_arr / n_expert_used_arr arrays; add per-layer accessor declarations. llama-hparams.cpp: implement n_ff_exp(il) and n_expert_used(il); out-of-range il returns impl safely (shared code, no abort). llama-model.cpp: central n_expert_used load changed to get_key_or_arr; derive impl as max-of-array for validations and backward compat; zero both new arrays; HunyuanVL override also zeroes n_expert_used_arr. llama-graph.cpp: aggregation…

read full article on github.com ↗
§ sources1 publication · timeline below
  1. github.comllama.cpp b10776primary