shipfeedAI news, curated daily

01:00:23 CET
16 JUL01:00:23shipfeed
pull to refreshlast sync
Just in — 30 new
§ local-llm · storyline

Prevents slow prefill from dropping healthy SSE connections

Server and UI now ping silent SSE streams every 1s and disconnect after 3s of inactivity to prevent slow prefill from dropping healthy connections.

Jul 3 · · primary fetch1 sourceupdated Jul 3 ·

server + ui: ping silent SSE streams every 1s and kick only after 3s so slow prefill never drops healthy connections (#25241) server + ui: ping silent SSE streams every 1s and kick only after 3s so slow prefill never drops healthy connections server + ui: sse_ping_interval becomes a per-request body field Address review from ngxson: the global default returns to 30 so API clients see no behavior change, and the WebUI sends sse_ping_interval: 1 in the request body since it owns the 3s visibility-kick contract and declares the cadence it needs. Positive values keep the existing > 0 gate, -1 keeps its disabled semantics.

server: move sse_ping_interval into the request schema Address review from ngxson: the field is now a typed field_num with hard limits (-1, INT32_MAX) bound to task_params, seeded from the CLI default alongside the other inherited parameters. The raw json_value read and its redundant comment are gone, and schema evaluation brings type and range validation for free. macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…

read full article on github.com
§ sources1 publication · timeline below
  1. github.comllama.cpp b9864primary