mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-08-02 03:28:36 -04:00
a95ac7b47d
These models OOM on --kv-cache-dtype auto (≈bf16) at any usable context with current tensor-parallel layouts. _detectModelOptimizations now seeds opts.kvCacheDtype='fp8' for them, and the serve panel's KV Cache select picks that up as the default unless the user has a saved override on this skill.