mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-06-30 00:22:10 -04:00
Add a 'Rebuild llama.cpp' Cookbook action to force a fresh GPU build (#1787)
The serve bootstrap builds llama-server from source only when it is missing from PATH, so a host that first compiled CPU-only (no nvcc present at build time) reuses that CPU-only binary on every later serve and never gets a GPU build, even after a CUDA/ROCm toolkit is installed. There was no UI lever to force a rebuild. Adds a 'Rebuild llama.cpp' button to the Cookbook Dependencies tab. It clears the cached ~/bin/llama-server symlink and ~/llama.cpp/build directory (locally or on the selected remote server) so the next serve recompiles and picks up CUDA/HIP if a toolchain is now present. It installs and downloads nothing. - routes/cookbook_helpers.py: _llama_cpp_rebuild_cmd() (single source of truth) - routes/shell_routes.py: POST /api/cookbook/rebuild-engine (admin-only, reuses the existing SSH plumbing for remote hosts) - static/js/cookbook.js: header button + handler honoring the deps server selector - tests: cover the command shape and a clean run on a fresh HOME Motivated by #831 (RTX 4070 user stuck on a CPU-only build with no way to re-trigger the build). Co-authored-by: ghreprimand <203024559+ghreprimand@users.noreply.github.com>
This commit is contained in:
@@ -552,6 +552,27 @@ def _append_llama_cpp_linux_accel_build_lines(runner_lines: list[str]) -> None:
|
||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||
runner_lines.append(' fi')
|
||||
|
||||
|
||||
def _llama_cpp_rebuild_cmd() -> str:
|
||||
"""Shell command that clears the Cookbook-managed llama.cpp build.
|
||||
|
||||
Removes the cached ``llama-server`` symlink and the ``~/llama.cpp/build``
|
||||
directory so the next llama.cpp serve recompiles from source, picking up a
|
||||
CUDA or HIP toolchain if one is now available. The serve bootstrap only
|
||||
builds when ``llama-server`` is missing from PATH, so without this an
|
||||
existing CPU-only build is reused forever. It deliberately installs and
|
||||
downloads nothing; the rebuild itself happens on the next serve.
|
||||
"""
|
||||
return (
|
||||
'mkdir -p "$HOME/bin" && '
|
||||
'rm -f "$HOME/bin/llama-server" && '
|
||||
'rm -rf "$HOME/llama.cpp/build" && '
|
||||
'echo "[odysseus] Cleared the cached llama.cpp build. '
|
||||
'Re-launch the serve task to rebuild llama-server from source '
|
||||
'(CUDA or HIP will be used if a toolchain is now available)."'
|
||||
)
|
||||
|
||||
|
||||
class ModelDownloadRequest(BaseModel):
|
||||
repo_id: str
|
||||
include: str | None = None # glob pattern e.g. "*Q4_K_M*"
|
||||
|
||||
Reference in New Issue
Block a user