mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-06-23 21:25:33 -04:00
fix: Real-ESRGAN install + Cookbook deps-panel crash on the Python 3.14 image (#4694)
* fix(docker): make Real-ESRGAN installable on the Python 3.14 image realesrgan's deps basicsr/gfpgan/facexlib (unmaintained since 2022) read their version in setup.py via `exec(...); locals()['__version__']`, which raises KeyError on Python 3.13+ — PEP 667 made locals() in a function an independent snapshot that exec() can no longer mutate. That fails the Cookbook "install realesrgan" sdist build on the python:3.14 base. Add a `realesrgan-wheels` builder stage that fetches the pinned sdists, patches get_version() to exec into an explicit namespace dict, and builds wheels; the final stage installs them --no-deps so a later `pip install realesrgan` resolves from wheels instead of rebuilding the broken sdists. torch stays a runtime pull to keep the base image lean. Also add the runtime libs opencv-python (cv2) needs — libgl1, libglib2.0-0t64, libxcb1 — which the slim base omits; without them the install succeeds but `import cv2` dies with `libxcb.so.1: cannot open shared object file`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cookbook): don't let a package's sys.exit() on import hang the deps panel The local optional-dependency probe imports each package in-process and catches ImportError / Exception. But a package can call sys.exit() at import time — e.g. rembg does `sys.exit(1)` when no onnxruntime backend loads. SystemExit is a BaseException, not Exception, so it escaped the probe, propagated out of the list_packages endpoint, and hung the whole Dependencies panel / worker (the UI loads forever). Catch (Exception, SystemExit) so one broken optional package is reported as not-usable instead of taking down the panel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+30
@@ -1,3 +1,14 @@
|
||||
# ---- builder: patch + build wheels for Real-ESRGAN's broken-on-3.14 deps ----
|
||||
# basicsr/gfpgan/facexlib read their version via exec()+locals()['__version__'],
|
||||
# which raises KeyError on Python 3.13+ (PEP 667). Build patched wheels here so
|
||||
# the final image / Cookbook never has to compile the broken sdists. See
|
||||
# docker/build-realesrgan-wheels.sh for the full rationale.
|
||||
FROM python:3.14-slim AS realesrgan-wheels
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends curl \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY docker/build-realesrgan-wheels.sh /usr/local/bin/build-realesrgan-wheels.sh
|
||||
RUN bash /usr/local/bin/build-realesrgan-wheels.sh /wheels
|
||||
|
||||
FROM python:3.14-slim
|
||||
|
||||
# System deps. tmux is required by Cookbook for background downloads/serves.
|
||||
@@ -18,8 +29,18 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
tmux \
|
||||
openssh-client \
|
||||
gosu \
|
||||
libgl1 \
|
||||
libglib2.0-0t64 \
|
||||
libxcb1 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# libgl1/libglib2.0-0t64/libxcb1 are runtime shared libs (libGL.so.1,
|
||||
# libglib-2.0/libgthread, libxcb.so.1) that opencv-python (cv2) loads. The
|
||||
# slim base omits them, so the Cookbook "install realesrgan" path imports cv2
|
||||
# and dies with `libxcb.so.1: cannot open shared object file` despite a clean
|
||||
# pip install. Using full opencv-python (not -headless) because basicsr/gfpgan/
|
||||
# facexlib/realesrgan all depend on the `opencv-python` distribution by name.
|
||||
|
||||
# Docker CLI (client only — daemon stays on the host via the
|
||||
# /var/run/docker.sock mount). The Debian `docker.io` package ships
|
||||
# dockerd but not the client binary on slim, so grab the static client
|
||||
@@ -46,6 +67,15 @@ COPY requirements.txt requirements-optional.txt ./
|
||||
RUN pip install --no-cache-dir -r requirements.txt \
|
||||
&& if [ "$INSTALL_OPTIONAL" = "true" ]; then pip install --no-cache-dir -r requirements-optional.txt; fi
|
||||
|
||||
# Pre-install the patched basicsr/gfpgan/facexlib wheels built in the
|
||||
# realesrgan-wheels stage (--no-deps keeps the image lean — torch & friends are
|
||||
# pulled only when realesrgan is actually installed). With these dists already
|
||||
# satisfied, the Cookbook's plain `pip install realesrgan` resolves them from
|
||||
# wheels instead of rebuilding the sdists that fail on Python 3.14.
|
||||
COPY --from=realesrgan-wheels /wheels/ /tmp/odysseus-wheels/
|
||||
RUN pip install --no-cache-dir --no-deps /tmp/odysseus-wheels/*.whl \
|
||||
&& rm -rf /tmp/odysseus-wheels
|
||||
|
||||
# Copy app code
|
||||
COPY . .
|
||||
|
||||
|
||||
Executable
+70
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build patched wheels for Real-ESRGAN's unmaintained dependencies.
|
||||
#
|
||||
# basicsr / gfpgan / facexlib (xinntao, last released 2022) read their version
|
||||
# in setup.py with:
|
||||
#
|
||||
# exec(compile(f.read(), version_file, 'exec'))
|
||||
# return locals()['__version__']
|
||||
#
|
||||
# Python 3.13+ implements PEP 667: locals() inside a function returns an
|
||||
# independent snapshot that exec() can no longer mutate, so the read raises
|
||||
# `KeyError: '__version__'` and the sdist build fails. That is why the Cookbook
|
||||
# "install realesrgan" button dies on the python:3.14 image. The packages have
|
||||
# no fixed release, so we patch get_version() to exec into an explicit namespace
|
||||
# dict (works on every Python) and build wheels from the patched source.
|
||||
#
|
||||
# Usage: build-realesrgan-wheels.sh [OUTPUT_DIR] (default: /wheels)
|
||||
set -euo pipefail
|
||||
|
||||
OUT="${1:-/wheels}"
|
||||
mkdir -p "$OUT"
|
||||
|
||||
work="$(mktemp -d)"
|
||||
trap 'rm -rf "$work"' EXIT
|
||||
cd "$work"
|
||||
|
||||
# Pinned to the versions Real-ESRGAN 0.3.0 resolves to.
|
||||
SPECS="basicsr==1.4.2 gfpgan==1.3.8 facexlib==0.3.0"
|
||||
|
||||
for spec in $SPECS; do
|
||||
name="${spec%%==*}"
|
||||
ver="${spec##*==}"
|
||||
# pip download builds metadata (and trips the same bug), so fetch the raw
|
||||
# sdist URL from the PyPI JSON API instead.
|
||||
url="$(python - "$name" "$ver" <<'PY'
|
||||
import json, sys, urllib.request
|
||||
name, ver = sys.argv[1], sys.argv[2]
|
||||
data = json.load(urllib.request.urlopen(f"https://pypi.org/pypi/{name}/{ver}/json"))
|
||||
for f in data["urls"]:
|
||||
if f["packagetype"] == "sdist":
|
||||
print(f["url"]); break
|
||||
else:
|
||||
sys.exit(f"no sdist found for {name}=={ver}")
|
||||
PY
|
||||
)"
|
||||
echo ">> fetching ${name} ${ver}: ${url}"
|
||||
curl -fsSL "$url" -o "${name}.tar.gz"
|
||||
tar xzf "${name}.tar.gz"
|
||||
done
|
||||
|
||||
echo ">> patching get_version()"
|
||||
python - <<'PY'
|
||||
import pathlib
|
||||
old_exec = "exec(compile(f.read(), version_file, 'exec'))"
|
||||
new_exec = "_ver_ns = {}\n exec(compile(f.read(), version_file, 'exec'), _ver_ns)"
|
||||
old_ret = "return locals()['__version__']"
|
||||
new_ret = "return _ver_ns['__version__']"
|
||||
patched = 0
|
||||
for setup in pathlib.Path(".").glob("*/setup.py"):
|
||||
s = setup.read_text()
|
||||
if old_exec in s and old_ret in s:
|
||||
setup.write_text(s.replace(old_exec, new_exec).replace(old_ret, new_ret))
|
||||
print(" patched", setup)
|
||||
patched += 1
|
||||
assert patched == 3, f"expected to patch 3 setup.py files, patched {patched}"
|
||||
PY
|
||||
|
||||
echo ">> building wheels into ${OUT}"
|
||||
pip wheel --no-deps -w "$OUT" ./basicsr-* ./gfpgan-* ./facexlib-*
|
||||
ls -l "$OUT"
|
||||
@@ -1377,11 +1377,16 @@ def setup_shell_routes() -> APIRouter:
|
||||
pkg["installed"] = False
|
||||
except importlib_metadata.PackageNotFoundError:
|
||||
pkg["installed"] = False
|
||||
except Exception:
|
||||
except (Exception, SystemExit):
|
||||
# Installed but crashes on import — e.g. a CUDA build of
|
||||
# llama-cpp-python raising FileNotFoundError when the CUDA
|
||||
# toolkit dir is absent. One broken optional package must not
|
||||
# 500 the entire packages panel; report it as not usable.
|
||||
# toolkit dir is absent, or rembg calling sys.exit(1) when no
|
||||
# onnxruntime backend can be loaded. SystemExit is a
|
||||
# BaseException, not Exception, so without catching it here a
|
||||
# single sys.exit-on-import package escapes and takes down the
|
||||
# whole packages panel / worker (the panel hangs forever). One
|
||||
# broken optional package must not 500 — or hang — the entire
|
||||
# panel; report it as not usable.
|
||||
pkg["installed"] = False
|
||||
|
||||
# llama_cpp partial-state probe: when the package is installed
|
||||
|
||||
Reference in New Issue
Block a user