mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-06-30 00:22:10 -04:00
fix: Real-ESRGAN install + Cookbook deps-panel crash on the Python 3.14 image (#4694)
* fix(docker): make Real-ESRGAN installable on the Python 3.14 image realesrgan's deps basicsr/gfpgan/facexlib (unmaintained since 2022) read their version in setup.py via `exec(...); locals()['__version__']`, which raises KeyError on Python 3.13+ — PEP 667 made locals() in a function an independent snapshot that exec() can no longer mutate. That fails the Cookbook "install realesrgan" sdist build on the python:3.14 base. Add a `realesrgan-wheels` builder stage that fetches the pinned sdists, patches get_version() to exec into an explicit namespace dict, and builds wheels; the final stage installs them --no-deps so a later `pip install realesrgan` resolves from wheels instead of rebuilding the broken sdists. torch stays a runtime pull to keep the base image lean. Also add the runtime libs opencv-python (cv2) needs — libgl1, libglib2.0-0t64, libxcb1 — which the slim base omits; without them the install succeeds but `import cv2` dies with `libxcb.so.1: cannot open shared object file`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cookbook): don't let a package's sys.exit() on import hang the deps panel The local optional-dependency probe imports each package in-process and catches ImportError / Exception. But a package can call sys.exit() at import time — e.g. rembg does `sys.exit(1)` when no onnxruntime backend loads. SystemExit is a BaseException, not Exception, so it escaped the probe, propagated out of the list_packages endpoint, and hung the whole Dependencies panel / worker (the UI loads forever). Catch (Exception, SystemExit) so one broken optional package is reported as not-usable instead of taking down the panel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+30
@@ -1,3 +1,14 @@
|
|||||||
|
# ---- builder: patch + build wheels for Real-ESRGAN's broken-on-3.14 deps ----
|
||||||
|
# basicsr/gfpgan/facexlib read their version via exec()+locals()['__version__'],
|
||||||
|
# which raises KeyError on Python 3.13+ (PEP 667). Build patched wheels here so
|
||||||
|
# the final image / Cookbook never has to compile the broken sdists. See
|
||||||
|
# docker/build-realesrgan-wheels.sh for the full rationale.
|
||||||
|
FROM python:3.14-slim AS realesrgan-wheels
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends curl \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
COPY docker/build-realesrgan-wheels.sh /usr/local/bin/build-realesrgan-wheels.sh
|
||||||
|
RUN bash /usr/local/bin/build-realesrgan-wheels.sh /wheels
|
||||||
|
|
||||||
FROM python:3.14-slim
|
FROM python:3.14-slim
|
||||||
|
|
||||||
# System deps. tmux is required by Cookbook for background downloads/serves.
|
# System deps. tmux is required by Cookbook for background downloads/serves.
|
||||||
@@ -18,8 +29,18 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
|||||||
tmux \
|
tmux \
|
||||||
openssh-client \
|
openssh-client \
|
||||||
gosu \
|
gosu \
|
||||||
|
libgl1 \
|
||||||
|
libglib2.0-0t64 \
|
||||||
|
libxcb1 \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# libgl1/libglib2.0-0t64/libxcb1 are runtime shared libs (libGL.so.1,
|
||||||
|
# libglib-2.0/libgthread, libxcb.so.1) that opencv-python (cv2) loads. The
|
||||||
|
# slim base omits them, so the Cookbook "install realesrgan" path imports cv2
|
||||||
|
# and dies with `libxcb.so.1: cannot open shared object file` despite a clean
|
||||||
|
# pip install. Using full opencv-python (not -headless) because basicsr/gfpgan/
|
||||||
|
# facexlib/realesrgan all depend on the `opencv-python` distribution by name.
|
||||||
|
|
||||||
# Docker CLI (client only — daemon stays on the host via the
|
# Docker CLI (client only — daemon stays on the host via the
|
||||||
# /var/run/docker.sock mount). The Debian `docker.io` package ships
|
# /var/run/docker.sock mount). The Debian `docker.io` package ships
|
||||||
# dockerd but not the client binary on slim, so grab the static client
|
# dockerd but not the client binary on slim, so grab the static client
|
||||||
@@ -46,6 +67,15 @@ COPY requirements.txt requirements-optional.txt ./
|
|||||||
RUN pip install --no-cache-dir -r requirements.txt \
|
RUN pip install --no-cache-dir -r requirements.txt \
|
||||||
&& if [ "$INSTALL_OPTIONAL" = "true" ]; then pip install --no-cache-dir -r requirements-optional.txt; fi
|
&& if [ "$INSTALL_OPTIONAL" = "true" ]; then pip install --no-cache-dir -r requirements-optional.txt; fi
|
||||||
|
|
||||||
|
# Pre-install the patched basicsr/gfpgan/facexlib wheels built in the
|
||||||
|
# realesrgan-wheels stage (--no-deps keeps the image lean — torch & friends are
|
||||||
|
# pulled only when realesrgan is actually installed). With these dists already
|
||||||
|
# satisfied, the Cookbook's plain `pip install realesrgan` resolves them from
|
||||||
|
# wheels instead of rebuilding the sdists that fail on Python 3.14.
|
||||||
|
COPY --from=realesrgan-wheels /wheels/ /tmp/odysseus-wheels/
|
||||||
|
RUN pip install --no-cache-dir --no-deps /tmp/odysseus-wheels/*.whl \
|
||||||
|
&& rm -rf /tmp/odysseus-wheels
|
||||||
|
|
||||||
# Copy app code
|
# Copy app code
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
|
|||||||
Executable
+70
@@ -0,0 +1,70 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Build patched wheels for Real-ESRGAN's unmaintained dependencies.
|
||||||
|
#
|
||||||
|
# basicsr / gfpgan / facexlib (xinntao, last released 2022) read their version
|
||||||
|
# in setup.py with:
|
||||||
|
#
|
||||||
|
# exec(compile(f.read(), version_file, 'exec'))
|
||||||
|
# return locals()['__version__']
|
||||||
|
#
|
||||||
|
# Python 3.13+ implements PEP 667: locals() inside a function returns an
|
||||||
|
# independent snapshot that exec() can no longer mutate, so the read raises
|
||||||
|
# `KeyError: '__version__'` and the sdist build fails. That is why the Cookbook
|
||||||
|
# "install realesrgan" button dies on the python:3.14 image. The packages have
|
||||||
|
# no fixed release, so we patch get_version() to exec into an explicit namespace
|
||||||
|
# dict (works on every Python) and build wheels from the patched source.
|
||||||
|
#
|
||||||
|
# Usage: build-realesrgan-wheels.sh [OUTPUT_DIR] (default: /wheels)
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
OUT="${1:-/wheels}"
|
||||||
|
mkdir -p "$OUT"
|
||||||
|
|
||||||
|
work="$(mktemp -d)"
|
||||||
|
trap 'rm -rf "$work"' EXIT
|
||||||
|
cd "$work"
|
||||||
|
|
||||||
|
# Pinned to the versions Real-ESRGAN 0.3.0 resolves to.
|
||||||
|
SPECS="basicsr==1.4.2 gfpgan==1.3.8 facexlib==0.3.0"
|
||||||
|
|
||||||
|
for spec in $SPECS; do
|
||||||
|
name="${spec%%==*}"
|
||||||
|
ver="${spec##*==}"
|
||||||
|
# pip download builds metadata (and trips the same bug), so fetch the raw
|
||||||
|
# sdist URL from the PyPI JSON API instead.
|
||||||
|
url="$(python - "$name" "$ver" <<'PY'
|
||||||
|
import json, sys, urllib.request
|
||||||
|
name, ver = sys.argv[1], sys.argv[2]
|
||||||
|
data = json.load(urllib.request.urlopen(f"https://pypi.org/pypi/{name}/{ver}/json"))
|
||||||
|
for f in data["urls"]:
|
||||||
|
if f["packagetype"] == "sdist":
|
||||||
|
print(f["url"]); break
|
||||||
|
else:
|
||||||
|
sys.exit(f"no sdist found for {name}=={ver}")
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
echo ">> fetching ${name} ${ver}: ${url}"
|
||||||
|
curl -fsSL "$url" -o "${name}.tar.gz"
|
||||||
|
tar xzf "${name}.tar.gz"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo ">> patching get_version()"
|
||||||
|
python - <<'PY'
|
||||||
|
import pathlib
|
||||||
|
old_exec = "exec(compile(f.read(), version_file, 'exec'))"
|
||||||
|
new_exec = "_ver_ns = {}\n exec(compile(f.read(), version_file, 'exec'), _ver_ns)"
|
||||||
|
old_ret = "return locals()['__version__']"
|
||||||
|
new_ret = "return _ver_ns['__version__']"
|
||||||
|
patched = 0
|
||||||
|
for setup in pathlib.Path(".").glob("*/setup.py"):
|
||||||
|
s = setup.read_text()
|
||||||
|
if old_exec in s and old_ret in s:
|
||||||
|
setup.write_text(s.replace(old_exec, new_exec).replace(old_ret, new_ret))
|
||||||
|
print(" patched", setup)
|
||||||
|
patched += 1
|
||||||
|
assert patched == 3, f"expected to patch 3 setup.py files, patched {patched}"
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo ">> building wheels into ${OUT}"
|
||||||
|
pip wheel --no-deps -w "$OUT" ./basicsr-* ./gfpgan-* ./facexlib-*
|
||||||
|
ls -l "$OUT"
|
||||||
@@ -1377,11 +1377,16 @@ def setup_shell_routes() -> APIRouter:
|
|||||||
pkg["installed"] = False
|
pkg["installed"] = False
|
||||||
except importlib_metadata.PackageNotFoundError:
|
except importlib_metadata.PackageNotFoundError:
|
||||||
pkg["installed"] = False
|
pkg["installed"] = False
|
||||||
except Exception:
|
except (Exception, SystemExit):
|
||||||
# Installed but crashes on import — e.g. a CUDA build of
|
# Installed but crashes on import — e.g. a CUDA build of
|
||||||
# llama-cpp-python raising FileNotFoundError when the CUDA
|
# llama-cpp-python raising FileNotFoundError when the CUDA
|
||||||
# toolkit dir is absent. One broken optional package must not
|
# toolkit dir is absent, or rembg calling sys.exit(1) when no
|
||||||
# 500 the entire packages panel; report it as not usable.
|
# onnxruntime backend can be loaded. SystemExit is a
|
||||||
|
# BaseException, not Exception, so without catching it here a
|
||||||
|
# single sys.exit-on-import package escapes and takes down the
|
||||||
|
# whole packages panel / worker (the panel hangs forever). One
|
||||||
|
# broken optional package must not 500 — or hang — the entire
|
||||||
|
# panel; report it as not usable.
|
||||||
pkg["installed"] = False
|
pkg["installed"] = False
|
||||||
|
|
||||||
# llama_cpp partial-state probe: when the package is installed
|
# llama_cpp partial-state probe: when the package is installed
|
||||||
|
|||||||
Reference in New Issue
Block a user