Allow heading tags (h1-h6) in ContentSanitizer so news editor headings render

This commit is contained in:
2026-06-04 07:52:57 +02:00
parent 0b33a1b074
commit 15870ddb1f
191 changed files with 15453 additions and 1786 deletions

View File

@@ -0,0 +1,36 @@
FROM python:3.11-slim
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
unzip \
libgomp1 \
libgl1 \
libglib2.0-0 \
curl \
libvulkan1 \
mesa-vulkan-drivers \
gosu \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app ./app
COPY bin ./bin
COPY models ./models
COPY scripts ./scripts
RUN useradd --system --create-home worker \
&& mkdir -p /app/bin /app/models /app/storage/tmp /app/storage/output /app/app/models \
&& chmod +x /app/scripts/*.sh \
&& chown -R worker:worker /app
EXPOSE 8095
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD curl --fail http://127.0.0.1:8095/health || exit 1
ENTRYPOINT ["/app/scripts/entrypoint.sh"]
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8095"]

View File

@@ -0,0 +1,209 @@
# Skinbase Enhance Worker
This service is the private HTTP worker for the Skinbase Enhance `external_worker` engine. Laravel remains responsible for job state, permissions, queueing, moderation, storage, and cleanup. The worker only downloads the copied Enhance source image, upscales it, exposes a temporary result, and deletes that temporary result when Laravel is done.
## Local build and run
```bash
cd services/enhance-worker
docker compose -f docker-compose.example.yml up --build
curl http://127.0.0.1:8095/health
```
The example compose file uses `WORKER_ENGINE=pillow` so local development works without Real-ESRGAN weights.
## Real-ESRGAN ncnn runtime
The worker supports these engine values:
- `WORKER_ENGINE=pillow`
- `WORKER_ENGINE=realesrgan-ncnn`
- `WORKER_ENGINE=realesrgan`
`realesrgan` currently aliases to the ncnn-vulkan CLI path. Pillow mode remains the safe local and CI fallback.
The worker expects the Real-ESRGAN ncnn runtime at `/app/bin/realesrgan-ncnn-vulkan` and model files under `/app/models`. These files are not committed. Keep `bin/` and `models/` private to your deployment or local environment.
## Endpoints
- `GET /health`
- `POST /v1/upscale` with `Authorization: Bearer <WORKER_TOKEN>`
- `GET /v1/results/{filename}` for temporary internal result download
- `DELETE /v1/results/{filename}` with bearer token for Laravel cleanup
## Worker env
```env
WORKER_HOST=0.0.0.0
WORKER_PORT=8095
WORKER_TOKEN=change-this-token
WORKER_ENGINE=pillow
WORKER_DEVICE=cpu
WORKER_MAX_UPLOAD_MB=20
WORKER_MAX_INPUT_WIDTH=4096
WORKER_MAX_INPUT_HEIGHT=4096
WORKER_MAX_OUTPUT_WIDTH=8192
WORKER_MAX_OUTPUT_HEIGHT=8192
WORKER_TMP_DIR=/app/storage/tmp
WORKER_OUTPUT_DIR=/app/storage/output
WORKER_RESULT_TTL_MINUTES=60
WORKER_MODEL_DIR=/app/app/models
WORKER_DEFAULT_MODEL=realesrgan-x4plus
WORKER_REALESRGAN_BIN=/app/bin/realesrgan-ncnn-vulkan
WORKER_REALESRGAN_MODEL_DIR=/app/models
WORKER_REALESRGAN_DEFAULT_MODEL=realesrgan-x4plus
WORKER_REALESRGAN_ANIME_MODEL=realesrgan-x4plus-anime
WORKER_REALESRGAN_TILE=0
WORKER_REALESRGAN_TTA=false
WORKER_REALESRGAN_VERBOSE=false
WORKER_REALESRGAN_TIMEOUT_SECONDS=900
WORKER_REALESRGAN_PREPROCESS_MAX_PIXELS=16777216
WORKER_REALESRGAN_OUTPUT_EXT=webp
WORKER_REALESRGAN_ALLOW_MODEL_FALLBACK=true
```
## Laravel env
```env
ENHANCE_ENGINE=external_worker
ENHANCE_WORKER_URL=http://127.0.0.1:8095
ENHANCE_WORKER_TIMEOUT=600
ENHANCE_WORKER_TOKEN=change-this-token
ENHANCE_WORKER_MAX_DOWNLOAD_MB=60
ENHANCE_QUEUE=enhance
```
After updating Laravel env:
```bash
php artisan config:clear
php artisan enhance:health
php artisan queue:work --queue=enhance,default
php artisan test --filter=EnhanceExternalWorker
```
## Example request
```bash
curl -X POST http://127.0.0.1:8095/v1/upscale \
-H "Authorization: Bearer change-this-token" \
-H "Content-Type: application/json" \
-d '{
"job_id": 1,
"source_url": "https://example.com/test.webp",
"scale": 2,
"mode": "artwork",
"output_format": "webp"
}'
```
## Installing runtime files
```bash
cd services/enhance-worker
bash scripts/download-realesrgan-ncnn.sh
bash scripts/verify-realesrgan.sh
```
You can override the upstream release tag with:
```bash
bash scripts/download-realesrgan-ncnn.sh --version v0.2.0
```
If you already manage the binary and models yourself, place them in `bin/` and `models/` instead of using the helper script.
## Model files
Expected examples:
- `realesrgan-x4plus.param`
- `realesrgan-x4plus.bin`
- `realesrgan-x4plus-anime.param`
- `realesrgan-x4plus-anime.bin`
Illustration mode prefers the anime model. If that model is missing and `WORKER_REALESRGAN_ALLOW_MODEL_FALLBACK=true`, the worker falls back to the default model and records the fallback in response metadata.
## Running in Pillow mode
- `docker compose -f docker-compose.example.yml up --build`
- health should report `engine: pillow`
- metadata reports `real_ai_upscale: false`
## Running in Real-ESRGAN mode
```bash
cd services/enhance-worker
docker compose -f docker-compose.realesrgan.example.yml up --build
curl http://127.0.0.1:8095/health
```
If the binary or models are missing, health will report `status: degraded` and the worker will return a safe unavailable error for upscale requests.
## Health checks
`GET /health` returns general worker status for every engine.
When using `realesrgan-ncnn`, health also reports:
- whether the binary is configured
- whether the binary exists
- whether it is executable
- whether the model directory exists
- the available model list
- the configured default model
## Engines
- `WORKER_ENGINE=pillow`: local development and tests. Metadata reports `real_ai_upscale: false`.
- `WORKER_ENGINE=realesrgan-ncnn`: runs the ncnn-vulkan CLI through a safe `subprocess.run(..., shell=False)` path.
- `WORKER_ENGINE=realesrgan`: alias to `realesrgan-ncnn`.
2x handling uses the 4x model output and a deterministic high-quality downsample. 4x requests return the native model scale.
## CPU and GPU
CPU mode is supported by default.
Optional GPU deployment later:
- install NVIDIA Container Toolkit
- use a CUDA-compatible image and runtime
- set `WORKER_DEVICE=cuda`
- install Real-ESRGAN and its GPU-capable dependencies
Do not remove CPU fallback.
For `realesrgan-ncnn`, `WORKER_DEVICE=vulkan` is the intended production setting when the host has compatible drivers and GPU access.
## Performance notes
- 2x requests still pay much of the 4x cost because the worker currently runs the 4x model then downsamples.
- Very large images may need tiling. `WORKER_REALESRGAN_TILE=0` keeps the runtime default behavior.
- `WORKER_REALESRGAN_TTA=true` may improve quality but is slower.
- CPU-only Real-ESRGAN is suitable for verification and low-volume use, but usually too slow for production.
- Start production with conservative input limits and a dedicated `enhance` queue.
## Security notes
- Keep the worker on `127.0.0.1` or a private container network.
- Do not expose the worker publicly.
- Use a strong `WORKER_TOKEN`.
- The worker only accepts `http` and `https` source URLs.
- The result endpoint only serves sanitized filenames from the worker output directory.
- Temporary inputs are deleted after processing, and stale temporary outputs are pruned by TTL.
- Real-ESRGAN model names are selected from an internal allowlist only. Users cannot inject arbitrary CLI flags.
- The worker uses `subprocess.run` without `shell=True`.
## Troubleshooting
- `401 Unauthorized`: check `WORKER_TOKEN` and Laravel `ENHANCE_WORKER_TOKEN`.
- `Upscale engine is not available...`: the worker is in `realesrgan-ncnn` mode but the binary or models are missing, not executable, or the CLI run failed.
- `Worker rejected the image.`: invalid mode, scale, output format, MIME type, dimensions, or download size.
- `status: degraded` on `/health`: use `bash scripts/verify-realesrgan.sh` and either fix the runtime files or switch back to `WORKER_ENGINE=pillow`.
- Jobs stuck in `queued`: if `ENHANCE_QUEUE=enhance`, make sure workers or Horizon consume the `enhance` queue.

View File

@@ -0,0 +1 @@
# Package marker for the enhance worker app.

View File

@@ -0,0 +1,100 @@
from __future__ import annotations
import os
from dataclasses import dataclass
from functools import lru_cache
@dataclass(frozen=True)
class Settings:
host: str = "0.0.0.0"
port: int = 8095
token: str = "change-this-token"
engine: str = "pillow"
device: str = "cpu"
max_upload_mb: int = 20
max_input_width: int = 4096
max_input_height: int = 4096
max_output_width: int = 8192
max_output_height: int = 8192
tmp_dir: str = "/app/storage/tmp"
output_dir: str = "/app/storage/output"
result_ttl_minutes: int = 60
model_dir: str = "/app/app/models"
default_model: str = "realesrgan-x4plus"
realesrgan_bin: str = "/app/bin/realesrgan-ncnn-vulkan"
realesrgan_model_dir: str = "/app/models"
realesrgan_default_model: str = "realesrgan-x4plus"
realesrgan_anime_model: str = "realesrgan-x4plus-anime"
realesrgan_gpu_id: int = -1
realesrgan_tile: int = 0
realesrgan_tta: bool = False
realesrgan_verbose: bool = False
realesrgan_timeout_seconds: int = 900
realesrgan_preprocess_max_pixels: int = 16_777_216
realesrgan_output_ext: str = "webp"
realesrgan_allow_model_fallback: bool = True
def _env_int(name: str, default: int) -> int:
try:
return int(os.getenv(name, str(default)).strip())
except ValueError:
return default
def _env_bool(name: str, default: bool) -> bool:
value = os.getenv(name)
if value is None:
return default
normalized = value.strip().lower()
if normalized in {"1", "true", "yes", "on"}:
return True
if normalized in {"0", "false", "no", "off"}:
return False
return default
@lru_cache(maxsize=1)
def get_settings() -> Settings:
legacy_model_dir = os.getenv("WORKER_MODEL_DIR", "/app/app/models").strip() or "/app/app/models"
legacy_default_model = os.getenv("WORKER_DEFAULT_MODEL", "realesrgan-x4plus").strip() or "realesrgan-x4plus"
realesrgan_model_dir = os.getenv(
"WORKER_REALESRGAN_MODEL_DIR",
legacy_model_dir if legacy_model_dir != "/app/app/models" else "/app/models",
).strip() or (legacy_model_dir if legacy_model_dir != "/app/app/models" else "/app/models")
return Settings(
host=os.getenv("WORKER_HOST", "0.0.0.0").strip() or "0.0.0.0",
port=_env_int("WORKER_PORT", 8095),
token=os.getenv("WORKER_TOKEN", "change-this-token").strip(),
engine=os.getenv("WORKER_ENGINE", "pillow").strip().lower() or "pillow",
device=os.getenv("WORKER_DEVICE", "cpu").strip().lower() or "cpu",
max_upload_mb=max(1, _env_int("WORKER_MAX_UPLOAD_MB", 20)),
max_input_width=max(1, _env_int("WORKER_MAX_INPUT_WIDTH", 4096)),
max_input_height=max(1, _env_int("WORKER_MAX_INPUT_HEIGHT", 4096)),
max_output_width=max(1, _env_int("WORKER_MAX_OUTPUT_WIDTH", 8192)),
max_output_height=max(1, _env_int("WORKER_MAX_OUTPUT_HEIGHT", 8192)),
tmp_dir=os.getenv("WORKER_TMP_DIR", "/app/storage/tmp").strip() or "/app/storage/tmp",
output_dir=os.getenv("WORKER_OUTPUT_DIR", "/app/storage/output").strip() or "/app/storage/output",
result_ttl_minutes=max(1, _env_int("WORKER_RESULT_TTL_MINUTES", 60)),
model_dir=legacy_model_dir,
default_model=legacy_default_model,
realesrgan_bin=os.getenv("WORKER_REALESRGAN_BIN", "/app/bin/realesrgan-ncnn-vulkan").strip() or "/app/bin/realesrgan-ncnn-vulkan",
realesrgan_model_dir=realesrgan_model_dir,
realesrgan_default_model=os.getenv("WORKER_REALESRGAN_DEFAULT_MODEL", legacy_default_model).strip() or legacy_default_model,
realesrgan_anime_model=os.getenv("WORKER_REALESRGAN_ANIME_MODEL", "realesrgan-x4plus-anime").strip() or "realesrgan-x4plus-anime",
realesrgan_gpu_id=_env_int("WORKER_REALESRGAN_GPU_ID", -1),
realesrgan_tile=max(0, _env_int("WORKER_REALESRGAN_TILE", 0)),
realesrgan_tta=_env_bool("WORKER_REALESRGAN_TTA", False),
realesrgan_verbose=_env_bool("WORKER_REALESRGAN_VERBOSE", False),
realesrgan_timeout_seconds=max(1, _env_int("WORKER_REALESRGAN_TIMEOUT_SECONDS", 900)),
realesrgan_preprocess_max_pixels=max(1, _env_int("WORKER_REALESRGAN_PREPROCESS_MAX_PIXELS", 16_777_216)),
realesrgan_output_ext=os.getenv("WORKER_REALESRGAN_OUTPUT_EXT", "webp").strip().lower() or "webp",
realesrgan_allow_model_fallback=_env_bool("WORKER_REALESRGAN_ALLOW_MODEL_FALLBACK", True),
)

View File

@@ -0,0 +1,12 @@
from .base import EngineHealth, UpscaleEngine, UpscaleEngineUnavailable, UpscaleResult
from .pillow_engine import PillowUpscaleEngine
from .realesrgan_ncnn_engine import RealEsrganNcnnEngine
__all__ = [
"EngineHealth",
"PillowUpscaleEngine",
"RealEsrganNcnnEngine",
"UpscaleEngine",
"UpscaleEngineUnavailable",
"UpscaleResult",
]

View File

@@ -0,0 +1,43 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from typing import Any
from PIL import Image
from ..image_io import DownloadedImage
class UpscaleEngineUnavailable(RuntimeError):
pass
@dataclass(frozen=True)
class UpscaleResult:
image: Image.Image
metadata: dict[str, Any]
@dataclass(frozen=True)
class EngineHealth:
status: str
engine: str
device: str
models_loaded: bool
details: dict[str, Any] = field(default_factory=dict)
class UpscaleEngine(ABC):
@abstractmethod
def health(self) -> EngineHealth:
raise NotImplementedError
def available(self) -> bool:
health = self.health()
return health.status == "ok" and health.models_loaded
@abstractmethod
def upscale(self, downloaded: DownloadedImage, scale: int, mode: str, output_format: str) -> UpscaleResult:
raise NotImplementedError

View File

@@ -0,0 +1,71 @@
from __future__ import annotations
import time
from fastapi import HTTPException, status
from PIL import Image, ImageFilter
from ..config import Settings
from ..image_io import DownloadedImage, load_normalized_image
from .base import EngineHealth, UpscaleEngine, UpscaleResult
MODE_PROFILES = {
"standard": {"profile": "general", "sharpen_percent": 120, "radius": 1.0, "threshold": 3},
"artwork": {"profile": "artwork", "sharpen_percent": 150, "radius": 1.2, "threshold": 2},
"photo": {"profile": "photo", "sharpen_percent": 95, "radius": 0.8, "threshold": 4},
"illustration": {"profile": "illustration", "sharpen_percent": 135, "radius": 1.0, "threshold": 2},
}
class PillowUpscaleEngine(UpscaleEngine):
def __init__(self, settings: Settings) -> None:
self.settings = settings
def health(self) -> EngineHealth:
return EngineHealth(
status="ok",
engine="pillow",
device=self.settings.device,
models_loaded=True,
)
def upscale(self, downloaded: DownloadedImage, scale: int, mode: str, output_format: str) -> UpscaleResult:
started_at = time.perf_counter()
profile = MODE_PROFILES[mode]
image = load_normalized_image(downloaded.path)
width, height = image.size
target_width = width * scale
target_height = height * scale
if target_width > self.settings.max_output_width or target_height > self.settings.max_output_height:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
result = image.resize((target_width, target_height), Image.Resampling.LANCZOS)
result = result.filter(
ImageFilter.UnsharpMask(
radius=profile["radius"],
percent=profile["sharpen_percent"],
threshold=profile["threshold"],
)
)
return UpscaleResult(
image=result,
metadata={
"engine": "pillow",
"model": "pillow-lanczos",
"requested_scale": scale,
"native_model_scale": scale,
"mode": mode,
"device": self.settings.device,
"profile": profile["profile"],
"real_ai_upscale": False,
"processing_seconds": round(time.perf_counter() - started_at, 3),
"input_width": width,
"input_height": height,
"output_width": target_width,
"output_height": target_height,
"output_format": output_format,
},
)

View File

@@ -0,0 +1,214 @@
from __future__ import annotations
import logging
import os
import subprocess
import time
import uuid
from pathlib import Path
from fastapi import HTTPException, status
from PIL import Image
from ..config import Settings
from ..image_io import DownloadedImage, delete_temp_file, prepare_input_for_engine, validate_generated_image
from .base import EngineHealth, UpscaleEngine, UpscaleEngineUnavailable, UpscaleResult
LOGGER = logging.getLogger("skinbase.enhance_worker.realesrgan_ncnn")
MODE_MODEL_MAP = {
"standard": "default",
"artwork": "default",
"photo": "default",
"illustration": "anime",
}
class RealEsrganNcnnEngine(UpscaleEngine):
def __init__(self, settings: Settings) -> None:
self.settings = settings
def health(self) -> EngineHealth:
available_models = self.available_models()
binary_path = Path(self.settings.realesrgan_bin)
model_dir = Path(self.settings.realesrgan_model_dir)
binary_exists = binary_path.exists()
binary_executable = binary_exists and binary_path.is_file() and os.access(binary_path, os.X_OK)
model_dir_exists = model_dir.exists() and model_dir.is_dir()
models_loaded = self.settings.realesrgan_default_model in available_models
return EngineHealth(
status="ok" if binary_exists and binary_executable and model_dir_exists and models_loaded else "degraded",
engine="realesrgan-ncnn",
device=self.settings.device,
models_loaded=models_loaded,
details={
"realesrgan": {
"binary_configured": self.settings.realesrgan_bin.strip() != "",
"binary_exists": binary_exists,
"binary_executable": binary_executable,
"model_dir_exists": model_dir_exists,
"available_models": available_models,
"default_model": self.settings.realesrgan_default_model,
}
},
)
def available_models(self) -> list[str]:
model_dir = Path(self.settings.realesrgan_model_dir)
if not model_dir.exists() or not model_dir.is_dir():
return []
params = {path.stem for path in model_dir.glob("*.param")}
bins = {path.stem for path in model_dir.glob("*.bin")}
return sorted(params & bins)
def upscale(self, downloaded: DownloadedImage, scale: int, mode: str, output_format: str) -> UpscaleResult:
if self.health().status != "ok":
raise UpscaleEngineUnavailable("Upscale engine is not available. Check model files and worker installation.")
prepared = prepare_input_for_engine(downloaded, self.settings)
temp_output = Path(self.settings.tmp_dir) / f"realesrgan-output-{uuid.uuid4().hex}.png"
started_at = time.perf_counter()
try:
requested_model, used_model, model_fallback = self.resolve_model(mode)
command = self.build_command(prepared.path, temp_output, used_model)
self.run_command(command)
native_scale = 4
image, _, _, _, _ = validate_generated_image(
temp_output,
self.settings,
expected_width=prepared.width * native_scale,
expected_height=prepared.height * native_scale,
)
post_downsampled = False
if scale == 2:
image = image.resize((prepared.width * 2, prepared.height * 2), Image.Resampling.LANCZOS)
post_downsampled = True
output_width, output_height = image.size
if output_width > self.settings.max_output_width or output_height > self.settings.max_output_height:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail="Upscaled output exceeded the maximum allowed dimensions.",
)
return UpscaleResult(
image=image,
metadata={
"engine": "realesrgan-ncnn",
"model": used_model,
"requested_model": requested_model,
"used_model": used_model,
"model_fallback": model_fallback,
"requested_scale": scale,
"native_model_scale": native_scale,
"post_downsampled": post_downsampled,
"mode": mode,
"device": self.settings.device,
"processing_seconds": round(time.perf_counter() - started_at, 3),
"input_width": prepared.width,
"input_height": prepared.height,
"output_width": output_width,
"output_height": output_height,
"output_format": output_format,
"real_ai_upscale": True,
"configured_output_ext": self.settings.realesrgan_output_ext,
},
)
finally:
delete_temp_file(prepared.path)
delete_temp_file(temp_output)
def resolve_model(self, mode: str) -> tuple[str, str, bool]:
available_models = set(self.available_models())
requested_model = self.settings.realesrgan_default_model
if MODE_MODEL_MAP.get(mode) == "anime":
requested_model = self.settings.realesrgan_anime_model
if requested_model in available_models:
return requested_model, requested_model, False
if self.settings.realesrgan_allow_model_fallback and self.settings.realesrgan_default_model in available_models:
return requested_model, self.settings.realesrgan_default_model, True
raise UpscaleEngineUnavailable("Upscale engine is not available. Check model files and worker installation.")
def build_command(self, input_path: Path, output_path: Path, model_name: str) -> list[str]:
command = [
self.settings.realesrgan_bin,
"-i",
str(input_path),
"-o",
str(output_path),
"-n",
model_name,
"-m",
self.settings.realesrgan_model_dir,
]
if self.settings.realesrgan_gpu_id >= 0:
command.extend(["-g", str(self.settings.realesrgan_gpu_id)])
if self.settings.realesrgan_tile > 0:
command.extend(["-t", str(self.settings.realesrgan_tile)])
if self.settings.realesrgan_tta:
command.append("-x")
if self.settings.realesrgan_verbose:
command.append("-v")
return command
def run_command(self, command: list[str]) -> None:
import signal
try:
proc = subprocess.Popen(
command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
start_new_session=True, # new process group so we can kill all descendants
)
except FileNotFoundError as exception:
raise UpscaleEngineUnavailable("Upscale engine is not available. Check model files and worker installation.") from exception
pgid = os.getpgid(proc.pid)
def _kill_group() -> None:
try:
os.killpg(pgid, signal.SIGKILL)
except ProcessLookupError:
pass
try:
stdout, stderr = proc.communicate(timeout=self.settings.realesrgan_timeout_seconds)
except subprocess.TimeoutExpired:
_kill_group()
proc.communicate()
LOGGER.warning("Real-ESRGAN ncnn command timed out after %s seconds", self.settings.realesrgan_timeout_seconds)
raise UpscaleEngineUnavailable("Upscale engine is not available. Check model files and worker installation.")
except BaseException:
# Thread cancellation or other unexpected error — ensure the process is killed
_kill_group()
proc.communicate()
raise
if proc.returncode != 0:
LOGGER.warning(
"Real-ESRGAN ncnn command failed with code %s; stdout bytes=%s stderr bytes=%s",
proc.returncode,
len(stdout or ""),
len(stderr or ""),
)
raise UpscaleEngineUnavailable("Upscale engine is not available. Check model files and worker installation.")

View File

@@ -0,0 +1,236 @@
from __future__ import annotations
import io
import os
import uuid
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path
import httpx
from fastapi import HTTPException, status
from PIL import Image, ImageOps
from .config import Settings
ALLOWED_MIMES = {"image/jpeg", "image/png", "image/webp"}
FORMAT_TO_MIME = {"jpg": "image/jpeg", "png": "image/png", "webp": "image/webp"}
FORMAT_TO_EXTENSION = {"JPEG": "jpg", "PNG": "png", "WEBP": "webp"}
OUTPUT_FORMATS = {"jpg": "JPEG", "png": "PNG", "webp": "WEBP"}
@dataclass(frozen=True)
class DownloadedImage:
path: Path
width: int
height: int
mime: str
filesize: int
@dataclass(frozen=True)
class StoredImage:
filename: str
path: Path
width: int
height: int
filesize: int
mime: str
@dataclass(frozen=True)
class PreparedImage:
path: Path
width: int
height: int
mime: str
def ensure_directories(settings: Settings) -> None:
Path(settings.tmp_dir).mkdir(parents=True, exist_ok=True)
Path(settings.output_dir).mkdir(parents=True, exist_ok=True)
Path(settings.model_dir).mkdir(parents=True, exist_ok=True)
Path(settings.realesrgan_model_dir).mkdir(parents=True, exist_ok=True)
Path(settings.realesrgan_bin).parent.mkdir(parents=True, exist_ok=True)
def cleanup_expired_files(settings: Settings) -> None:
threshold = datetime.now(timezone.utc) - timedelta(minutes=settings.result_ttl_minutes)
for directory in (Path(settings.tmp_dir), Path(settings.output_dir)):
if not directory.exists():
continue
for item in directory.iterdir():
if not item.is_file():
continue
modified_at = datetime.fromtimestamp(item.stat().st_mtime, tz=timezone.utc)
if modified_at <= threshold:
item.unlink(missing_ok=True)
def validate_image_bytes(binary: bytes, max_width: int, max_height: int) -> tuple[int, int, str]:
try:
with Image.open(io.BytesIO(binary)) as image:
width, height = image.size
mime = Image.MIME.get(image.format or "", "").lower()
except Exception as exc: # pragma: no cover - Pillow raises multiple subclasses.
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.") from exc
if mime not in ALLOWED_MIMES:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
if width < 1 or height < 1 or width > max_width or height > max_height:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
return width, height, mime
def download_source_image(source_url: str, settings: Settings) -> DownloadedImage:
max_bytes = settings.max_upload_mb * 1024 * 1024
try:
with httpx.stream("GET", source_url, follow_redirects=True, timeout=30.0) as response:
response.raise_for_status()
content_length = response.headers.get("content-length")
if content_length and int(content_length) > max_bytes:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
buffer = bytearray()
for chunk in response.iter_bytes():
buffer.extend(chunk)
if len(buffer) > max_bytes:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
binary = bytes(buffer)
except HTTPException:
raise
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail="The source file could not be downloaded by the worker.",
) from exc
width, height, mime = validate_image_bytes(binary, settings.max_input_width, settings.max_input_height)
extension = mime.split("/")[-1].replace("jpeg", "jpg")
path = Path(settings.tmp_dir) / f"input-{uuid.uuid4().hex}.{extension}"
path.write_bytes(binary)
return DownloadedImage(path=path, width=width, height=height, mime=mime, filesize=len(binary))
def save_output_image(image: Image.Image, output_format: str, settings: Settings, job_id: int) -> StoredImage:
width, height = image.size
if width < 1 or height < 1 or width > settings.max_output_width or height > settings.max_output_height:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
target_format = OUTPUT_FORMATS[output_format]
filename = f"job-{job_id}-{uuid.uuid4().hex}.{FORMAT_TO_EXTENSION[target_format]}"
path = Path(settings.output_dir) / filename
save_image = image
if target_format == "JPEG" and image.mode not in {"RGB", "L"}:
save_image = image.convert("RGB")
kwargs: dict[str, int] = {}
if target_format == "WEBP":
kwargs = {"quality": 90, "method": 6}
elif target_format == "JPEG":
kwargs = {"quality": 92}
save_image.save(path, target_format, **kwargs)
return StoredImage(
filename=filename,
path=path,
width=width,
height=height,
filesize=path.stat().st_size,
mime=FORMAT_TO_MIME[output_format],
)
def prepare_input_for_engine(downloaded: DownloadedImage, settings: Settings) -> PreparedImage:
image = load_normalized_image(downloaded.path)
width, height = image.size
if width * height > settings.realesrgan_preprocess_max_pixels:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
prepared_path = Path(settings.tmp_dir) / f"prepared-{uuid.uuid4().hex}.png"
prepared_path.parent.mkdir(parents=True, exist_ok=True)
prepared_image = image
if prepared_image.mode not in {"RGB", "RGBA", "L", "LA"}:
prepared_image = prepared_image.convert("RGBA" if "A" in prepared_image.getbands() else "RGB")
prepared_image.save(prepared_path, "PNG")
return PreparedImage(
path=prepared_path,
width=width,
height=height,
mime="image/png",
)
def validate_generated_image(
path: Path,
settings: Settings,
*,
expected_width: int | None = None,
expected_height: int | None = None,
) -> tuple[Image.Image, int, int, int, str]:
if not path.exists() or not path.is_file():
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
filesize = path.stat().st_size
if filesize <= 0:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
image = load_normalized_image(path)
width, height = image.size
if width > settings.max_output_width or height > settings.max_output_height:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail="Upscaled output exceeded the maximum allowed dimensions.",
)
if expected_width is not None and expected_height is not None and (width != expected_width or height != expected_height):
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
mime = Image.MIME.get(image.format or "", "").lower() or "image/png"
if mime not in ALLOWED_MIMES:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail="Worker rejected the image.")
return image, width, height, filesize, mime
def delete_temp_file(path: Path | None) -> None:
if path is None:
return
path.unlink(missing_ok=True)
def resolve_result_path(settings: Settings, filename: str) -> Path:
safe_name = os.path.basename(filename)
if safe_name != filename or safe_name == "":
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Not found")
return Path(settings.output_dir) / safe_name
def load_normalized_image(path: Path) -> Image.Image:
with Image.open(path) as image:
normalized = ImageOps.exif_transpose(image)
normalized.load()
return normalized

View File

@@ -0,0 +1,100 @@
from __future__ import annotations
from contextlib import suppress
from fastapi import Depends, FastAPI, HTTPException, Request, status
from fastapi.responses import FileResponse, JSONResponse
from .config import Settings, get_settings
from .image_io import (
cleanup_expired_files,
delete_temp_file,
download_source_image,
ensure_directories,
resolve_result_path,
save_output_image,
)
from .schemas import HealthResponse, UpscaleRequest, UpscaleResponse
from .security import verify_bearer_token
from .upscaler import UpscaleEngineUnavailable, build_upscaler
def create_app(settings: Settings | None = None) -> FastAPI:
app = FastAPI(title="skinbase-enhance-worker", version="1.0.0")
resolved_settings = settings or get_settings()
ensure_directories(resolved_settings)
cleanup_expired_files(resolved_settings)
app.state.settings = resolved_settings
app.state.upscaler = build_upscaler(resolved_settings)
@app.get("/health", response_model=HealthResponse)
def health() -> HealthResponse:
engine_health = app.state.upscaler.health()
with suppress(Exception):
cleanup_expired_files(app.state.settings)
return HealthResponse(
status=engine_health.status,
service="skinbase-enhance-worker",
engine=engine_health.engine,
device=engine_health.device,
models_loaded=engine_health.models_loaded,
max_input_width=app.state.settings.max_input_width,
max_input_height=app.state.settings.max_input_height,
max_output_width=app.state.settings.max_output_width,
max_output_height=app.state.settings.max_output_height,
realesrgan=engine_health.details.get("realesrgan"),
)
@app.post("/v1/upscale", response_model=UpscaleResponse)
def upscale(payload: UpscaleRequest, request: Request, _: None = Depends(verify_bearer_token)):
cleanup_expired_files(app.state.settings)
downloaded = None
try:
downloaded = download_source_image(payload.source_url, app.state.settings)
result = app.state.upscaler.upscale(downloaded, payload.scale, payload.mode, payload.output_format)
stored = save_output_image(result.image, payload.output_format, app.state.settings, payload.job_id)
return UpscaleResponse(
success=True,
job_id=payload.job_id,
output_url=str(request.base_url).rstrip("/") + f"/v1/results/{stored.filename}",
width=stored.width,
height=stored.height,
filesize=stored.filesize,
mime=stored.mime,
metadata=result.metadata,
)
except HTTPException as exc:
return JSONResponse(status_code=exc.status_code, content={"success": False, "error": exc.detail})
except UpscaleEngineUnavailable as exc:
return JSONResponse(status_code=status.HTTP_503_SERVICE_UNAVAILABLE, content={"success": False, "error": str(exc)})
finally:
delete_temp_file(downloaded.path if downloaded is not None else None)
@app.get("/v1/results/{filename}")
def result(filename: str):
path = resolve_result_path(app.state.settings, filename)
if not path.exists() or not path.is_file():
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Not found")
return FileResponse(path)
@app.delete("/v1/results/{filename}")
def delete_result(filename: str, _: None = Depends(verify_bearer_token)):
path = resolve_result_path(app.state.settings, filename)
if not path.exists() or not path.is_file():
return {"success": True, "deleted": False}
path.unlink(missing_ok=True)
return {"success": True, "deleted": True}
return app
app = create_app()

View File

@@ -0,0 +1 @@
keep

View File

@@ -0,0 +1,64 @@
from __future__ import annotations
from typing import Any, Literal
from urllib.parse import urlparse
from pydantic import BaseModel, ConfigDict, Field, field_validator
SUPPORTED_MODES = {"standard", "artwork", "photo", "illustration"}
SUPPORTED_FORMATS = {"webp", "png", "jpg"}
SUPPORTED_SCALES = {2, 4}
class UpscaleRequest(BaseModel):
job_id: int = Field(..., gt=0)
source_url: str
scale: Literal[2, 4]
mode: Literal["standard", "artwork", "photo", "illustration"]
output_format: Literal["webp", "png", "jpg"] = "webp"
@field_validator("source_url")
@classmethod
def validate_source_url(cls, value: str) -> str:
candidate = value.strip()
if candidate == "":
raise ValueError("source_url is required")
if candidate.startswith(("/", "./", "../", "file://")):
raise ValueError("source_url must be an http or https URL")
parsed = urlparse(candidate)
if parsed.scheme not in {"http", "https"} or parsed.netloc == "":
raise ValueError("source_url must be an http or https URL")
return candidate
class HealthResponse(BaseModel):
status: str
service: str
engine: str
device: str
models_loaded: bool
max_input_width: int
max_input_height: int
max_output_width: int
max_output_height: int
realesrgan: dict[str, Any] | None = None
class UpscaleResponse(BaseModel):
success: bool
job_id: int
output_url: str | None = None
output_base64: str | None = None
width: int
height: int
filesize: int
mime: str
metadata: dict[str, Any] = Field(default_factory=dict)
model_config = ConfigDict(extra="forbid")

View File

@@ -0,0 +1,12 @@
from __future__ import annotations
from fastapi import HTTPException, Request, status
def verify_bearer_token(request: Request) -> None:
settings = request.app.state.settings
authorization = request.headers.get("Authorization", "")
scheme, _, token = authorization.partition(" ")
if scheme.lower() != "bearer" or token != settings.token:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Unauthorized")

View File

@@ -0,0 +1,36 @@
from __future__ import annotations
from .config import Settings
from .engines.base import EngineHealth, UpscaleEngine, UpscaleEngineUnavailable, UpscaleResult
from .engines.pillow_engine import PillowUpscaleEngine
from .engines.realesrgan_ncnn_engine import RealEsrganNcnnEngine
from .image_io import DownloadedImage
class UnavailableEngine(UpscaleEngine):
def __init__(self, settings: Settings, engine_name: str) -> None:
self.settings = settings
self.engine_name = engine_name
def health(self) -> EngineHealth:
return EngineHealth(
status="degraded",
engine=self.engine_name,
device=self.settings.device,
models_loaded=False,
)
def upscale(self, downloaded: DownloadedImage, scale: int, mode: str, output_format: str) -> UpscaleResult:
raise UpscaleEngineUnavailable("Upscale engine is not available. Check model files and worker installation.")
def build_upscaler(settings: Settings) -> UpscaleEngine:
engine_name = settings.engine.strip().lower()
if engine_name == "pillow":
return PillowUpscaleEngine(settings)
if engine_name in {"realesrgan", "realesrgan-ncnn"}:
return RealEsrganNcnnEngine(settings)
return UnavailableEngine(settings, engine_name or "unknown")

View File

@@ -0,0 +1 @@
keep

View File

@@ -0,0 +1,23 @@
services:
enhance-worker:
build: .
container_name: skinbase-enhance-worker
ports:
- "127.0.0.1:8095:8095"
environment:
WORKER_TOKEN: "change-this-token"
WORKER_ENGINE: "pillow"
WORKER_DEVICE: "cpu"
WORKER_MAX_UPLOAD_MB: "20"
WORKER_MAX_INPUT_WIDTH: "4096"
WORKER_MAX_INPUT_HEIGHT: "4096"
WORKER_MAX_OUTPUT_WIDTH: "8192"
WORKER_MAX_OUTPUT_HEIGHT: "8192"
WORKER_RESULT_TTL_MINUTES: "60"
WORKER_REALESRGAN_OUTPUT_EXT: "webp"
volumes:
- ./storage/tmp:/app/storage/tmp
- ./storage/output:/app/storage/output
- ./bin:/app/bin:ro
- ./models:/app/models:ro
restart: unless-stopped

View File

@@ -0,0 +1,23 @@
services:
enhance-worker:
build: .
container_name: skinbase-enhance-worker-realesrgan
ports:
- "127.0.0.1:8095:8095"
environment:
WORKER_TOKEN: "change-this-token"
WORKER_ENGINE: "realesrgan-ncnn"
WORKER_DEVICE: "cpu"
WORKER_REALESRGAN_GPU_ID: "-1"
WORKER_REALESRGAN_BIN: "/app/bin/realesrgan-ncnn-vulkan"
WORKER_REALESRGAN_MODEL_DIR: "/app/models"
WORKER_REALESRGAN_DEFAULT_MODEL: "realesrgan-x4plus"
WORKER_REALESRGAN_ANIME_MODEL: "realesrgan-x4plus-anime"
WORKER_REALESRGAN_TIMEOUT_SECONDS: "3600"
WORKER_REALESRGAN_OUTPUT_EXT: "webp"
volumes:
- ./storage/tmp:/app/storage/tmp
- ./storage/output:/app/storage/output
- ./bin:/app/bin:ro
- ./models:/app/models:ro
restart: unless-stopped

View File

@@ -0,0 +1 @@
keep

View File

@@ -0,0 +1,5 @@
fastapi==0.115.5
uvicorn[standard]==0.32.1
Pillow==11.0.0
httpx==0.27.2
pytest==8.3.3

View File

@@ -0,0 +1,71 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
BIN_DIR="${ROOT_DIR}/bin"
MODEL_DIR="${ROOT_DIR}/models"
TMP_DIR="${ROOT_DIR}/.tmp"
FORCE=0
RELEASE_VERSION="${REALESRGAN_RELEASE_VERSION:-v0.2.5.0}"
ASSET_NAME="${REALESRGAN_RELEASE_ASSET:-realesrgan-ncnn-vulkan-20220424-ubuntu.zip}"
while [[ $# -gt 0 ]]; do
case "$1" in
--force)
FORCE=1
shift
;;
--version)
RELEASE_VERSION="${2:?--version requires a value}"
shift 2
;;
*)
echo "Unknown argument: $1" >&2
exit 1
;;
esac
done
if [[ "${RELEASE_VERSION}" == "latest" ]]; then
RELEASE_URL="${REALESRGAN_RELEASE_URL:-https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/${ASSET_NAME}}"
else
RELEASE_URL="${REALESRGAN_RELEASE_URL:-https://github.com/xinntao/Real-ESRGAN/releases/download/${RELEASE_VERSION}/${ASSET_NAME}}"
fi
mkdir -p "${BIN_DIR}" "${MODEL_DIR}" "${TMP_DIR}"
if [[ -f "${BIN_DIR}/realesrgan-ncnn-vulkan" && "${FORCE}" -ne 1 ]]; then
echo "Binary already exists at ${BIN_DIR}/realesrgan-ncnn-vulkan. Use --force to replace it."
exit 0
fi
ARCHIVE_PATH="${TMP_DIR}/${ASSET_NAME}"
EXTRACT_DIR="${TMP_DIR}/realesrgan-extract"
rm -rf "${EXTRACT_DIR}"
mkdir -p "${EXTRACT_DIR}"
echo "Downloading ${RELEASE_URL}"
curl -L --fail --output "${ARCHIVE_PATH}" "${RELEASE_URL}"
if [[ ! -f "${ARCHIVE_PATH}" ]]; then
echo "Download failed: archive not found at ${ARCHIVE_PATH}" >&2
exit 1
fi
unzip -o "${ARCHIVE_PATH}" -d "${EXTRACT_DIR}" >/dev/null
BINARY_PATH="$(find "${EXTRACT_DIR}" -type f -name 'realesrgan-ncnn-vulkan' | head -n 1)"
if [[ -z "${BINARY_PATH}" ]]; then
echo "Could not find realesrgan-ncnn-vulkan in the downloaded archive." >&2
exit 1
fi
install -m 0755 "${BINARY_PATH}" "${BIN_DIR}/realesrgan-ncnn-vulkan"
find "${EXTRACT_DIR}" -type f \( -name '*.param' -o -name '*.bin' \) -print0 | while IFS= read -r -d '' model_file; do
install -m 0644 "${model_file}" "${MODEL_DIR}/$(basename "${model_file}")"
done
echo "Installed binary: ${BIN_DIR}/realesrgan-ncnn-vulkan"
echo "Installed models in: ${MODEL_DIR}"

View File

@@ -0,0 +1,9 @@
#!/usr/bin/env bash
set -euo pipefail
# Ensure storage directories are writable by the worker user.
# Bind-mounted host directories may have been created with restrictive
# permissions, so we fix them here before dropping to the worker user.
chmod 777 /app/storage/tmp /app/storage/output 2>/dev/null || true
exec gosu worker "$@"

View File

@@ -0,0 +1,46 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
BIN_PATH="${WORKER_REALESRGAN_BIN:-${ROOT_DIR}/bin/realesrgan-ncnn-vulkan}"
MODEL_DIR="${WORKER_REALESRGAN_MODEL_DIR:-${ROOT_DIR}/models}"
echo "Binary path: ${BIN_PATH}"
echo "Model dir: ${MODEL_DIR}"
if [[ ! -f "${BIN_PATH}" ]]; then
echo "binary missing"
exit 1
fi
if [[ ! -x "${BIN_PATH}" ]]; then
echo "binary exists but is not executable"
exit 1
fi
if [[ ! -d "${MODEL_DIR}" ]]; then
echo "model directory missing"
exit 1
fi
echo "binary exists"
echo "binary executable"
available_models=()
while IFS= read -r param_file; do
stem="$(basename "${param_file}" .param)"
if [[ -f "${MODEL_DIR}/${stem}.bin" ]]; then
available_models+=("${stem}")
fi
done < <(find "${MODEL_DIR}" -maxdepth 1 -type f -name '*.param' | sort)
if [[ ${#available_models[@]} -eq 0 ]]; then
echo "no models found"
else
echo "models listed"
for model in "${available_models[@]}"; do
echo " - ${model}"
done
fi
"${BIN_PATH}" -h >/dev/null 2>&1 || true

View File

@@ -0,0 +1 @@
keep

View File

@@ -0,0 +1 @@
keep

View File

@@ -0,0 +1,39 @@
from pathlib import Path
from fastapi.testclient import TestClient
from app.config import Settings
from app.main import create_app
def make_settings(tmp_path: Path) -> Settings:
return Settings(
host="127.0.0.1",
port=8095,
token="secret-token",
engine="pillow",
device="cpu",
max_upload_mb=20,
max_input_width=4096,
max_input_height=4096,
max_output_width=8192,
max_output_height=8192,
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
result_ttl_minutes=60,
model_dir=str(tmp_path / "models"),
default_model="realesrgan-x4plus",
)
def test_health_returns_ok(tmp_path: Path) -> None:
client = TestClient(create_app(make_settings(tmp_path)))
response = client.get("/health")
assert response.status_code == 200
payload = response.json()
assert payload["status"] == "ok"
assert payload["service"] == "skinbase-enhance-worker"
assert payload["engine"] == "pillow"
assert payload["models_loaded"] is True

View File

@@ -0,0 +1,61 @@
from pathlib import Path
from fastapi.testclient import TestClient
from app.config import Settings
from app.main import create_app
def test_health_reports_degraded_when_realesrgan_binary_is_missing(tmp_path: Path) -> None:
settings = Settings(
engine="realesrgan-ncnn",
device="vulkan",
token="secret-token",
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
realesrgan_bin=str(tmp_path / "bin" / "realesrgan-ncnn-vulkan"),
realesrgan_model_dir=str(tmp_path / "models"),
)
(tmp_path / "models").mkdir(parents=True)
client = TestClient(create_app(settings))
response = client.get("/health")
assert response.status_code == 200
payload = response.json()
assert payload["status"] == "degraded"
assert payload["engine"] == "realesrgan-ncnn"
assert payload["realesrgan"]["binary_exists"] is False
assert payload["realesrgan"]["available_models"] == []
def test_health_reports_available_models_for_realesrgan(tmp_path: Path) -> None:
binary = tmp_path / "bin" / "realesrgan-ncnn-vulkan"
binary.parent.mkdir(parents=True)
binary.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8")
binary.chmod(0o755)
model_dir = tmp_path / "models"
model_dir.mkdir(parents=True)
for name in ("realesrgan-x4plus", "realesrgan-x4plus-anime"):
(model_dir / f"{name}.param").write_text("param", encoding="utf-8")
(model_dir / f"{name}.bin").write_text("bin", encoding="utf-8")
settings = Settings(
engine="realesrgan-ncnn",
device="vulkan",
token="secret-token",
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
realesrgan_bin=str(binary),
realesrgan_model_dir=str(model_dir),
)
client = TestClient(create_app(settings))
response = client.get("/health")
assert response.status_code == 200
payload = response.json()
assert payload["status"] == "ok"
assert payload["models_loaded"] is True
assert payload["realesrgan"]["available_models"] == ["realesrgan-x4plus", "realesrgan-x4plus-anime"]

View File

@@ -0,0 +1,110 @@
from pathlib import Path
from subprocess import CompletedProcess
import pytest
from PIL import Image
from app.config import Settings
from app.engines.base import UpscaleEngineUnavailable
from app.image_io import DownloadedImage
from app.upscaler import build_upscaler
def make_downloaded_image(tmp_path: Path) -> DownloadedImage:
source = tmp_path / "source.png"
Image.new("RGBA", (12, 8), (25, 50, 75, 255)).save(source, "PNG")
return DownloadedImage(
path=source,
width=12,
height=8,
mime="image/png",
filesize=source.stat().st_size,
)
def make_runtime_settings(tmp_path: Path) -> Settings:
binary = tmp_path / "bin" / "realesrgan-ncnn-vulkan"
binary.parent.mkdir(parents=True)
binary.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8")
binary.chmod(0o755)
model_dir = tmp_path / "models"
model_dir.mkdir(parents=True)
(model_dir / "realesrgan-x4plus.param").write_text("param", encoding="utf-8")
(model_dir / "realesrgan-x4plus.bin").write_text("bin", encoding="utf-8")
return Settings(
engine="realesrgan-ncnn",
device="vulkan",
token="secret-token",
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
realesrgan_bin=str(binary),
realesrgan_model_dir=str(model_dir),
realesrgan_anime_model="realesrgan-x4plus-anime",
realesrgan_allow_model_fallback=True,
)
def test_realesrgan_command_is_built_without_shell_and_2x_records_downsample(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
settings = make_runtime_settings(tmp_path)
engine = build_upscaler(settings)
downloaded = make_downloaded_image(tmp_path)
captured: dict[str, object] = {}
def fake_run(command, **kwargs):
captured["command"] = command
captured["kwargs"] = kwargs
output_index = command.index("-o") + 1
output_path = Path(command[output_index])
output_path.parent.mkdir(parents=True, exist_ok=True)
Image.new("RGBA", (downloaded.width * 4, downloaded.height * 4), (120, 80, 20, 255)).save(output_path, "PNG")
return CompletedProcess(command, 0, stdout="ok", stderr="")
monkeypatch.setattr("app.engines.realesrgan_ncnn_engine.subprocess.run", fake_run)
result = engine.upscale(downloaded, 2, "illustration", "webp")
assert result.image.size == (downloaded.width * 2, downloaded.height * 2)
assert result.metadata["requested_scale"] == 2
assert result.metadata["native_model_scale"] == 4
assert result.metadata["post_downsampled"] is True
assert result.metadata["model_fallback"] is True
assert result.metadata["used_model"] == "realesrgan-x4plus"
assert captured["command"][0] == settings.realesrgan_bin
assert captured["kwargs"].get("check") is False
assert captured["kwargs"].get("shell", False) is False
def test_missing_binary_raises_safe_engine_unavailable(tmp_path: Path) -> None:
engine = build_upscaler(
Settings(
engine="realesrgan-ncnn",
token="secret-token",
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
realesrgan_bin=str(tmp_path / "bin" / "missing-binary"),
realesrgan_model_dir=str(tmp_path / "models"),
)
)
with pytest.raises(UpscaleEngineUnavailable, match="Upscale engine is not available"):
engine.upscale(make_downloaded_image(tmp_path), 4, "artwork", "webp")
def test_pillow_engine_still_works(tmp_path: Path) -> None:
engine = build_upscaler(
Settings(
engine="pillow",
token="secret-token",
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
)
)
result = engine.upscale(make_downloaded_image(tmp_path), 2, "standard", "webp")
assert result.metadata["engine"] == "pillow"
assert result.metadata["real_ai_upscale"] is False

View File

@@ -0,0 +1,50 @@
from pathlib import Path
import pytest
from app.config import Settings
from app.engines.base import UpscaleEngineUnavailable
from app.upscaler import build_upscaler
def test_invalid_engine_returns_degraded_health(tmp_path: Path) -> None:
upscaler = build_upscaler(
Settings(
engine="broken-engine",
token="secret-token",
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
)
)
health = upscaler.health()
assert health.status == "degraded"
assert health.models_loaded is False
def test_missing_model_raises_safe_error_when_fallback_disabled(tmp_path: Path) -> None:
binary = tmp_path / "bin" / "realesrgan-ncnn-vulkan"
binary.parent.mkdir(parents=True)
binary.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8")
binary.chmod(0o755)
model_dir = tmp_path / "models"
model_dir.mkdir(parents=True)
(model_dir / "realesrgan-x4plus.param").write_text("param", encoding="utf-8")
(model_dir / "realesrgan-x4plus.bin").write_text("bin", encoding="utf-8")
engine = build_upscaler(
Settings(
engine="realesrgan-ncnn",
token="secret-token",
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
realesrgan_bin=str(binary),
realesrgan_model_dir=str(model_dir),
realesrgan_allow_model_fallback=False,
)
)
with pytest.raises(UpscaleEngineUnavailable, match="Upscale engine is not available"):
engine.resolve_model("illustration")

View File

@@ -0,0 +1,63 @@
from pathlib import Path
from fastapi.testclient import TestClient
from app.config import Settings
from app.main import create_app
def make_settings(tmp_path: Path) -> Settings:
return Settings(
host="127.0.0.1",
port=8095,
token="secret-token",
engine="pillow",
device="cpu",
max_upload_mb=20,
max_input_width=4096,
max_input_height=4096,
max_output_width=8192,
max_output_height=8192,
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
result_ttl_minutes=60,
model_dir=str(tmp_path / "models"),
default_model="realesrgan-x4plus",
)
def test_upscale_requires_bearer_token(tmp_path: Path) -> None:
client = TestClient(create_app(make_settings(tmp_path)))
response = client.post(
"/v1/upscale",
json={
"job_id": 1,
"source_url": "https://example.com/source.webp",
"scale": 2,
"mode": "standard",
"output_format": "webp",
},
)
assert response.status_code == 401
assert response.json()["detail"] == "Unauthorized"
def test_upscale_rejects_invalid_bearer_token(tmp_path: Path) -> None:
client = TestClient(create_app(make_settings(tmp_path)))
response = client.post(
"/v1/upscale",
headers={"Authorization": "Bearer wrong-token"},
json={
"job_id": 1,
"source_url": "https://example.com/source.webp",
"scale": 2,
"mode": "standard",
"output_format": "webp",
},
)
assert response.status_code == 401
assert response.json()["detail"] == "Unauthorized"

View File

@@ -0,0 +1,98 @@
from pathlib import Path
from fastapi.testclient import TestClient
from app.config import Settings
from app.main import create_app
def make_settings(tmp_path: Path) -> Settings:
return Settings(
host="127.0.0.1",
port=8095,
token="secret-token",
engine="pillow",
device="cpu",
max_upload_mb=20,
max_input_width=4096,
max_input_height=4096,
max_output_width=8192,
max_output_height=8192,
tmp_dir=str(tmp_path / "tmp"),
output_dir=str(tmp_path / "output"),
result_ttl_minutes=60,
model_dir=str(tmp_path / "models"),
default_model="realesrgan-x4plus",
)
def test_validation_rejects_invalid_scale(tmp_path: Path) -> None:
client = TestClient(create_app(make_settings(tmp_path)))
response = client.post(
"/v1/upscale",
headers={"Authorization": "Bearer secret-token"},
json={
"job_id": 1,
"source_url": "https://example.com/source.webp",
"scale": 3,
"mode": "standard",
"output_format": "webp",
},
)
assert response.status_code == 422
def test_validation_rejects_invalid_mode(tmp_path: Path) -> None:
client = TestClient(create_app(make_settings(tmp_path)))
response = client.post(
"/v1/upscale",
headers={"Authorization": "Bearer secret-token"},
json={
"job_id": 1,
"source_url": "https://example.com/source.webp",
"scale": 2,
"mode": "broken",
"output_format": "webp",
},
)
assert response.status_code == 422
def test_validation_rejects_invalid_output_format(tmp_path: Path) -> None:
client = TestClient(create_app(make_settings(tmp_path)))
response = client.post(
"/v1/upscale",
headers={"Authorization": "Bearer secret-token"},
json={
"job_id": 1,
"source_url": "https://example.com/source.webp",
"scale": 2,
"mode": "standard",
"output_format": "gif",
},
)
assert response.status_code == 422
def test_validation_rejects_local_file_source_url(tmp_path: Path) -> None:
client = TestClient(create_app(make_settings(tmp_path)))
response = client.post(
"/v1/upscale",
headers={"Authorization": "Bearer secret-token"},
json={
"job_id": 1,
"source_url": "file:///tmp/source.webp",
"scale": 2,
"mode": "standard",
"output_format": "webp",
},
)
assert response.status_code == 422