klevze/SkinbaseNova
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5af95f6533a129e6c22f051c632ab821240e8317
SkinbaseNova/docs/enhance-setup.md

14 KiB
Raw Blame History

Skinbase Enhance Setup Guide

This guide explains how to set up, enable, verify, and operate the Skinbase Enhance module end to end.

Use this document when you need to:

  • enable Enhance in a local environment
  • switch from stub mode to the external worker
  • run the worker in Pillow mode or Real-ESRGAN mode
  • configure queues, cleanup, and health checks
  • understand what the module stores and how it behaves in production

What the module does

Enhance accepts an uploaded or selected source image, creates an Enhance job, processes that job through the configured engine, stores the generated output on the Enhance storage disk, and keeps the original source file untouched.

Current supported engines on the Laravel side:

  • ENHANCE_ENGINE=stub
  • ENHANCE_ENGINE=external_worker

Current supported worker engines:

  • WORKER_ENGINE=pillow
  • WORKER_ENGINE=realesrgan-ncnn
  • WORKER_ENGINE=realesrgan

WORKER_ENGINE=realesrgan currently aliases to realesrgan-ncnn.

Architecture

The Enhance module is split into two layers.

Laravel application:

  • accepts the Enhance request
  • validates allowed file types, dimensions, scales, and modes
  • stores job records
  • dispatches the job to the queue
  • owns permanent storage for Enhance sources and outputs
  • exposes moderation, cleanup, and health commands

Optional external worker:

  • downloads the copied Enhance source image
  • upscales it with Pillow or Real-ESRGAN
  • exposes a temporary internal result URL
  • deletes its temporary result after Laravel confirms download

Laravel remains the source of truth. The worker is temporary processing only.

Default behavior and limits

Default config values from config/enhance.php:

  • disk: ENHANCE_DISK or the app filesystem default
  • source prefix: enhance/sources
  • output prefix: enhance/outputs
  • preview prefix: enhance/previews
  • default engine: stub
  • allowed MIME types: image/jpeg, image/png, image/webp
  • allowed modes: standard, artwork, photo, illustration
  • allowed scales: 2, 4
  • daily limit: 10
  • default queue: default

Lifecycle defaults:

  • completed jobs expire after 30 days
  • failed jobs expire after 7 days
  • deleted job files get a 1 day grace period

Health defaults:

  • processing jobs are considered stuck after 30 minutes
  • queued jobs are considered stale after 60 minutes

Prerequisites

Laravel requirements:

  • the application boots normally
  • database migrations are current
  • the configured filesystem disk is writable
  • a queue worker or Horizon is running for the queue you assign to Enhance
  • Laravel scheduler is enabled if you want automatic cleanup

Worker requirements for external_worker mode:

  • Docker or a Python runtime for services/enhance-worker
  • network access from Laravel to the worker URL
  • a shared bearer token between Laravel and the worker
  • for Real-ESRGAN mode, the realesrgan-ncnn-vulkan binary and model files

Setup paths

There are three practical ways to run Enhance.

  1. Stub mode
  • safest local starting point
  • exercises the Laravel flow without a real AI runtime
  • no worker needed
  1. External worker with Pillow
  • good for local integration testing and CI-like validation
  • real HTTP worker contract
  • deterministic fallback upscale path
  • no Real-ESRGAN runtime files required
  1. External worker with Real-ESRGAN
  • production-oriented path
  • uses the realesrgan-ncnn-vulkan CLI runtime
  • requires runtime files and a verified worker host

Laravel setup

Minimum env for stub mode:

ENHANCE_ENGINE=stub
ENHANCE_QUEUE=default

Recommended env for external worker mode:

ENHANCE_ENGINE=external_worker
ENHANCE_QUEUE=enhance
ENHANCE_WORKER_URL=http://127.0.0.1:8095
ENHANCE_WORKER_TIMEOUT=900
ENHANCE_WORKER_TOKEN=change-this-token
ENHANCE_WORKER_MAX_DOWNLOAD_MB=60

Optional Laravel env keys you may tune:

ENHANCE_DISK=public
ENHANCE_SOURCE_PREFIX=enhance/sources
ENHANCE_OUTPUT_PREFIX=enhance/outputs
ENHANCE_PREVIEW_PREFIX=enhance/previews

ENHANCE_MAX_UPLOAD_MB=20
ENHANCE_MAX_INPUT_WIDTH=4096
ENHANCE_MAX_INPUT_HEIGHT=4096
ENHANCE_MAX_OUTPUT_WIDTH=8192
ENHANCE_MAX_OUTPUT_HEIGHT=8192

ENHANCE_DAILY_LIMIT=10

ENHANCE_COMPLETED_EXPIRES_AFTER_DAYS=30
ENHANCE_FAILED_EXPIRES_AFTER_DAYS=7
ENHANCE_DELETED_FILE_GRACE_DAYS=1
ENHANCE_CLEANUP_CHUNK_SIZE=100

ENHANCE_STUCK_PROCESSING_AFTER_MINUTES=30
ENHANCE_STUCK_QUEUED_AFTER_MINUTES=60

ENHANCE_STUB_SHOW_WARNING=true

After changing env:

php artisan config:clear

Queue setup

Enhance jobs run on ENHANCE_QUEUE.

If you keep the default queue:

php artisan queue:work --queue=default

If you use a dedicated Enhance queue:

php artisan queue:work --queue=enhance,default

If Horizon or workers do not consume the configured queue, Enhance jobs will stay queued.

Scheduler and cleanup

Enhance cleanup is scheduled from routes/console.php and runs:

php artisan enhance:cleanup --force

Useful cleanup and health commands:

php artisan enhance:health
php artisan enhance:health --json
php artisan enhance:cleanup --dry-run
php artisan enhance:cleanup --force

Cleanup only removes files under the configured Enhance prefixes. It does not delete artwork originals or unrelated storage paths.

Enable stub mode

Use stub mode first if you want to validate the Laravel module without introducing worker runtime variables.

  1. Set:
ENHANCE_ENGINE=stub
ENHANCE_QUEUE=default
  1. Clear config:
php artisan config:clear
  1. Start a queue worker:
php artisan queue:work --queue=default
  1. Check health:
php artisan enhance:health
  1. Open /enhance/create, submit a small image, and verify the job completes.

Enable external worker mode with Pillow

This is the safest real integration path because it exercises the HTTP worker contract without requiring Real-ESRGAN files.

  1. Start the worker:
cd services/enhance-worker
docker compose -f docker-compose.example.yml up --build
  1. Confirm worker health:
curl http://127.0.0.1:8095/health

Expected result:

  • status: ok
  • engine: pillow
  1. Set Laravel env:
ENHANCE_ENGINE=external_worker
ENHANCE_QUEUE=enhance
ENHANCE_WORKER_URL=http://127.0.0.1:8095
ENHANCE_WORKER_TIMEOUT=600
ENHANCE_WORKER_TOKEN=change-this-token
ENHANCE_WORKER_MAX_DOWNLOAD_MB=60
  1. Clear config and start queue workers:
php artisan config:clear
php artisan queue:work --queue=enhance,default
  1. Verify the Laravel side:
php artisan enhance:health
php artisan test --filter=EnhanceExternalWorker

Enable external worker mode with Real-ESRGAN

This is the production-oriented setup.

  1. Install or mount the runtime files inside services/enhance-worker:
cd services/enhance-worker
bash scripts/download-realesrgan-ncnn.sh
bash scripts/verify-realesrgan.sh

Required file locations:

  • binary: bin/realesrgan-ncnn-vulkan
  • models: models/*.param and models/*.bin
  1. Start the Real-ESRGAN worker:
docker compose -f docker-compose.realesrgan.example.yml up --build
  1. Confirm worker health:
curl http://127.0.0.1:8095/health

Expected result when ready:

  • status: ok
  • engine: realesrgan-ncnn

Expected result when runtime files are missing or invalid:

  • status: degraded
  1. Set Laravel env:
ENHANCE_ENGINE=external_worker
ENHANCE_QUEUE=enhance
ENHANCE_WORKER_URL=http://127.0.0.1:8095
ENHANCE_WORKER_TIMEOUT=900
ENHANCE_WORKER_TOKEN=change-this-token
ENHANCE_WORKER_MAX_DOWNLOAD_MB=60
  1. Clear config and start the queue worker:
php artisan config:clear
php artisan queue:work --queue=enhance,default
  1. Verify through the application by submitting a small image from /enhance/create.

Worker configuration

Main worker env values:

WORKER_HOST=0.0.0.0
WORKER_PORT=8095
WORKER_TOKEN=change-this-token

WORKER_ENGINE=pillow
WORKER_DEVICE=cpu

WORKER_MAX_UPLOAD_MB=20
WORKER_MAX_INPUT_WIDTH=4096
WORKER_MAX_INPUT_HEIGHT=4096
WORKER_MAX_OUTPUT_WIDTH=8192
WORKER_MAX_OUTPUT_HEIGHT=8192

WORKER_TMP_DIR=/app/storage/tmp
WORKER_OUTPUT_DIR=/app/storage/output
WORKER_RESULT_TTL_MINUTES=60

Real-ESRGAN-specific worker env values:

WORKER_REALESRGAN_BIN=/app/bin/realesrgan-ncnn-vulkan
WORKER_REALESRGAN_MODEL_DIR=/app/models
WORKER_REALESRGAN_DEFAULT_MODEL=realesrgan-x4plus
WORKER_REALESRGAN_ANIME_MODEL=realesrgan-x4plus-anime
WORKER_REALESRGAN_TILE=0
WORKER_REALESRGAN_TTA=false
WORKER_REALESRGAN_VERBOSE=false
WORKER_REALESRGAN_TIMEOUT_SECONDS=900
WORKER_REALESRGAN_PREPROCESS_MAX_PIXELS=16777216
WORKER_REALESRGAN_OUTPUT_EXT=webp
WORKER_REALESRGAN_ALLOW_MODEL_FALLBACK=true

Compatibility values kept by the worker:

WORKER_MODEL_DIR=/app/app/models
WORKER_DEFAULT_MODEL=realesrgan-x4plus

Worker behavior

Worker request contract:

  • endpoint: POST /v1/upscale
  • bearer auth required
  • accepted output formats: webp, png, jpg
  • allowed scales: 2, 4
  • allowed modes: standard, artwork, photo, illustration

Health and temp file endpoints:

  • GET /health
  • GET /v1/results/{filename}
  • DELETE /v1/results/{filename}

Mode and scale behavior

Real-ESRGAN mode mapping:

  • standard -> default model
  • artwork -> default model
  • photo -> default model
  • illustration -> anime model when available

Fallback behavior:

  • if the requested model exists, it is used
  • if it is missing and fallback is enabled, the default model is used
  • if it is missing and fallback is disabled, processing fails safely

Scale behavior:

  • 4x returns native 4x output
  • 2x currently runs the 4x model and then downsamples to 2x

Storage and data flow

Laravel stores Enhance files under the configured prefixes:

  • sources under enhance/sources
  • outputs under enhance/outputs
  • previews under enhance/previews

The worker does not permanently store Enhance results.

When ENHANCE_ENGINE=external_worker:

  1. Laravel prepares a copied Enhance source file.
  2. Laravel sends the worker a temporary URL or a temporary signed internal route.
  3. The worker downloads the source file.
  4. The worker processes the image.
  5. The worker exposes a temporary result URL.
  6. Laravel downloads, validates, and stores the final output.
  7. Laravel instructs the worker to delete the temporary result.

Verification checklist

Laravel verification:

php artisan config:clear
php artisan enhance:health
php artisan enhance:health --json

Worker verification:

curl http://127.0.0.1:8095/health

Queue verification:

php artisan queue:work --queue=enhance,default

Application verification:

  1. Open /enhance/create.
  2. Upload or choose a small source image.
  3. Select 2x or 4x and a valid mode.
  4. Submit the job.
  5. Confirm the job transitions from queued to completed.
  6. Confirm output exists on the Enhance disk.
  7. Confirm the source file and original artwork remain untouched.

Health states

Laravel health command helps identify:

  • configured engine
  • queue usage
  • stuck jobs
  • lifecycle status

Worker /health helps identify:

  • current worker engine
  • maximum input and output limits
  • Real-ESRGAN binary availability
  • Real-ESRGAN model directory readiness
  • available Real-ESRGAN models

If the worker is in Real-ESRGAN mode and health returns degraded, do not treat the runtime as production-ready yet.

Troubleshooting

Worker URL is missing.

  • set ENHANCE_WORKER_URL
  • clear config

Worker token is missing.

  • set ENHANCE_WORKER_TOKEN
  • make sure the worker uses the same WORKER_TOKEN

Worker is unavailable.

  • confirm the worker is reachable
  • confirm the worker container is running
  • confirm the URL points to the worker base URL

Upscale engine is not available. Check model files and worker installation.

  • confirm WORKER_ENGINE=realesrgan-ncnn
  • confirm bin/realesrgan-ncnn-vulkan exists and is executable
  • confirm the required .param and .bin model files exist
  • run bash scripts/verify-realesrgan.sh

Jobs stay queued

  • confirm queue workers consume ENHANCE_QUEUE
  • if using enhance, run workers with --queue=enhance,default

status: degraded from worker health

  • verify binary and model directory paths
  • verify runtime files are mounted inside the container
  • fall back to WORKER_ENGINE=pillow until the runtime is fixed

Operations and safety notes

  • Keep the worker bound to 127.0.0.1 or a private container network.
  • Do not expose the worker publicly.
  • Use a strong shared token.
  • Keep Enhance on a dedicated queue when load increases.
  • Keep cleanup enabled so stale outputs and failed files do not accumulate.
  • Do not commit Real-ESRGAN binary or model weight files unless explicitly approved.
  • The worker only serves generated files from its own output directory.
  • The worker rejects unsupported source URLs and unsafe output paths.
  • Original artwork files are never replaced by the Enhance flow.

Production rollout recommendation

Recommended rollout sequence:

  1. Enable ENHANCE_ENGINE=stub and verify the Laravel workflow.
  2. Move to ENHANCE_ENGINE=external_worker with WORKER_ENGINE=pillow.
  3. Verify queue, storage, cleanup, and health behavior.
  4. Install Real-ESRGAN runtime files and switch the worker to WORKER_ENGINE=realesrgan-ncnn.
  5. Confirm worker health is ok before calling the runtime production-ready.

Related docs

Reference in New Issue View Git Blame Copy Permalink