Fix Windows/WSL production deploy hang and permissions.

Skip slow Git worktree walks on /mnt, run Vite on Windows npm, reuse Windows SSH keys, and stop rewriting the klevze-owned public app symlink that skinbase cannot replace.
This commit is contained in:
2026-08-29 12:25:50 +02:00
parent bf9ca12469
commit ab8fa6d845
12 changed files with 1557 additions and 140 deletions
+3 -1
View File
@@ -20,7 +20,9 @@ Examples below are representative. For the full option list of any Artisan comma
| Entry point | Why it is used | Example |
| --- | --- | --- |
| `php artisan` | Main Laravel CLI for all custom app commands listed below | `php artisan list --raw` |
| `bash sync.sh` | Main production deploy wrapper; delegates to the safe release-based production deploy script | `bash sync.sh` |
| `deploy.cmd` / `deploy.bat` | Windows Command Prompt production deploy entrypoint; switches into WSL and runs `deploy.sh` | `deploy.cmd` |
| `bash deploy.sh` | Canonical bash production deploy entrypoint; stages versioned releases and switches `current` | `bash deploy.sh` |
| `bash sync.sh` | Legacy alias for `bash deploy.sh` | `bash sync.sh` |
| `bash sync_dev.sh` | Push the development environment to the configured remote dev host | `bash sync_dev.sh` |
## Maintained Standalone Scripts
+77 -39
View File
@@ -1,18 +1,25 @@
# Deployment
This repository uses a Bash-based production deploy flow.
This repository uses a Bash-based production deploy flow with a Windows Command Prompt wrapper.
## Normal deploy
Run the existing entrypoint:
Preferred entrypoints:
```bash
bash sync.sh
```bat
deploy.cmd
```
`bash sync.sh` delegates to the safe production deploy wrapper, which stages each deploy into a versioned release directory and switches traffic by updating the server-side `current` symlink.
```bash
bash deploy.sh
```
If you launch `bash sync.sh` from WSL against this Windows checkout, the script will automatically run the frontend build with `npm.cmd` on Windows so Rollup/Vite use the correct optional native package set.
`deploy.cmd` is the Windows Command Prompt entrypoint. It switches into WSL and runs `deploy.sh`.
`deploy.sh` is the canonical bash entrypoint and delegates to the safe production deploy wrapper, which stages each deploy into a versioned release directory and switches traffic by updating the server-side `current` symlink.
`bash sync.sh` remains as a legacy alias for the same flow.
`deploy.cmd` runs `npm.cmd run build` on Windows first, then enters WSL for rsync/ssh. That is required when the Ubuntu distro cannot execute Windows `.exe` files (`Exec format error` on `powershell.exe`). If WSL interop does work, `bash deploy.sh` can still launch `npm.cmd` through PowerShell. Local Linux `php`/`composer` are not required for a normal deploy; Artisan and Composer run on the production server. `--with-tests` uses WSL `php` when present, otherwise Windows `php.exe`.
This will:
@@ -28,7 +35,16 @@ This will:
This is now the low-downtime default path for normal code and feature deploys.
Each deploy generates a release ID automatically from UTC time and the local Git revision. Releases are retained under `REMOTE_RELEASE_ROOT/releases/<release-id>`, and production switches between them on the server by updating `REMOTE_RELEASE_ROOT/current`. The public/runtime path stays fixed at `REMOTE_FOLDER`, which is now treated as a stable symlink to the active release.
Each deploy generates:
- a monotonic local **build number** (stored in `var/deploy/build-number`)
- a **release ID** from UTC time + build number + Git revision, for example `20260829-141522-b42-a1b2c3d`
- local deploy history under `var/deploy/` (`latest.json`, `history.jsonl`, and per-run logs in `var/deploy/logs/`)
- remote metadata under `REMOTE_RELEASE_ROOT/deployments/<release-id>.json` and `current-release.json`
Console output includes step progress with elapsed time, rsync transfer progress, and a final duration summary. Override the build number with `--build-number N` or `BUILD_NUMBER=N` when needed. Dry-runs preview the next build number without consuming it.
Releases are retained under `REMOTE_RELEASE_ROOT/releases/<release-id>`, and production switches between them on the server by updating `REMOTE_RELEASE_ROOT/current`. The public/runtime path stays fixed at `REMOTE_FOLDER`, which is now treated as a stable symlink to the active release.
On the first deploy with this layout, the existing live folder is adopted into the release archive automatically and `REMOTE_FOLDER` is converted into that stable symlink path. After that, switching back to an older release does not require any local re-upload.
@@ -36,8 +52,12 @@ On the first deploy with this layout, the existing live folder is adopted into t
Use a full upgrade when the release also needs broad Meilisearch work or non-code service operations.
```bat
deploy.cmd --full-upgrade
```
```bash
bash sync.sh --full-upgrade
bash deploy.sh --full-upgrade
```
Full-upgrade mode:
@@ -49,7 +69,7 @@ Full-upgrade mode:
Example with service hooks:
```bash
bash sync.sh --full-upgrade \
bash deploy.sh --full-upgrade \
--upgrade-pre-hook='sudo systemctl stop reverb' \
--upgrade-post-hook='sudo systemctl restart reverb meilisearch'
```
@@ -59,39 +79,52 @@ You can also provide those hooks through environment variables instead of CLI fl
```bash
FULL_UPGRADE_PRE_HOOK='sudo systemctl stop reverb' \
FULL_UPGRADE_POST_HOOK='sudo systemctl restart reverb meilisearch' \
bash sync.sh --full-upgrade
bash deploy.sh --full-upgrade
```
## Deploy options
```bat
deploy.cmd --skip-build
deploy.cmd --skip-migrate
deploy.cmd --no-maintenance
deploy.cmd --mode=full-upgrade
deploy.cmd --keep-releases=8
deploy.cmd --release-id=release-2026-04-25
deploy.cmd --build-number=100
deploy.cmd --no-rsync-progress
```
```bash
bash sync.sh --skip-build
bash sync.sh --skip-migrate
bash sync.sh --no-maintenance
bash sync.sh --mode=full-upgrade
bash sync.sh --keep-releases=8
bash sync.sh --release-id=release-2026-04-25
bash deploy.sh --skip-build
bash deploy.sh --skip-migrate
bash deploy.sh --no-maintenance
bash deploy.sh --mode=full-upgrade
bash deploy.sh --keep-releases=8
bash deploy.sh --release-id=release-2026-04-25
bash deploy.sh --build-number=100
bash deploy.sh --no-rsync-progress
```
Environment overrides:
```bash
REMOTE_SERVER=user@example.com REMOTE_FOLDER=/var/www/app bash sync.sh
REMOTE_RELEASE_ROOT=/var/www/app.releases RELEASE_RETENTION=8 bash sync.sh
REMOTE_SERVER=user@example.com REMOTE_FOLDER=/var/www/app bash deploy.sh
REMOTE_RELEASE_ROOT=/var/www/app.releases RELEASE_RETENTION=8 bash deploy.sh
```
You can also override the local build command explicitly:
```bash
LOCAL_BUILD_COMMAND='npm run build' bash sync.sh
LOCAL_BUILD_COMMAND='pnpm build' bash sync.sh
LOCAL_BUILD_COMMAND='npm run build' bash deploy.sh
LOCAL_BUILD_COMMAND='pnpm build' bash deploy.sh
```
Upgrade hooks can also be supplied via environment variables:
```bash
FULL_UPGRADE_PRE_HOOK='sudo systemctl stop reverb' bash sync.sh --full-upgrade
FULL_UPGRADE_POST_HOOK='sudo systemctl restart reverb meilisearch' bash sync.sh --full-upgrade
FULL_UPGRADE_PRE_HOOK='sudo systemctl stop reverb' bash deploy.sh --full-upgrade
FULL_UPGRADE_POST_HOOK='sudo systemctl restart reverb meilisearch' bash deploy.sh --full-upgrade
```
## Rollback and release history
@@ -126,26 +159,26 @@ Operational notes:
- Each retained release already contains its own code and vendor tree, so rollback is primarily a symlink switch plus cache refresh and `queue:restart`.
- Rollback does not reverse database migrations. If a release includes incompatible schema changes, handle the database separately.
- Release retention defaults to 5 releases and can be changed with `--keep-releases` or `RELEASE_RETENTION`.
- Release data lives outside the active app path by default at `REMOTE_FOLDER.releases`, so switching releases happens entirely on the production server.
## Replace production database from local
This is intentionally separate from a normal deploy because it overwrites production data.
```bash
bash scripts/push-db-to-prod.sh --force
```
Or combine it with deploy:
```bash
bash sync.sh --with-db-from=local
- Reldeploy.sh --with-db-from=local
```
When run interactively, the deploy script will ask you to confirm the exact remote server and type a confirmation phrase before replacing production data.
For non-interactive use, pass both confirmations explicitly:
```bash
bash deploy.sh --with-db-from=local \
--confirm-db-sync-target=klevze@server3.klevze.si \
--confirm-db-sync-phrase='replace production db from local'
```
Legacy compatibility still exists for:
```bash
bash deployinteractively, the deploy script will ask you to confirm the exact remote server and type a confirmation phrase before replacing production data.
For non-interactive use, pass both confirmations explicitly:
```bash
bash sync.sh --with-db-from=local \
--confirm-db-sync-target=klevze@server3.klevze.si \
@@ -179,11 +212,16 @@ LOCAL_MYSQLDUMP_COMMAND='mysqldump --host=10.0.0.5 --port=3306 --user=app dbname
## Safety notes
- Normal deployments should use `bash sync.sh` without `--with-db`.
- Use `bash sync.sh --full-upgrade` only when the release also includes Meilisearch-wide refreshes or remote service changes.
- Normal deployments should use `deploy.cmd` or `bash deploy.sh` without `--with-db`.
- Use `--full-upgrade` only when the release also includes Meilisearch-wide refreshes or remote service changes.
- Use database replacement only for first-time bootstrap, staging, or an intentional full production reset.
- Use `bash scripts/rollback-production.sh --previous` for a fast server-side release switch when the last deploy needs to be reverted.
- Route caching now runs through `php artisan optimize` in deploy automation; if that starts failing again, fix the route definitions instead of dropping route caching from deploy.
- On Windows, prefer `deploy.cmd` so the process always enters WSL before rsync/ssh/php. It defaults to the local `Ubuntu` WSL distro. The production server itself runs Debian 13; that is separate from the local WSL choice. Override with `DEPLOY_WSL_DISTRO` only if needed. If WSL has no usable `~/.ssh` keys (common when the distro runs as root and keys live in `C:\Users\<you>\.ssh`), the script copies those Windows identities into a 0600 temp dir and uses them for BatchMode SSH. Override the source with `WINDOWS_SSH_DIR`.
- SSH still authenticates as `REMOTE_SERVER` (for example `klevze@host`), but remote rsync/composer/artisan/release work runs as `REMOTE_APP_USER` (default `skinbase`) via `sudo -n -u skinbase`. This keeps release files and runtime dirs owned by the app user and avoids prune blockers like `owner=skinbase mode=2700 .config`. Set `REMOTE_APP_USER=-` only to disable that and run as the SSH login user.
- Remote app-user shells start in `/tmp` (not the SSH user's home). The stable `REMOTE_FOLDER` symlink is left alone once it already points at `.../current`; only `current` is rewritten each deploy. That avoids `Permission denied` on root-owned parents like `/opt/www/virtual`.
- The deploy script refuses missing Vite/SSR build artifacts by default (`REQUIRE_BUILD_MANIFEST=1`), verifies SSH non-interactively first, and takes a local deploy lock so two overlapping deploys do not race.
- Inertia SSR restart is owned exclusively by Supervisor program `skinbase-ssr` (`deploy/supervisor/skinbase-ssr.conf`). The remote application phase runs as `skinbase`, while the restart is issued separately over the SSH deployment session as the privileged login account with `sudo -n /usr/bin/supervisorctl`. The deploy fails if privileged Supervisor access or the configured program is unavailable, and verifies that the program reaches `RUNNING`; it never starts or stops SSR through Artisan.
## Nginx upstream error pages
@@ -202,4 +240,4 @@ On the current `skinbase.org` vhost, the required FastCGI locations are:
- `location ^~ /api/uploads/`
- `location = /index.php`
This intentionally intercepts only `502` and `504`, so Laravel remains responsible for normal `404`, `419`, `429`, `500`, and `503` rendering when the application is actually running.
This intentionally intercepts only `502` and `504`, so Laravel remains responsible for normal `404`, `419`, `429`, `500`, and `503` rendering when the application is actually running.