Ship production optimization M1-M12.5A: queues, metrics, HTTP observability, and vector search reliability.

Keep similar-ai from tripping the global circuit on a lone URL 502, clamp Qdrant search to 100, and add Server-Timing plus slow-request logging. Studio shared props, Academy S3 exists caching, heat chunking, and Redis/scheduler hygiene stay in this rollout.
This commit is contained in:
2026-08-25 07:58:47 +02:00
parent f52879edbb
commit 8a80aae21e
114 changed files with 9293 additions and 284 deletions
+79
View File
@@ -0,0 +1,79 @@
# M2 — Mail Queue Consumption
## Root cause
Application mail jobs are pushed to Redis queue **`mail`**. Production Horizon supervisors only consume:
```text
search, default
broadcasts, notifications
```
There is **no** `queue:work` process for `mail`. The three items have `attempts=0` — they were never reserved.
Isolation is **intentional** in code (`onQueue('mail')`) and in `docs/QUEUE.md`. Horizon simply never subscribed.
## The three queued items (no PII)
Inspected with `LRANGE`/`LINDEX` via Laravel Redis; only class metadata:
| # | displayName | maxTries | attempts |
| - | ----------- | -------: | -------: |
| 0 | `App\Jobs\SendVerificationEmailJob` | 5 | 0 |
| 1 | `App\Mail\EmailChangedSecurityAlertMail` | 3 | 0 |
| 2 | `App\Mail\EmailChangeVerificationCodeMail` | 3 | 0 |
`pushedAt` unix times: 1777665006, 1777697989, 1784800925 (older registration/email-change mail sitting until a worker exists).
## Local vs production
Same gap on both: Horizon defaults omitted `mail`. Production `config('horizon.defaults.supervisor-*.queue')` confirmed `["search","default"]` and `["broadcasts","notifications"]`.
Legacy `deploy/supervisor/skinbase-queue.conf` **does** include `mail`, but production runs Horizon, not that unit.
## Fix
Added Horizon `supervisor-mail`:
```text
queue: mail
connection: redis
tries: 5 (matches SendVerificationEmailJob; rate-limit uses release())
timeout: 90
maxProcesses: 1 local / 2 production
```
Kept isolation so SMTP does not block rec/search workers and so `--tries=1` on default workers cannot fail mail jobs on `release()`.
## Files changed
```text
config/horizon.php
tests/Unit/HorizonMailQueueTest.php
docs/QUEUE.md
docs/optimization-m2-mail-queue.md
```
## Deployment
1. Deploy release with `config/horizon.php`.
2. `php artisan config:cache` / `optimize` as usual.
3. Restart Horizon (Supervisor `skinbase-horizon` / `queue:restart`).
4. Confirm `php artisan horizon:status` and a worker with `--queue=mail`.
No `.env` change required. No Redis delete.
## Should the 3 messages be processed after deploy?
**Yes, automatically** once `supervisor-mail` starts — do not delete the list.
Risks:
- Recipients may receive **late** verification/security emails.
- Tokens/codes may already be expired; user sees a failed verify, not a security hole from us sending mail.
- `SendVerificationEmailJob` may still send `RegistrationVerificationMail` (also `mail` queue) — second job is fine once workers run.
- Do **not** manually `queue:retry` or `lpop` before Horizon is up.
## Tests
`php artisan test --filter=HorizonMailQueue`