Ship production optimization M1-M12.5A: queues, metrics, HTTP observability, and vector search reliability.

Keep similar-ai from tripping the global circuit on a lone URL 502, clamp Qdrant search to 100, and add Server-Timing plus slow-request logging. Studio shared props, Academy S3 exists caching, heat chunking, and Redis/scheduler hygiene stay in this rollout.
2026-08-25 07:58:47 +02:00
parent f52879edbb
commit 8a80aae21e
114 changed files with 9293 additions and 284 deletions
@@ -0,0 +1,127 @@
+# M3 — Sitemap Production Audit & Reliability
+
+## Current architecture
+
+Routes:
+
+```text
+GET /sitemap.xml              SitemapController@index
+GET /sitemaps/{name}.xml      SitemapController@show
+GET /robots.txt               RobotsTxtController  (Sitemap: {APP_URL}/sitemap.xml)
+```
+
+Generation: `php artisan skinbase:sitemaps:generate` (scheduler 10:30 and 22:30) writes static XML on `sitemaps_public` (`public/`).
+
+Also: `sitemaps:publish`, `validate`, release artifacts, optional live-build fallback.
+
+Families (no groups/years/videos sitemaps):
+
+artworks (sharded 10k), academy-*, users (sharded 10k), tags (now shardable at 25k), categories, collections, cards, stories, web-stories, news, news-google, forum-*, static-pages.
+
+Canonical artwork URLs: `route('art.show')` `/art/{id}/{slug}`. Public/published only.
+
+## Exact production issue
+
+`/sitemap.xml` is a **stale file from 2026-05-13**, owned by **klevze** (`-rw-r--r--`). Children (artworks shards, academy, users, …) refresh daily as **skinbase**. Scheduler cannot overwrite the index (`filesystems.sitemaps_public.throw=false`, generate ignored failed `put`). Nginx `try_files $uri` serves the stale root file and **never** hits Laravel. Academy families exist on disk but are **absent from the May index**.
+
+`location @php` is missing in the live vhost (documented in M0).
+
+## Local vs production
+
+Generate command already writes `sitemaps/sitemap.xml` on both. Production **file ownership** plus **nginx `$uri` = public/sitemap.xml** is the operational failure. Local did not fail writes or serve a dual path.
+
+## Root cause
+
+1. Index file not writable by `skinbase`.
+2. `put()` failures ignored.
+3. Nginx prefers the stale `/sitemap.xml` over `/sitemaps/index.xml` (which did not exist).
+
+## Implementation
+
+- Generate writes `sitemaps/index.xml` (new, creatable), plus tries `sitemaps/sitemap.xml` and `sitemap.xml`; **fails the command if none of the index paths write**.
+- Child writes report failure instead of counting a silent skip as success.
+- `SitemapController::index()` prefers static `sitemaps/index.xml`.
+- `deploy/nginx/sitemaps.conf`: `try_files /sitemaps/index.xml /sitemaps/sitemap.xml /index.php?$query_string;` (no `@php`)
+- Tags builder is shardable (25k) so it cannot exceed 50k URLs without an index.
+
+## Caching
+
+Unchanged: nginx `max-age=21600`; Laravel file response uses `sitemaps.cache_ttl_seconds` (900). No HTML cache. Generate is the refresh.
+
+## Chunking
+
+Artworks/users/forum-threads/collections/cards/stories already sharded at 10k. Tags 25k. Protocol: 50k URLs / 50MB. Production files are all well under 50MB (largest artworks shard ~3.5MB, tags ~2MB).
+
+## Deployment
+
+Production vhost (`/etc/nginx/sites-enabled/skinbase.org.conf`) has **no** `location @php`. PHP-FPM is:
+
+```nginx
+location / {
+    try_files $uri $uri/ /index.php?$query_string;
+}
+location = /index.php {
+    fastcgi_pass unix:/run/php/php8.4-fpm-skinbase.sock;
+    ...
+}
+```
+
+**Do not** use `try_files … @php` (undefined named location). Use `/index.php?$query_string` like `location /`.
+
+Exact replacement for the two sitemap blocks (currently ~lines 91–110):
+
+```nginx
+    location = /sitemap.xml {
+        try_files /sitemaps/index.xml /sitemaps/sitemap.xml /index.php?$query_string;
+
+        add_header Cache-Control "public, max-age=21600" always;
+        add_header Content-Type  "application/xml; charset=UTF-8" always;
+        etag on;
+    }
+
+    location ~ ^/sitemaps/[A-Za-z0-9_\-]++\.xml$ {
+        try_files $uri /index.php?$query_string;
+
+        add_header Cache-Control "public, max-age=21600" always;
+        add_header Content-Type  "application/xml; charset=UTF-8" always;
+        etag on;
+    }
+```
+
+Do **not** add `$uri` to the `/sitemap.xml` try_files list: `$uri` is `/sitemap.xml`, the stale klevze-owned file, and would shadow PHP if `index.xml` is missing.
+
+Order:
+
+1. Deploy app.
+2. Run `php artisan skinbase:sitemaps:generate` as `skinbase` so `sitemaps/index.xml` exists.
+3. Patch the vhost, then:
+
+```bash
+sudo nginx -t && sudo systemctl reload nginx
+```
+
+`nginx -t` should pass: last `try_files` argument matches the existing front-controller pattern. Introducing `@php` would not.
+
+Optional: `chown skinbase:skinbase` the old `sitemap.xml`.
+
+## Legacy index files after `sitemaps/index.xml`
+
+| Path | Role after M3 |
+| ---- | ------------- |
+| `public/sitemaps/index.xml` | **Canonical.** Scheduler can create this even when the May 2026 file is unwritable. Nginx prefers it. |
+| `public/sitemaps/sitemap.xml` | **Legacy fallback** for `/sitemap.xml` if `index.xml` is missing. Same inode as root `sitemap.xml` on current prod (symlink). Best-effort write; failure must not fail generate if `index.xml` succeeded. |
+| `public/sitemap.xml` | **Legacy `$uri`.** On prod this is a symlink to the unwritable klevze file. **Not required** once nginx prefers `index.xml`. Generate may skip it. |
+
+Generate still *attempts* all three; success requires **at least one** index write (`index.xml` is enough).
+
+## robots.txt
+
+Already declares `Sitemap: https://skinbase.org/sitemap.xml`. **No robots change** for M3.
+
+## URLs that should not be in sitemaps
+
+Already excluded: private/unapproved artworks, inactive users, inactive tags, academy pricing query variants, `/pages/about` duplicate, forbidden `/admin` `/cp` etc. No extra removals in M3.
+
+## Tests
+
+`php artisan test tests/Feature/SitemapTest.php` (run after this doc).