SkinbaseNova/docs/optimization-m3-sitemaps.md

# M3 — Sitemap Production Audit & Reliability

## Current architecture

Routes:

```text
GET /sitemap.xml              SitemapController@index
GET /sitemaps/{name}.xml      SitemapController@show
GET /robots.txt               RobotsTxtController  (Sitemap: {APP_URL}/sitemap.xml)
```

Generation: `php artisan skinbase:sitemaps:generate` (scheduler 10:30 and 22:30) writes static XML on `sitemaps_public` (`public/`).

Also: `sitemaps:publish`, `validate`, release artifacts, optional live-build fallback.

Families (no groups/years/videos sitemaps):

artworks (sharded 10k), academy-*, users (sharded 10k), tags (now shardable at 25k), categories, collections, cards, stories, web-stories, news, news-google, forum-*, static-pages.

Canonical artwork URLs: `route('art.show')` `/art/{id}/{slug}`. Public/published only.

## Exact production issue

`/sitemap.xml` is a **stale file from 2026-05-13**, owned by **klevze** (`-rw-r--r--`). Children (artworks shards, academy, users, …) refresh daily as **skinbase**. Scheduler cannot overwrite the index (`filesystems.sitemaps_public.throw=false`, generate ignored failed `put`). Nginx `try_files $uri` serves the stale root file and **never** hits Laravel. Academy families exist on disk but are **absent from the May index**.

`location @php` is missing in the live vhost (documented in M0).

## Local vs production

Generate command already writes `sitemaps/sitemap.xml` on both. Production **file ownership** plus **nginx `$uri` = public/sitemap.xml** is the operational failure. Local did not fail writes or serve a dual path.

## Root cause

1. Index file not writable by `skinbase`.
2. `put()` failures ignored.
3. Nginx prefers the stale `/sitemap.xml` over `/sitemaps/index.xml` (which did not exist).

## Implementation

- Generate writes `sitemaps/index.xml` (new, creatable), plus tries `sitemaps/sitemap.xml` and `sitemap.xml`; **fails the command if none of the index paths write**.
- Child writes report failure instead of counting a silent skip as success.
- `SitemapController::index()` prefers static `sitemaps/index.xml`.
- `deploy/nginx/sitemaps.conf`: `try_files /sitemaps/index.xml /sitemaps/sitemap.xml /index.php?$query_string;` (no `@php`)
- Tags builder is shardable (25k) so it cannot exceed 50k URLs without an index.

## Caching

Unchanged: nginx `max-age=21600`; Laravel file response uses `sitemaps.cache_ttl_seconds` (900). No HTML cache. Generate is the refresh.

## Chunking

Artworks/users/forum-threads/collections/cards/stories already sharded at 10k. Tags 25k. Protocol: 50k URLs / 50MB. Production files are all well under 50MB (largest artworks shard ~3.5MB, tags ~2MB).

## Deployment

Production vhost (`/etc/nginx/sites-enabled/skinbase.org.conf`) has **no** `location @php`. PHP-FPM is:

```nginx
location / {
    try_files $uri $uri/ /index.php?$query_string;
}
location = /index.php {
    fastcgi_pass unix:/run/php/php8.4-fpm-skinbase.sock;
    ...
}
```

**Do not** use `try_files … @php` (undefined named location). Use `/index.php?$query_string` like `location /`.

Exact replacement for the two sitemap blocks (currently ~lines 91–110):

```nginx
    location = /sitemap.xml {
        try_files /sitemaps/index.xml /sitemaps/sitemap.xml /index.php?$query_string;

        add_header Cache-Control "public, max-age=21600" always;
        add_header Content-Type  "application/xml; charset=UTF-8" always;
        etag on;
    }

    location ~ ^/sitemaps/[A-Za-z0-9_\-]++\.xml$ {
        try_files $uri /index.php?$query_string;

        add_header Cache-Control "public, max-age=21600" always;
        add_header Content-Type  "application/xml; charset=UTF-8" always;
        etag on;
    }
```

Do **not** add `$uri` to the `/sitemap.xml` try_files list: `$uri` is `/sitemap.xml`, the stale klevze-owned file, and would shadow PHP if `index.xml` is missing.

Order:

1. Deploy app.
2. Run `php artisan skinbase:sitemaps:generate` as `skinbase` so `sitemaps/index.xml` exists.
3. Patch the vhost, then:

```bash
sudo nginx -t && sudo systemctl reload nginx
```

`nginx -t` should pass: last `try_files` argument matches the existing front-controller pattern. Introducing `@php` would not.

Optional: `chown skinbase:skinbase` the old `sitemap.xml`.

## Legacy index files after `sitemaps/index.xml`

| Path | Role after M3 |
| ---- | ------------- |
| `public/sitemaps/index.xml` | **Canonical.** Scheduler can create this even when the May 2026 file is unwritable. Nginx prefers it. |
| `public/sitemaps/sitemap.xml` | **Legacy fallback** for `/sitemap.xml` if `index.xml` is missing. Same inode as root `sitemap.xml` on current prod (symlink). Best-effort write; failure must not fail generate if `index.xml` succeeded. |
| `public/sitemap.xml` | **Legacy `$uri`.** On prod this is a symlink to the unwritable klevze file. **Not required** once nginx prefers `index.xml`. Generate may skip it. |

Generate still *attempts* all three; success requires **at least one** index write (`index.xml` is enough).

## robots.txt

Already declares `Sitemap: https://skinbase.org/sitemap.xml`. **No robots change** for M3.

## URLs that should not be in sitemaps

Already excluded: private/unapproved artworks, inactive users, inactive tags, academy pricing query variants, `/pages/about` duplicate, forbidden `/admin` `/cp` etc. No extra removals in M3.

## Tests

`php artisan test tests/Feature/SitemapTest.php` (run after this doc).