SkinbaseNova/docs/optimization-m3-sitemaps.md

M3 — Sitemap Production Audit & Reliability

Current architecture

Routes:

GET /sitemap.xml              SitemapController@index
GET /sitemaps/{name}.xml      SitemapController@show
GET /robots.txt               RobotsTxtController  (Sitemap: {APP_URL}/sitemap.xml)

Generation: php artisan skinbase:sitemaps:generate (scheduler 10:30 and 22:30) writes static XML on sitemaps_public (public/).

Also: sitemaps:publish, validate, release artifacts, optional live-build fallback.

Families (no groups/years/videos sitemaps):

artworks (sharded 10k), academy-, users (sharded 10k), tags (now shardable at 25k), categories, collections, cards, stories, web-stories, news, news-google, forum-, static-pages.

Canonical artwork URLs: route('art.show') /art/{id}/{slug}. Public/published only.

Exact production issue

/sitemap.xml is a stale file from 2026-05-13, owned by klevze (-rw-r--r--). Children (artworks shards, academy, users, …) refresh daily as skinbase. Scheduler cannot overwrite the index (filesystems.sitemaps_public.throw=false, generate ignored failed put). Nginx try_files $uri serves the stale root file and never hits Laravel. Academy families exist on disk but are absent from the May index.

location @php is missing in the live vhost (documented in M0).

Local vs production

Generate command already writes sitemaps/sitemap.xml on both. Production file ownership plus nginx $uri = public/sitemap.xml is the operational failure. Local did not fail writes or serve a dual path.

Root cause

1. Index file not writable by skinbase.
2. put() failures ignored.
3. Nginx prefers the stale /sitemap.xml over /sitemaps/index.xml (which did not exist).

Implementation

• Generate writes sitemaps/index.xml (new, creatable), plus tries sitemaps/sitemap.xml and sitemap.xml; fails the command if none of the index paths write.
• Child writes report failure instead of counting a silent skip as success.
• SitemapController::index() prefers static sitemaps/index.xml.
• deploy/nginx/sitemaps.conf: try_files /sitemaps/index.xml /sitemaps/sitemap.xml /index.php?$query_string; (no @php)
• Tags builder is shardable (25k) so it cannot exceed 50k URLs without an index.

Caching

Unchanged: nginx max-age=21600; Laravel file response uses sitemaps.cache_ttl_seconds (900). No HTML cache. Generate is the refresh.

Chunking

Artworks/users/forum-threads/collections/cards/stories already sharded at 10k. Tags 25k. Protocol: 50k URLs / 50MB. Production files are all well under 50MB (largest artworks shard ~3.5MB, tags ~2MB).

Deployment

Production vhost (/etc/nginx/sites-enabled/skinbase.org.conf) has no location @php. PHP-FPM is:

location / {
    try_files $uri $uri/ /index.php?$query_string;
}
location = /index.php {
    fastcgi_pass unix:/run/php/php8.4-fpm-skinbase.sock;
    ...
}

Do not use try_files … @php (undefined named location). Use /index.php?$query_string like location /.

Exact replacement for the two sitemap blocks (currently ~lines 91–110):

    location = /sitemap.xml {
        try_files /sitemaps/index.xml /sitemaps/sitemap.xml /index.php?$query_string;

        add_header Cache-Control "public, max-age=21600" always;
        add_header Content-Type  "application/xml; charset=UTF-8" always;
        etag on;
    }

    location ~ ^/sitemaps/[A-Za-z0-9_\-]++\.xml$ {
        try_files $uri /index.php?$query_string;

        add_header Cache-Control "public, max-age=21600" always;
        add_header Content-Type  "application/xml; charset=UTF-8" always;
        etag on;
    }

Do not add $uri to the /sitemap.xml try_files list: $uri is /sitemap.xml, the stale klevze-owned file, and would shadow PHP if index.xml is missing.

Order:

1. Deploy app.
2. Run php artisan skinbase:sitemaps:generate as skinbase so sitemaps/index.xml exists.
3. Patch the vhost, then:

sudo nginx -t && sudo systemctl reload nginx

nginx -t should pass: last try_files argument matches the existing front-controller pattern. Introducing @php would not.

Optional: chown skinbase:skinbase the old sitemap.xml.

Legacy index files after sitemaps/index.xml

Path	Role after M3
public/sitemaps/index.xml	Canonical. Scheduler can create this even when the May 2026 file is unwritable. Nginx prefers it.
public/sitemaps/sitemap.xml	Legacy fallback for /sitemap.xml if index.xml is missing. Same inode as root sitemap.xml on current prod (symlink). Best-effort write; failure must not fail generate if index.xml succeeded.
public/sitemap.xml	Legacy $uri. On prod this is a symlink to the unwritable klevze file. Not required once nginx prefers index.xml. Generate may skip it.

Generate still attempts all three; success requires at least one index write (index.xml is enough).

robots.txt

Already declares Sitemap: https://skinbase.org/sitemap.xml. No robots change for M3.

URLs that should not be in sitemaps

Already excluded: private/unapproved artworks, inactive users, inactive tags, academy pricing query variants, /pages/about duplicate, forbidden /admin /cp etc. No extra removals in M3.

Tests

php artisan test tests/Feature/SitemapTest.php (run after this doc).