# M7.1 — Queue Compatibility & Backlog Containment

```text
STATUS: COMPLETE + FINALIZED (local; production unchanged)
```

---

## 1. RecCompute serialization root cause

M1 added constructor-promoted `private readonly ?int $afterArtworkId = null`.

**Constructor defaults are not applied on `unserialize`.** PHP 8 leaves a **typed property uninitialized** if it is missing from the serialized payload. First access (handle, failed(), logging) throws:

```text
Typed property App\Jobs\RecComputeSimilar*::$afterArtworkId must not be accessed before initialization
```

Old queued jobs (pre-M1, two constructor args) and Laravel `SerializesModels` (skips default-null properties on serialize) both omit the key.

**Fix:** class-level `private ?int $afterArtworkId = null` (not readonly promotion) + `__unserialize` calls `isset` restore. Getter `afterArtworkId()` never reads uninitialized state.

Cursor batching unchanged: `WHERE id > afterArtworkId`, `ORDER BY id`, `LIMIT batch`, no OFFSET.

---

## 2. Stale Tags jobs — worst case (live 2026-08-23 18:xx)

Live `queues:default` RecCompute* (not failed_jobs):

- **Behavior: 0 waiting**
- **Hybrid: 0 waiting**
- **Tags: 169 waiting**, all `artworkId=null` (batch mode), **all already have `afterArtworkId` set** (cursors 1284–4517). These are overlapping remainder chains, not missing-property payloads.

What `afterArtworkId=null` would do: process first 200 public artworks ordered by id, then `dispatch(null, 200, lastId)` until the table is exhausted. One full rebuild ≈ `ceil(N/200)` jobs. With ~50k snapshot-eligible artworks that is **~250 jobs per chain**.

If all 169 were null-cursor roots: **169 × ~250 ≈ 42,000 tag jobs** (duplicate full rebuilds). That is the theoretical worst case.

**Actual live worst case:** they continue from mid-table. Example: 47 jobs with `after=3709` each walk the remainder (~232 batches) → **~11,000 overlapping jobs from that cursor group alone**, plus the other cursor groups. Still dangerous.

**Do not clear `queues:default` wholesale.** Operator command (not scheduled):

```bash
php artisan skinbase:purge-queued-rec-tags --queue=default --dry-run
php artisan skinbase:purge-queued-rec-tags --queue=default --execute
```

Removes **waiting** `RecComputeSimilarByTagsJob` only. Preserves ranking/index/other. Does not touch reserved or `failed_jobs`. Then tonight’s scheduler dispatches **one** clean Tags rebuild (`afterArtworkId=null` once).

---

## 3. Failed-job breakdown (complete, n=133, all queue=default)

M7 sampled 80 rows. Full table:

| Class | Count | Cause |
| ----- | ----: | ----- |
| RecComputeSimilarHybridJob | **66** | `$afterArtworkId` uninitialized |
| RecComputeSimilarByBehaviorJob | **42** | same |
| RecComputeSimilarByTagsJob | **10** | same |
| Enhance\ProcessEnhanceJob | **8** | curl 127.0.0.1:8095 refused |
| App\Mail\AcademyAccessIssue | **7** | mail on default |
| **Total** | **133** | |

66+42+10+8+7=133. Do not retry.

---

## 4. Index* queue

`IndexArtworkJob` / `IndexUserJob` use `config('scout.queue.queue')` (default `search`).

`ShouldBeUniqueUntilProcessing`:

| | IndexUser | IndexArtwork |
| --- | --- | --- |
| uniqueId | userId | artworkId |
| uniqueFor | 60s | 120s |

handle() loads the model from DB, so a dropped duplicate while queued still indexes **current** stats. After the worker **starts**, a later change can enqueue again (not lost). Dead-worker lock expires at uniqueFor.

---

## 5. Ranking dedupe

`RankBuildScopeListsJob` `ShouldBeUnique`, `uniqueId = scopeType:scopeId`.

**uniqueFor=21600 (6 hours)** from `ranking.scope_job_unique_for` / `RANK_SCOPE_JOB_UNIQUE_FOR`. 360s expired long before a 2.5h wait, so hourly fan-out stacked duplicates. 6h > current wait + 300s timeout; dead locks recover the same day. Different scopes stay independent.

---

## 6. Presence memory

`SMEMBERS` removed. `SSCAN` + cap `traffic.online_visitors.index_read_limit` (default 2000). Admin UI may under-count vs 2.48M stale members until prune.

### Stale-member prune (design only — not executed)

1. Detect: SSCAN members; `GET skinbase:presence:online:{member}`; if missing/expired JSON, member is stale.
2. Batches: SSCAN COUNT 200; `SREM` up to 200 stale ids; sleep 50ms.
3. Cost: 2.48M SSCAN + GET. At 200/batch ~12k round trips. Off-peak 5–15 min. Do not `DEL` the index key (would drop live members).
4. Cadence: hourly `withoutOverlapping`, stop after N batches/run.

---

## 7. Orphan producers

| Queue | Consumer? | Enabled? | Useful if months late? | Dispatch today? | M7.1 |
| ----- | --------- | -------- | --------------------- | --------------- | ---- |
| collections | **No** Horizon supervisor | scheduler hourlyAt(43) | Refresh is idempotent; delayed run OK | yes until gate | **`COLLECTIONS_V5_DISPATCH_ENABLED=false`** (default) |
| forum-moderation | **No** Horizon supervisor | gated | **No** for months-old posts | was yes | **`dispatchAsyncScan()` now returns without queueing when `skinbase_ai_moderation.queue.dispatch_enabled` is false.** Topic create/edit and `forum:ai-scan` (async) are gated. `forum:scan-posts` and `forum:ai-scan --sync` stay in-process. |
| forum-security | **No** | gated | stale in hours | 5.5k | **`forum:bot-scan` / `forum:firewall-scan` async dispatch gated.** `--sync` still runs. Scheduler omits those commands when dispatch is false. |

Do not add Horizon workers (M6 RAM).

---

## 8. Existing orphan jobs (do not delete)

| Queue | Class | Disposition |
| ----- | ----- | ----------- |
| collections | Health / recommendation / duplicate | **KEEP AND PROCESS LATER** or **SAFE TO DISCARD** (recompute from current rows). Product: either. |
| forum-moderation | AnalyzeForumPostJob × 552,888 | **SAFE TO DISCARD** for age > 48h; **NEEDS PRODUCT DECISION** for last day. Analyzing May–August posts has no moderation value. |
| forum-security | ~5.5k | **SAFE TO DISCARD** (burst/firewall windows expired). |

Later cleanup (not now): `UNLINK queues:forum-moderation` (async, non-blocking) after stopping producers; or `LTRIM`/`RPOP` in 5k batches. Never one giant `DEL` in a request.

---

## 9. Legacy prefix `skinbasenova-database-queues:*`

Current Laravel prefix is `skinbase-database-`. `skinbasenova-database-*` is leftover from an older `REDIS_PREFIX` / app name. Nothing in this codebase reads it. Horizon also has `skinbasenova_horizon:*` vs `skinbase_horizon:*`.

Reclaimable: ~11 MB default + ~9 MB forum + ~2.5 MB collections + Horizon recent lists (~3 MB) ≈ **25–30 MB** (not the 557 MB live forum list).

Cleanup only after confirming no process uses `skinbasenova-database-` (`HORIZON_PREFIX`, `REDIS_PREFIX` on all PHP/Horizon). Then `UNLINK` those keys. Do not delete `skinbase-database-queues:forum-moderation`.

---

## 10. Files changed

```text
app/Jobs/Concerns/RestoresAfterArtworkIdCursor.php
app/Jobs/RecComputeSimilarByTagsJob.php
app/Jobs/RecComputeSimilarByBehaviorJob.php
app/Jobs/RecComputeSimilarHybridJob.php
app/Jobs/IndexArtworkJob.php
app/Jobs/IndexUserJob.php
app/Jobs/RankBuildScopeListsJob.php
app/Services/Traffic/OnlineVisitorRepository.php
app/Services/CollectionBackgroundJobService.php
app/Console/Commands/DispatchCollectionMaintenanceCommand.php
routes/console.php
config/collections.php
config/forum_security.php
config/skinbase_ai_moderation.php
config/traffic.php
.env.example
phpunit.xml
tests/...
docs/optimization-m7.1-queue-compatibility.md
```

---

## 11. Tests

15 passed: RecCompute unserialize, cursor batching, Index* search queue, ranking uniqueId per scope, presence SSCAN/no SMEMBERS, collection dispatch gate.

---

## 12. Deploy sequence (later; not now)

1. Ship this release (jobs, uniqueness, presence SSCAN, collection gate, purge command).
2. Production `.env`:
   ```text
   COLLECTIONS_V5_DISPATCH_ENABLED=false
   RANK_SCOPE_JOB_UNIQUE_FOR=21600
   ```
   Do not rely on `FORUM_QUEUE_DISPATCH_ENABLED` until the Forum plugin is changed.
3. `php artisan config:cache`
4. `sudo supervisorctl restart skinbase-horizon`
5. Optional: `sudo systemctl reload php8.4-fpm`
6. **After Horizon is on the new code**, purge stale Tags (operator, not cron):
   ```bash
   php artisan skinbase:purge-queued-rec-tags --queue=default --dry-run
   php artisan skinbase:purge-queued-rec-tags --queue=default --execute
   ```
7. Confirm `queues:default` RecComputeSimilarByTagsJob count is 0. Leave ranking/index jobs.
8. Tonight’s scheduler may dispatch **one** Tags rebuild.

**Horizon restart: YES.** Do not retry `failed_jobs`. Do not `SMEMBERS` presence index. Do not `DEL` forum-moderation.

---

## 13. Post-deploy verification

```bash
php artisan tinker --execute="echo json_encode([
  'default' => Redis::llen('queues:default'),
  'search' => Redis::llen('queues:search'),
  'collections' => Redis::llen('queues:collections'),
  'forum' => Redis::llen('queues:forum-moderation'),
]);"
# After some traffic, search LLEN should rise (Index* jobs), default ranking unique should stop stacking
# Do not retry failed_jobs
# Do not SMEMBERS skinbase:presence:online:index
```

---

## 14. Risks

- 169 in-queue tag jobs become “start from id 0” batches; duplicate compute, extra load, no crash.
- Ranking unique skips a scope for up to 360s if a lock is stuck.
- Presence UI shows ≤2000 members until prune.
- Forum enqueue may continue if the plugin ignores `FORUM_QUEUE_DISPATCH_ENABLED`.
- Collection tests set dispatch enabled in phpunit; production default is false.