admin/vds-gc-cron.md

vds-gc-cron

Goal

Weekly GC для двух container-сервисов на VDS, чтобы storage не разрастался:

1. Registry (/opt/stacks/registry/) — registry garbage-collect -m против nested mount ./docker:/var/lib/registry (host data в /opt/stacks/registry/docker/docker/registry/v2/...).
2. Verdaccio (/opt/stacks/verdaccio/) — keep N=10 most-recent versions + dist-tagged per-package, drop ТОЛЬКО proxied tarballs (_distfiles[tgzName] defined). Локально-published versions (@snollajs/* + similar) защищены.

Key files

• /opt/stacks/gc/scripts/notify.sh — ntfy helper, sources /opt/stacks/ntfy/.env, posts на vds-ops topic.
• /opt/stacks/gc/scripts/registry-gc.sh — stop registry → offline garbage-collect -m → start; empty-storage guard в начале.
• /opt/stacks/gc/scripts/verdaccio-prune.sh — wrapper, stop verdaccio → docker run node:20-alpine mounts storage + script → start.
• /opt/stacks/gc/scripts/verdaccio-prune.js — ядро prune: walk storage, sort versions by time[v] desc, keep top-N + dist-tagged, delete tgz + _attachments entries for proxied versions, atomic JSON rewrite + _rev bump.
• /etc/cron.d/vds-gc — Sunday 03:00 / 03:30 MSK.
• /var/log/vds-gc/{registry,verdaccio}-YYYY-MM-DD.log — append-only logs.

Decisions log

• 2026-05-21: keep N=10 per package для verdaccio (включая @snollajs/* — uniform, no special-case scope). Reasoning: N=10 покрывает любые reasonable rollback scenarios, локально-published уже защищены логикой _distfiles[tgzName] absent.
• 2026-05-21: weekly Sunday 03:00 / 03:30 MSK (2 часа до 05:00 backup cron — нет конфликта). Reasoning: weekly достаточно для personal scale; verdaccio cache растёт ~MB/день не GB/день.
• 2026-05-21: registry GC = stop→-m→start (~30s downtime) instead of readonly+restart-twice. Reasoning: проще, более atomic; на personal registry 30s downtime в воскресенье ночью acceptable.
• 2026-05-21: verdaccio prune = stop+prune+start (~1min downtime). Reasoning: избегаем torn-read race window между tgz delete и package.json rewrite.
• 2026-05-21: verdaccio detection of locally-published — через _distfiles[<tgzName>] (НЕ _distfiles[<version>]). Initial code был bug — verdaccio keys _distfiles by tarball filename like lodash-0.1.0.tgz, не version string. Dry-run показал 4597 "locally-published" из 7021 — обнаружен через inspection lodash/react/@snollajs storage. Real run после fix: 73 версии legitimately protected.
• 2026-05-21: notify policy — success-low (broom tag, prio low) если freed < 10 MiB AND duration < 5 min; success-default иначе; fail-high (x,boom tag, prio high) на любой rc≠0; skip-min на empty registry.
• 2026-05-21: atomic package.json rewrite через write tmp + rename (POSIX atomic on same fs). Bump _rev (<num+1>-<random_hex>) чтобы npm clients не закешировали stale packument.
• 2026-05-21: verdaccio prune skip versions{} and time{} entries — оставляем metadata in-place даже для удалённых tarballs. Reasoning: npm может re-fetch tarball from upstream при следующем install request (proxy behaviour); metadata weighs ~KB не GB.

Open questions

• (post-первого live cron run) — phone-verify notification приходит на vds-ops.
• Logrotate для /var/log/vds-gc/ — пока nope (weekly cadence + лог ~10KB/run = 500KB/year, терпимо).

Completed steps

• recon ntfy creds + verdaccio storage layout (package.json structure: versions{}, _distfiles{}, _attachments{}, _rev)
• write notify.sh, registry-gc.sh, verdaccio-prune.sh+verdaccio-prune.js
• dry-run verdaccio prune (initial bug discovery — _distfiles[v] vs _distfiles[tgzName])
• fix detection → re-dry-run: 4524 tgz / 1.7 GiB candidate
• real verdaccio prune: 8.5G→6.8G, 1551 pkgs modified, 0 errors
• verify verdaccio responsive post-prune (lodash metadata 117 versions)
• push alpine v1+v2 to registry → test GC scenario
• delete manifest via DELETE API → re-run GC → freed 61M, registry restarted
• patch registry-gc.sh — empty-storage guard
• install /etc/cron.d/vds-gc, restart cron service

Notes

• Verdaccio _distfiles keyed by tarball filename (lodash-0.1.0.tgz), не version. Gotcha — easy mistake.
• Multi-arch image push через современный buildkit оставляет manifest как OCI index. Для DELETE через API нужно Accept: application/vnd.oci.image.manifest.v1+json (или index variants). Иначе registry возвращает 404.
• _attachments field optional — на proxied packages обычно ~5-10 entries (только cached tarballs), на locally-published — все версии package.
• du -sh (default block-size) vs du -sb (bytes) могут rounding-discrepancy на 5-10%. Used du -sb в скриптах для precise byte math.
• registry compose mount ./docker:/var/lib/registry → registry создаёт data в /var/lib/registry/docker/registry/v2/... → host path nested /opt/stacks/registry/docker/docker/registry/v2/.... Странно но работает.