vitya
c55cb11948
- 7 admin per-task files moved from .tasks-imported/ to .tasks/ (renames) - NEXT-SESSION-PROMPT.md (iis-migration session handoff) kept as sibling - .tasks-imported/STATUS.md removed (content merged into admin STATUS.md) - admin STATUS.md gains 'Imported from MoreThenCms' section with all 7 admin done tasks verbatim. 4 CMS-domain task blocks excluded (stay in MoreThenCms): cms-admin-assets-root-folders-seed, cms-port-leak-fix, traefik-maljarka-502-bug, cms-maljarka-https-mode-bug-fix - morecms-subtree-split marked done (acceptance: 4 split branches exist with per-file history) - admin-subtree-import-and-cleanup marked active (in-progress this session) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
54 lines
5.4 KiB
Markdown
54 lines
5.4 KiB
Markdown
# vds-gc-cron
|
||
|
||
## Goal
|
||
|
||
Weekly GC для двух container-сервисов на VDS, чтобы storage не разрастался:
|
||
1. **Registry** (`/opt/stacks/registry/`) — `registry garbage-collect -m` против nested mount `./docker:/var/lib/registry` (host data в `/opt/stacks/registry/docker/docker/registry/v2/...`).
|
||
2. **Verdaccio** (`/opt/stacks/verdaccio/`) — keep N=10 most-recent versions + dist-tagged per-package, drop ТОЛЬКО proxied tarballs (`_distfiles[tgzName]` defined). Локально-published versions (`@snollajs/*` + similar) защищены.
|
||
|
||
## Key files
|
||
|
||
- `/opt/stacks/gc/scripts/notify.sh` — ntfy helper, sources `/opt/stacks/ntfy/.env`, posts на `vds-ops` topic.
|
||
- `/opt/stacks/gc/scripts/registry-gc.sh` — stop registry → offline `garbage-collect -m` → start; empty-storage guard в начале.
|
||
- `/opt/stacks/gc/scripts/verdaccio-prune.sh` — wrapper, stop verdaccio → docker run `node:20-alpine` mounts storage + script → start.
|
||
- `/opt/stacks/gc/scripts/verdaccio-prune.js` — ядро prune: walk storage, sort versions by `time[v]` desc, keep top-N + dist-tagged, delete tgz + `_attachments` entries for proxied versions, atomic JSON rewrite + `_rev` bump.
|
||
- `/etc/cron.d/vds-gc` — Sunday 03:00 / 03:30 MSK.
|
||
- `/var/log/vds-gc/{registry,verdaccio}-YYYY-MM-DD.log` — append-only logs.
|
||
|
||
## Decisions log
|
||
|
||
- **2026-05-21:** keep N=10 per package для verdaccio (включая `@snollajs/*` — uniform, no special-case scope). Reasoning: N=10 покрывает любые reasonable rollback scenarios, локально-published уже защищены логикой `_distfiles[tgzName]` absent.
|
||
- **2026-05-21:** weekly Sunday 03:00 / 03:30 MSK (2 часа до 05:00 backup cron — нет конфликта). Reasoning: weekly достаточно для personal scale; verdaccio cache растёт ~MB/день не GB/день.
|
||
- **2026-05-21:** registry GC = stop→`-m`→start (~30s downtime) instead of readonly+restart-twice. Reasoning: проще, более atomic; на personal registry 30s downtime в воскресенье ночью acceptable.
|
||
- **2026-05-21:** verdaccio prune = stop+prune+start (~1min downtime). Reasoning: избегаем torn-read race window между tgz delete и package.json rewrite.
|
||
- **2026-05-21:** verdaccio detection of locally-published — через `_distfiles[<tgzName>]` (НЕ `_distfiles[<version>]`). Initial code был bug — verdaccio keys `_distfiles` by tarball filename like `lodash-0.1.0.tgz`, не version string. Dry-run показал 4597 "locally-published" из 7021 — обнаружен через inspection lodash/react/`@snollajs` storage. Real run после fix: 73 версии legitimately protected.
|
||
- **2026-05-21:** notify policy — success-low (`broom` tag, prio low) если freed < 10 MiB AND duration < 5 min; success-default иначе; fail-high (`x,boom` tag, prio high) на любой rc≠0; skip-min на empty registry.
|
||
- **2026-05-21:** atomic package.json rewrite через `write tmp + rename` (POSIX atomic on same fs). Bump `_rev` (`<num+1>-<random_hex>`) чтобы npm clients не закешировали stale packument.
|
||
- **2026-05-21:** verdaccio prune skip `versions{}` and `time{}` entries — оставляем metadata in-place даже для удалённых tarballs. Reasoning: npm может re-fetch tarball from upstream при следующем install request (proxy behaviour); metadata weighs ~KB не GB.
|
||
|
||
## Open questions
|
||
|
||
- [ ] (post-первого live cron run) — phone-verify notification приходит на `vds-ops`.
|
||
- [ ] Logrotate для `/var/log/vds-gc/` — пока nope (weekly cadence + лог ~10KB/run = 500KB/year, терпимо).
|
||
|
||
## Completed steps
|
||
|
||
- [x] recon ntfy creds + verdaccio storage layout (package.json structure: `versions{}`, `_distfiles{}`, `_attachments{}`, `_rev`)
|
||
- [x] write `notify.sh`, `registry-gc.sh`, `verdaccio-prune.sh`+`verdaccio-prune.js`
|
||
- [x] dry-run verdaccio prune (initial bug discovery — `_distfiles[v]` vs `_distfiles[tgzName]`)
|
||
- [x] fix detection → re-dry-run: 4524 tgz / 1.7 GiB candidate
|
||
- [x] real verdaccio prune: 8.5G→6.8G, 1551 pkgs modified, 0 errors
|
||
- [x] verify verdaccio responsive post-prune (lodash metadata 117 versions)
|
||
- [x] push alpine v1+v2 to registry → test GC scenario
|
||
- [x] delete manifest via DELETE API → re-run GC → freed 61M, registry restarted
|
||
- [x] patch registry-gc.sh — empty-storage guard
|
||
- [x] install `/etc/cron.d/vds-gc`, restart cron service
|
||
|
||
## Notes
|
||
|
||
- Verdaccio `_distfiles` keyed by tarball filename (`lodash-0.1.0.tgz`), не version. Gotcha — easy mistake.
|
||
- Multi-arch image push через современный buildkit оставляет manifest как OCI index. Для DELETE через API нужно `Accept: application/vnd.oci.image.manifest.v1+json` (или index variants). Иначе registry возвращает 404.
|
||
- `_attachments` field optional — на proxied packages обычно ~5-10 entries (только cached tarballs), на locally-published — все версии package.
|
||
- `du -sh` (default block-size) vs `du -sb` (bytes) могут rounding-discrepancy на 5-10%. Used `du -sb` в скриптах для precise byte math.
|
||
- registry compose mount `./docker:/var/lib/registry` → registry создаёт data в `/var/lib/registry/docker/registry/v2/...` → host path nested `/opt/stacks/registry/docker/docker/registry/v2/...`. Странно но работает.
|