import-stage: .tasks/ via split (temp prefix, await STATUS.md merge)

git-subtree-dir: .tasks-imported
git-subtree-mainline: c40418239e
git-subtree-split: cbdc2b39aa

.tasks-imported/vds-backup-rsync-kreknin.md

@@ -0,0 +1,102 @@
+# vds-backup-rsync-kreknin
+
+## Goal
+
+Ежедневный rsync-бэкап важных директорий с VDS (`89.253.255.94 / vds.kzntsv.site`) → [[kreknin-synology]] (`195.19.90.188 / kreknin.site`) в 05:00 MSK. После каждого прохода — email-нотификация на `vitya.kuznetsov@gmail.com` со статусом (success/fail + transferred bytes + duration).
+
+## Open questions
+
+- [x] ~~**Что бэкапить:**~~ Resolved 2026-05-20. Sources в `/opt/stacks/backup/scripts/run.sh`:
+  - `/opt/stacks/{gitea,verdaccio/storage,verdaccio/config,registry,traefik,portainer,ntfy,backup}`
+  - `/etc/{ssh,ufw,hosts,docker}`
+  - `/home/vitya/.ssh`
+  - DB dumps в `$DUMP_DIR` (pg/maria/mongo/redis) — добавляются в rsync source list.
+  - **Не бэкапим:** `/opt/stacks/databases/*/data` (raw datadirs — running container locks; dumps вместо).
+- [x] ~~**Куда на kreknin:**~~ `/volume1/NetBackup/vds-kzntsv/<YYYY-MM-DD>/` + symlink `latest`. Retention 7 daily snapshots (RETENTION_DAYS env). Weekly/monthly — defer (overkill для текущих ~12G).
+- [x] ~~**SSH key:**~~ продолжаем `/home/vitya/.ssh/id_ed25519_kreknin` — скрипт читает абсолютным path (work as root too).
+- [x] ~~**Tool:**~~ rsync `--link-dest` (built-in, hardlink-incremental). Encrypted (restic/borg) defer — VDS↔kreknin link trusted (оба ours), encryption key management = ops overhead. Если канал potentially compromised — переключить позже.
+- [x] ~~**Email механизм:**~~ `msmtp` + `msmtp-mta` apt-installed на VDS. Config в `/etc/msmtprc` (chmod 600 root:root, system-wide since cron runs as root). SMTP `smtp.yandex.ru:465` + creds `noreply@snolla.com` из `noreply-snolla-smtp.env`. Test send → Yandex 250 OK ✅.
+- [ ] **Phone confirm:** user verifies ntfy push на `vds-backup` topic пришёл из реального cron run (или manual run).
+
+## Implementation sketch
+
+`/etc/cron.d/vds-backup`:
+
+```cron
+# 05:00 MSK ежедневно
+0 5 * * * vitya /opt/stacks/backup/scripts/run.sh
+```
+
+`/opt/stacks/backup/scripts/run.sh`:
+
+```bash
+#!/bin/bash
+set -e
+SOURCE_DIRS=(/opt/stacks/gitea/data /opt/stacks/verdaccio/storage /opt/stacks/verdaccio/config /opt/stacks/registry/docker /opt/stacks/registry/auth /opt/stacks/traefik /opt/stacks/portainer/data /etc/ssh /etc/ufw /etc/hosts /etc/docker /etc/letsencrypt)
+DEST_BASE=/volume1/NetBackup/vds-kzntsv
+TODAY=$(date +%Y-%m-%d)
+LATEST=$DEST_BASE/latest
+
+START_TIME=$(date +%s)
+LOG=/tmp/backup-$TODAY.log
+
+# DB dumps первым шагом — pg/maria/mongo в /tmp/db-dumps/$TODAY/, потом включаются в rsync
+mkdir -p /tmp/db-dumps/$TODAY
+docker exec postgres pg_dumpall -U postgres > /tmp/db-dumps/$TODAY/postgres.sql
+docker exec mariadb mariadb-dump --all-databases -u root -p"$MARIA_PASS" > /tmp/db-dumps/$TODAY/mariadb.sql
+docker exec mongo mongodump --out=/tmp/mongo --uri="mongodb://root:$MONGO_PASS@localhost:27017?tls=true&tlsAllowInvalidCertificates=true"
+docker exec redis redis-cli --tls --insecure -a "$REDIS_PASS" --rdb /data/dump.rdb
+
+# rsync с --link-dest для hardlink-incremental
+rsync -azh --link-dest=$LATEST "${SOURCE_DIRS[@]}" /tmp/db-dumps/$TODAY \
+  -e "ssh -i ~/.ssh/id_ed25519_kreknin" \
+  vitya@195.19.90.188:$DEST_BASE/$TODAY/
+
+ssh -i ~/.ssh/id_ed25519_kreknin vitya@195.19.90.188 \
+  "rm -f $LATEST && ln -sf $DEST_BASE/$TODAY $LATEST"
+
+END_TIME=$(date +%s)
+DURATION=$((END_TIME - START_TIME))
+SIZE=$(du -sh $DEST_BASE/$TODAY 2>/dev/null | cut -f1)
+
+# Email via msmtp
+echo "Subject: VDS backup $TODAY — SUCCESS
+
+Size: $SIZE, duration: ${DURATION}s
+Source: vds.kzntsv.site
+Dest: kreknin.site:$DEST_BASE/$TODAY/
+" | msmtp vitya.kuznetsov@gmail.com
+```
+
+(скетч; добавить retention prune — `find $DEST_BASE -maxdepth 1 -type d -name "20*" | sort | head -n -7 | xargs rm -rf`, error trap, etc.)
+
+## Decisions log
+
+- **2026-05-20** — rsync `--link-dest` chosen over restic/borg. Built-in, no encryption key management, VDS↔kreknin trusted link. Encryption defer if threat model changes.
+- **2026-05-20** — Cron runs as **root**, not vitya. Initial vitya run hit ~30 Permission Denied (gitea/portainer container-owned files, /etc/ssh host keys, /etc/ufw rules). Trade-off: больше privilege чем нужно, но альтернатива (sudo wrap rsync) — сложнее. Mitigation: `/etc/msmtprc` chmod 600 root:root, `/opt/stacks/backup/.env` chmod 600.
+- **2026-05-20** — DB dump approach: `docker exec <container>` с CLI-passed credentials, не URI params. Mongo требует **legacy `--ssl --sslAllowInvalidCertificates`** (НЕ `--tls --tlsInsecure` despite --help listing `--tlsInsecure` — fails as unknown option pre-`--ssl`).
+- **2026-05-20** — Retention начали с 7 daily snapshots (RETENTION_DAYS=7). Weekly/monthly defer — 12G/day @ 7 days = 84G headroom вне hardlinks, на kreknin 5.6T free; нет смысла усложнять.
+- **2026-05-20** — Cron не был установлен на VDS из-коробки на Ubuntu 24.04 — apt install cron был нужен. Lesson для будущих vds-* tasks: проверять `which cron` в preflight.
+- **2026-05-20** — msmtp `/etc/msmtprc` system-wide (а не `~/.msmtprc`) потому что cron как root → msmtp picks up /etc/ first.
+- **2026-05-20** — Smoke run (2026-05-20 23:37 → 00:00) successful: 71,867 files / 11.74G transferred в 21m27s, all 4 DB dumps OK, latest symlink updated, ntfy + email sent.
+
+## Completed steps
+
+- [x] msmtp + msmtp-mta apt-installed; `/etc/msmtprc` written с Yandex SMTP creds (chmod 600 root:root)
+- [x] `/opt/stacks/backup/.env` создан с DB passwords + kreknin creds + retention (chmod 600)
+- [x] `/opt/stacks/backup/scripts/run.sh` написан (bash + flock single-instance + ERR trap)
+- [x] DB dumps protocol: pg_dumpall, mariadb-dump --single-transaction, mongodump --ssl --sslAllowInvalidCertificates, redis-cli SAVE + docker cp
+- [x] rsync sources finalized: /opt/stacks/*, /etc/{ssh,ufw,hosts,docker}, /home/vitya/.ssh, $DUMP_DIR
+- [x] `/var/log/vds-backup/` log dir (pre-created via sudo + chown vitya)
+- [x] `/etc/cron.d/vds-backup` installed (root, `0 5 * * *`)
+- [x] apt install cron (Ubuntu 24.04 не имел его out-of-box); cron.service active + enabled
+- [x] Smoke run as root: 71,867 files / 11.74G на kreknin:/volume1/NetBackup/vds-kzntsv/2026-05-20, latest symlink set, ntfy publish 200, email Yandex 250 OK
+- [x] Permission errors из vitya-run выявлены и устранены переходом на root cron
+
+## Notes
+
+- Триггер: после `[[vds-kzntsv-bootstrap]]` Phase 3 + verdaccio/registry стабильны (done).
+- Integration с [[vds-ntfy-push]] — backup publish на topic `vds-backup` (admin auth).
+- Retention vs disk: kreknin 5.7T free, daily 7 × ~12G full ≈ 85G headroom (hardlinks делают incremental ~1G/day после day 1).
+- **acme.json** проходит через rsync (внутри /opt/stacks/traefik) — содержит LE private keys, но VDS↔kreknin link trusted, kreknin ACL restricted to vitya. Acceptable trade-off vs шифрование через restic.
+- **Atomic revert:** `ssh vitya@89.253.255.94 'sudo systemctl stop cron; sudo rm /etc/cron.d/vds-backup; sudo apt-get remove -y cron msmtp msmtp-mta; sudo rm -rf /etc/msmtprc /opt/stacks/backup /var/log/vds-backup'`