vitya
c55cb11948
- 7 admin per-task files moved from .tasks-imported/ to .tasks/ (renames) - NEXT-SESSION-PROMPT.md (iis-migration session handoff) kept as sibling - .tasks-imported/STATUS.md removed (content merged into admin STATUS.md) - admin STATUS.md gains 'Imported from MoreThenCms' section with all 7 admin done tasks verbatim. 4 CMS-domain task blocks excluded (stay in MoreThenCms): cms-admin-assets-root-folders-seed, cms-port-leak-fix, traefik-maljarka-502-bug, cms-maljarka-https-mode-bug-fix - morecms-subtree-split marked done (acceptance: 4 split branches exist with per-file history) - admin-subtree-import-and-cleanup marked active (in-progress this session) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
103 lines
7.3 KiB
Markdown
103 lines
7.3 KiB
Markdown
# vds-backup-rsync-kreknin
|
||
|
||
## Goal
|
||
|
||
Ежедневный rsync-бэкап важных директорий с VDS (`89.253.255.94 / vds.kzntsv.site`) → [[kreknin-synology]] (`195.19.90.188 / kreknin.site`) в 05:00 MSK. После каждого прохода — email-нотификация на `vitya.kuznetsov@gmail.com` со статусом (success/fail + transferred bytes + duration).
|
||
|
||
## Open questions
|
||
|
||
- [x] ~~**Что бэкапить:**~~ Resolved 2026-05-20. Sources в `/opt/stacks/backup/scripts/run.sh`:
|
||
- `/opt/stacks/{gitea,verdaccio/storage,verdaccio/config,registry,traefik,portainer,ntfy,backup}`
|
||
- `/etc/{ssh,ufw,hosts,docker}`
|
||
- `/home/vitya/.ssh`
|
||
- DB dumps в `$DUMP_DIR` (pg/maria/mongo/redis) — добавляются в rsync source list.
|
||
- **Не бэкапим:** `/opt/stacks/databases/*/data` (raw datadirs — running container locks; dumps вместо).
|
||
- [x] ~~**Куда на kreknin:**~~ `/volume1/NetBackup/vds-kzntsv/<YYYY-MM-DD>/` + symlink `latest`. Retention 7 daily snapshots (RETENTION_DAYS env). Weekly/monthly — defer (overkill для текущих ~12G).
|
||
- [x] ~~**SSH key:**~~ продолжаем `/home/vitya/.ssh/id_ed25519_kreknin` — скрипт читает абсолютным path (work as root too).
|
||
- [x] ~~**Tool:**~~ rsync `--link-dest` (built-in, hardlink-incremental). Encrypted (restic/borg) defer — VDS↔kreknin link trusted (оба ours), encryption key management = ops overhead. Если канал potentially compromised — переключить позже.
|
||
- [x] ~~**Email механизм:**~~ `msmtp` + `msmtp-mta` apt-installed на VDS. Config в `/etc/msmtprc` (chmod 600 root:root, system-wide since cron runs as root). SMTP `smtp.yandex.ru:465` + creds `noreply@snolla.com` из `noreply-snolla-smtp.env`. Test send → Yandex 250 OK ✅.
|
||
- [ ] **Phone confirm:** user verifies ntfy push на `vds-backup` topic пришёл из реального cron run (или manual run).
|
||
|
||
## Implementation sketch
|
||
|
||
`/etc/cron.d/vds-backup`:
|
||
|
||
```cron
|
||
# 05:00 MSK ежедневно
|
||
0 5 * * * vitya /opt/stacks/backup/scripts/run.sh
|
||
```
|
||
|
||
`/opt/stacks/backup/scripts/run.sh`:
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
set -e
|
||
SOURCE_DIRS=(/opt/stacks/gitea/data /opt/stacks/verdaccio/storage /opt/stacks/verdaccio/config /opt/stacks/registry/docker /opt/stacks/registry/auth /opt/stacks/traefik /opt/stacks/portainer/data /etc/ssh /etc/ufw /etc/hosts /etc/docker /etc/letsencrypt)
|
||
DEST_BASE=/volume1/NetBackup/vds-kzntsv
|
||
TODAY=$(date +%Y-%m-%d)
|
||
LATEST=$DEST_BASE/latest
|
||
|
||
START_TIME=$(date +%s)
|
||
LOG=/tmp/backup-$TODAY.log
|
||
|
||
# DB dumps первым шагом — pg/maria/mongo в /tmp/db-dumps/$TODAY/, потом включаются в rsync
|
||
mkdir -p /tmp/db-dumps/$TODAY
|
||
docker exec postgres pg_dumpall -U postgres > /tmp/db-dumps/$TODAY/postgres.sql
|
||
docker exec mariadb mariadb-dump --all-databases -u root -p"$MARIA_PASS" > /tmp/db-dumps/$TODAY/mariadb.sql
|
||
docker exec mongo mongodump --out=/tmp/mongo --uri="mongodb://root:$MONGO_PASS@localhost:27017?tls=true&tlsAllowInvalidCertificates=true"
|
||
docker exec redis redis-cli --tls --insecure -a "$REDIS_PASS" --rdb /data/dump.rdb
|
||
|
||
# rsync с --link-dest для hardlink-incremental
|
||
rsync -azh --link-dest=$LATEST "${SOURCE_DIRS[@]}" /tmp/db-dumps/$TODAY \
|
||
-e "ssh -i ~/.ssh/id_ed25519_kreknin" \
|
||
vitya@195.19.90.188:$DEST_BASE/$TODAY/
|
||
|
||
ssh -i ~/.ssh/id_ed25519_kreknin vitya@195.19.90.188 \
|
||
"rm -f $LATEST && ln -sf $DEST_BASE/$TODAY $LATEST"
|
||
|
||
END_TIME=$(date +%s)
|
||
DURATION=$((END_TIME - START_TIME))
|
||
SIZE=$(du -sh $DEST_BASE/$TODAY 2>/dev/null | cut -f1)
|
||
|
||
# Email via msmtp
|
||
echo "Subject: VDS backup $TODAY — SUCCESS
|
||
|
||
Size: $SIZE, duration: ${DURATION}s
|
||
Source: vds.kzntsv.site
|
||
Dest: kreknin.site:$DEST_BASE/$TODAY/
|
||
" | msmtp vitya.kuznetsov@gmail.com
|
||
```
|
||
|
||
(скетч; добавить retention prune — `find $DEST_BASE -maxdepth 1 -type d -name "20*" | sort | head -n -7 | xargs rm -rf`, error trap, etc.)
|
||
|
||
## Decisions log
|
||
|
||
- **2026-05-20** — rsync `--link-dest` chosen over restic/borg. Built-in, no encryption key management, VDS↔kreknin trusted link. Encryption defer if threat model changes.
|
||
- **2026-05-20** — Cron runs as **root**, not vitya. Initial vitya run hit ~30 Permission Denied (gitea/portainer container-owned files, /etc/ssh host keys, /etc/ufw rules). Trade-off: больше privilege чем нужно, но альтернатива (sudo wrap rsync) — сложнее. Mitigation: `/etc/msmtprc` chmod 600 root:root, `/opt/stacks/backup/.env` chmod 600.
|
||
- **2026-05-20** — DB dump approach: `docker exec <container>` с CLI-passed credentials, не URI params. Mongo требует **legacy `--ssl --sslAllowInvalidCertificates`** (НЕ `--tls --tlsInsecure` despite --help listing `--tlsInsecure` — fails as unknown option pre-`--ssl`).
|
||
- **2026-05-20** — Retention начали с 7 daily snapshots (RETENTION_DAYS=7). Weekly/monthly defer — 12G/day @ 7 days = 84G headroom вне hardlinks, на kreknin 5.6T free; нет смысла усложнять.
|
||
- **2026-05-20** — Cron не был установлен на VDS из-коробки на Ubuntu 24.04 — apt install cron был нужен. Lesson для будущих vds-* tasks: проверять `which cron` в preflight.
|
||
- **2026-05-20** — msmtp `/etc/msmtprc` system-wide (а не `~/.msmtprc`) потому что cron как root → msmtp picks up /etc/ first.
|
||
- **2026-05-20** — Smoke run (2026-05-20 23:37 → 00:00) successful: 71,867 files / 11.74G transferred в 21m27s, all 4 DB dumps OK, latest symlink updated, ntfy + email sent.
|
||
|
||
## Completed steps
|
||
|
||
- [x] msmtp + msmtp-mta apt-installed; `/etc/msmtprc` written с Yandex SMTP creds (chmod 600 root:root)
|
||
- [x] `/opt/stacks/backup/.env` создан с DB passwords + kreknin creds + retention (chmod 600)
|
||
- [x] `/opt/stacks/backup/scripts/run.sh` написан (bash + flock single-instance + ERR trap)
|
||
- [x] DB dumps protocol: pg_dumpall, mariadb-dump --single-transaction, mongodump --ssl --sslAllowInvalidCertificates, redis-cli SAVE + docker cp
|
||
- [x] rsync sources finalized: /opt/stacks/*, /etc/{ssh,ufw,hosts,docker}, /home/vitya/.ssh, $DUMP_DIR
|
||
- [x] `/var/log/vds-backup/` log dir (pre-created via sudo + chown vitya)
|
||
- [x] `/etc/cron.d/vds-backup` installed (root, `0 5 * * *`)
|
||
- [x] apt install cron (Ubuntu 24.04 не имел его out-of-box); cron.service active + enabled
|
||
- [x] Smoke run as root: 71,867 files / 11.74G на kreknin:/volume1/NetBackup/vds-kzntsv/2026-05-20, latest symlink set, ntfy publish 200, email Yandex 250 OK
|
||
- [x] Permission errors из vitya-run выявлены и устранены переходом на root cron
|
||
|
||
## Notes
|
||
|
||
- Триггер: после `[[vds-kzntsv-bootstrap]]` Phase 3 + verdaccio/registry стабильны (done).
|
||
- Integration с [[vds-ntfy-push]] — backup publish на topic `vds-backup` (admin auth).
|
||
- Retention vs disk: kreknin 5.7T free, daily 7 × ~12G full ≈ 85G headroom (hardlinks делают incremental ~1G/day после day 1).
|
||
- **acme.json** проходит через rsync (внутри /opt/stacks/traefik) — содержит LE private keys, но VDS↔kreknin link trusted, kreknin ACL restricted to vitya. Acceptable trade-off vs шифрование через restic.
|
||
- **Atomic revert:** `ssh vitya@89.253.255.94 'sudo systemctl stop cron; sudo rm /etc/cron.d/vds-backup; sudo apt-get remove -y cron msmtp msmtp-mta; sudo rm -rf /etc/msmtprc /opt/stacks/backup /var/log/vds-backup'`
|