backup(books-vds): daily 06:00 MSK pipeline to kreknin live 🟢
- scripts/books-vds-backup-daily-kreknin/ — run.sh + .env.example + README - ES path.repo bootstrap (one-time stack edit, snapshot repo 'kreknin' registered) - curl smtps://yandex:465 email (CRLF + Date headers, no msmtp dep on CentOS 7) - StrictHostKeyChecking=yes + pre-populated known_hosts (CentOS 7 no accept-new) - cron /etc/cron.d/books-vds-backup live, initial sync 4.87 GB in 13m28s, ntfy push sent - wiki: entities/books-vds.md + sources/books-vds-backup-daily-kreknin-2026-05-25.md created - wiki: concepts/books-ssh-access → vds-kzntsv-ssh-access (was lying about books on vds-kzntsv) - .tasks/books-vds-backup-daily-kreknin.md closed 🟢 pending phone-side verify Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
115
scripts/books-vds-backup-daily-kreknin/README.md
Normal file
115
scripts/books-vds-backup-daily-kreknin/README.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# books-vds-backup-daily-kreknin
|
||||
|
||||
Daily backup pipeline: **books VDS** (`89.253.255.133` / host4g.ru CentOS 7) → **kreknin Synology** (`195.19.90.188:/volume1/NetBackup/books-vds/<date>/`) via SSH + rsync.
|
||||
|
||||
Schedule: `0 6 * * *` Europe/Moscow (after VDS-infra at 05:00, after windows-host at 05:30). Pattern mirrors `scripts/vds-backup-rsync-kreknin/run.sh`.
|
||||
|
||||
## What's backed up
|
||||
|
||||
| Component | Method | Source |
|
||||
|---|---|---|
|
||||
| MariaDB (books-db) | `mariadb-dump --all-databases --single-transaction` | container `books-db` |
|
||||
| Mongo (shared) | `mongodump --archive` w/ root auth | container `mongo` |
|
||||
| Mongo (job-scheduler) | `mongodump --archive` no-auth | container `books-job-scheduler-mongo` |
|
||||
| Elasticsearch | REST snapshot API → fs repo `kreknin` → rsync | container `elasticsearch` (path.repo=/snapshots) |
|
||||
| MinIO blobs | rsync direct (immutable content-addressed) | `/usr/docker/minio/data` |
|
||||
| App configs | rsync | `/opt/books/{api,job-scheduler,task-runner,ntfy,books-ops-mcp}` |
|
||||
| Traefik state | rsync | `/usr/docker/traefik/{letsencrypt,data}` (acme.json + config) |
|
||||
| System config | rsync | `/etc/{ssh,hosts,cron.d}`, `/root/.ssh` |
|
||||
|
||||
Retention: **7 daily snapshots** on kreknin (+ on ES repo). Older — pruned at end of each run.
|
||||
|
||||
## Notifications
|
||||
|
||||
Dual-channel after every run:
|
||||
|
||||
- **ntfy** → topic `vds-backup` on `ntfy.vds.kzntsv.site` (shared с VDS + RUVDS + windows-host backups).
|
||||
- Success: title `BOOKS-VDS backup OK <date>`, tag `green_circle`.
|
||||
- Fail: title `BOOKS-VDS backup FAILED <date>`, tag `red_circle`, priority `high`.
|
||||
- **Email** via Yandex SMTP (`noreply@snolla.com` → `OPS_NOTIFY_EMAIL`).
|
||||
- Sent via `curl --ssl-reqd --url smtp://...:587` (no msmtp dep on CentOS 7).
|
||||
- Subject: `[BOOKS-VDS] backup OK <date>` / `[BOOKS-VDS] backup FAILED <date>`.
|
||||
|
||||
If notify-channels fail — backup itself does NOT fail (`|| true`).
|
||||
|
||||
## Files
|
||||
|
||||
- `run.sh` — backup logic. Synced to `/opt/stacks/backup/run.sh` on books VDS.
|
||||
- `.env.example` — placeholder. Live `.env` built from `pass show` entries during install.
|
||||
|
||||
## Initial install (rebuild from scratch)
|
||||
|
||||
```bash
|
||||
# From workstation, with id_ed25519_books_ops authorized:
|
||||
ssh root@89.253.255.133 'mkdir -p /opt/stacks/backup'
|
||||
scp scripts/books-vds-backup-daily-kreknin/run.sh root@89.253.255.133:/opt/stacks/backup/
|
||||
|
||||
# Build .env locally from pass entries:
|
||||
{
|
||||
echo "DEST_USER=vitya"
|
||||
echo "DEST_HOST=195.19.90.188"
|
||||
echo "DEST_BASE=/volume1/NetBackup/books-vds"
|
||||
echo "RETENTION_DAYS=7"
|
||||
echo "SSH_KEY=/opt/stacks/backup/kreknin-key"
|
||||
echo "ES_SNAPSHOT_REPO=kreknin"
|
||||
pass show books-vds/full-env | grep -E '^(BOOKS_DB_ROOT_PASSWORD|BOOKS_MONGO_)'
|
||||
pass show vds-kzntsv/full-env | grep -E '^NTFY_'
|
||||
pass show snolla-smtp/full-env | grep -E '^(SMTP_|OPS_NOTIFY_EMAIL)'
|
||||
} | scp /dev/stdin root@89.253.255.133:/opt/stacks/backup/.env
|
||||
ssh root@89.253.255.133 'chmod 600 /opt/stacks/backup/.env && chmod 755 /opt/stacks/backup/run.sh'
|
||||
|
||||
# Generate kreknin SSH key on books VDS:
|
||||
ssh root@89.253.255.133 'ssh-keygen -t ed25519 -f /opt/stacks/backup/kreknin-key -N "" -C "books-vds-backup"'
|
||||
|
||||
# Get pubkey and authorize it on kreknin (workstation pivots — books VDS has no kreknin SSH yet):
|
||||
PUBKEY=$(ssh root@89.253.255.133 cat /opt/stacks/backup/kreknin-key.pub)
|
||||
ssh -i ~/.ssh/id_ed25519_kreknin vitya@195.19.90.188 \
|
||||
"echo '$PUBKEY' >> ~/.ssh/authorized_keys && mkdir -p /volume1/NetBackup/books-vds"
|
||||
|
||||
# Verify SSH books → kreknin
|
||||
ssh root@89.253.255.133 'ssh -i /opt/stacks/backup/kreknin-key -o StrictHostKeyChecking=accept-new vitya@195.19.90.188 hostname'
|
||||
|
||||
# Smoke run (manual)
|
||||
ssh root@89.253.255.133 '/opt/stacks/backup/run.sh'
|
||||
|
||||
# Install cron
|
||||
ssh root@89.253.255.133 'echo "0 6 * * * root /opt/stacks/backup/run.sh" > /etc/cron.d/books-vds-backup && chmod 644 /etc/cron.d/books-vds-backup'
|
||||
```
|
||||
|
||||
## Decisions log
|
||||
|
||||
- **ES snapshot via REST API**, not data-dir rsync. Filesystem snapshot repo (`type=fs`, `location=/snapshots`, `compress=true`) registered as `kreknin` on 2026-05-25. Required one-time stack edit (add `path.repo=/snapshots` env + `./snapshots:/snapshots` bind in `/usr/docker/elasticsearch/docker-compose.yml`).
|
||||
- **`curl --ssl-reqd` SMTP** instead of msmtp — avoid CentOS 7 msmtp install (EOL repo issues). Native `curl` works on STARTTLS port 587.
|
||||
- **No `tasks.yml`-style auth on `books-job-scheduler-mongo`** — container has no `MONGO_INITDB_ROOT_*` env. `mongodump --archive` без `--uri` достаточен.
|
||||
- **rsync /opt/books + /usr/docker/{minio,elasticsearch/snapshots,traefik}** — bind paths из inspect 2026-05-25; не `/var/lib/docker/volumes/` (named volumes для контейнеров пустые, кроме portainer/scheduler-mongo).
|
||||
- **`books-vds-portainer/full-env` consolidated** into `books-vds/full-env` for one-file-per-host pattern parity with `vds-kzntsv/full-env`.
|
||||
- **No `BackupConfig` snapshot/git pre-commit-like step** — VDS-infra `run.sh` тоже не делает, и нет аналогичной системы на books VDS.
|
||||
|
||||
## Smoke run (manual)
|
||||
|
||||
```bash
|
||||
ssh root@89.253.255.133 '/opt/stacks/backup/run.sh'
|
||||
# Watch log:
|
||||
ssh root@89.253.255.133 'tail -F /var/log/books-vds-backup/$(date +%Y-%m-%d).log'
|
||||
```
|
||||
|
||||
Verify on kreknin:
|
||||
```bash
|
||||
ssh vitya@195.19.90.188 'du -sh /volume1/NetBackup/books-vds/*/'
|
||||
```
|
||||
|
||||
## Atomic revert (uninstall)
|
||||
|
||||
```bash
|
||||
# On books VDS:
|
||||
ssh root@89.253.255.133 'rm /etc/cron.d/books-vds-backup; rm -rf /opt/stacks/backup /var/log/books-vds-backup'
|
||||
|
||||
# On kreknin (via SSH):
|
||||
ssh vitya@195.19.90.188 'rm -rf /volume1/NetBackup/books-vds'
|
||||
# Remove the books-vds pubkey line from authorized_keys (one line with 'books-vds-backup' comment).
|
||||
|
||||
# On books VDS — revert ES path.repo change:
|
||||
ssh root@89.253.255.133 'cd /usr/docker/elasticsearch && \
|
||||
cp docker-compose.yml.bak-pre-snapshots-2026-05-25 docker-compose.yml && \
|
||||
docker rm -f elasticsearch && docker-compose up -d'
|
||||
```
|
||||
Reference in New Issue
Block a user