Compare commits

...

3 Commits

Author SHA1 Message Date
bb7dd40436 ruvds-backup: add email-notify (Yandex SMTP 587 STARTTLS) + fix size calc
Add dual-channel notifications — ntfy (already there) + email via
Send-MailMessage / Yandex SMTP, mirroring VDS msmtp pipeline.

Email: noreply@snolla.com -> vitya.kuznetsov@gmail.com.
Verified delivered (DKIM pass, SPF pass) via run #2 + manual smoke.

Settings sourced from C:\sites\snolla\Web.config <mailSettings> (Yandex
SMTP), cross-checked against pass-store snolla-smtp/full-env. Same
creds, same account as VDS backup msmtp uses.

SMTP port: 587 STARTTLS, NOT 465 implicit-TLS:
.NET SmtpClient / Send-MailMessage support only STARTTLS. Yandex
accepts both; we use 587 for native PS tooling. VDS msmtp uses 465
implicit-TLS — both work, different tools.

Size in email: switched from `rclone size --json | ConvertFrom-Json`
(parses fail when rclone NOTICE stderr leaks into stdout) to local
Get-ChildItem on C:\sites\snolla. Instant, no JSON dance.

setup.ps1: ned params for SMTP creds + OPS_NOTIFY_EMAIL; config.env
template extended.

README.md: notifications section split into ntfy + email subsections,
new SMTP-port + size-calc decisions in Decisions log.

Run #2 (12:30:03) and run #3 (12:33) both produced email delivery
receipts in user inbox.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:35:25 +03:00
8db4b0d71c tasks(ruvds-backup-daily-kreknin): close 🟢 — live на 04:30 MSK daily
End-to-end pipeline отработан, run #1 verified 20 sec, 8.8 GB snolla +
IIS configs + 14 certs + sshd state → /volume1/NetBackup/ruvds-iis/<date>/
на kreknin. Retention 7 daily, ntfy vds-backup topic.

Scripts checked into scripts/ruvds-backup-daily-kreknin/:
- setup.ps1 (one-time install: SSH key + rclone + configs + ScheduledTask)
- run.ps1 (live backup logic; Invoke-Rclone wrapper для NOTICE-on-stderr)
- README.md (decisions log, smoke instructions, atomic revert)

Bugs found and fixed during smoke (см. README Decisions log):
1. Backup-WebConfiguration -Force параметра нет → check + Remove first
2. rclone --log-file lock с PS Start-Transcript → drop --log-file
3. rclone NOTICE на stderr + $ErrorActionPreference=Stop → Invoke-Rclone
   wrapper temporarily switches к Continue
4. ssh-keyscan known_hosts не parsится rclone go-sftp → drop pinning,
   rely on key-auth

Закрывает "Backup strategy для RUVDS IIS" Open question в
[iis-migration-to-ruvds].

Open follow-ups (не блокер):
- PFX export pass plaintext в скрипте — TODO move to gpg/DPAPI
- Retention prune (kept 1 today) — verify в day 8
- Phone-side ntfy push — user verifies

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 11:49:44 +03:00
df5878cdad tasks: + [ruvds-backup-daily-kreknin] ready — параллель к VDS backup
Создаю задачу пока [iis-migration-to-ruvds] в soak window. Закрывает
Open-question "Backup strategy для RUVDS IIS" из migration task.

Pattern параллель [vds-backup-rsync-kreknin] 🟢 — daily backup → kreknin
synology, 7 daily snapshots retention, ntfy на vds-backup topic.

Differences vs VDS:
- Win Server 2025 Core → rclone single-binary, не rsync (no cygwin/WSL)
- Task Scheduler SYSTEM-account, не cron-root
- 04:30 MSK (за 30 мин до VDS backup чтобы не пересекать uplink)
- Backup scope: C:\sites\snolla + applicationHost.config + IIS native
  Backup-WebConfiguration + cert store PFX export + sshd config

Decisions log в task file: tool=rclone (rejected restic encrypted-dedupe
как overkill для transitional setup, robocopy-over-SMB fragile);
SYSTEM-principal mirrors VDS root-cron pattern; PFX export pass=temp
'pfximport' (TODO: pass-equivalent на Windows когда найдём).

Open questions для решения user'а: when to start (recommend сейчас, не
ждать full cutover), kreknin SSH ACL add pubkey to authorized_keys
(требует user action).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 10:11:34 +03:00
5 changed files with 658 additions and 1 deletions

View File

@@ -1,5 +1,5 @@
# Admin Task Board # Admin Task Board
_Updated: 2026-05-24 (IIS migration to RUVDS — **partial cutover live**: snolla site (8.66 GB / 44725 files) transferred via scp (после ISP outbound-445 block обнаружен), IIS recreated, 25 HTTPS SNI bindings с LE certs (re-used from traefik acme.json), 7 prod hostnames live-smoked через VDS (третья сеть) → 200 OK + correct content. DNS A для kupimknigi.spb.ru flipped на 80.64.31.36 (после initial misroute на VDS). Soak window: 24h+. Source IIS:8089 + traefik routes ALIVE — rollback ready. Previous: temp admin start 2026-05-23 09:37-18:08, dropped на UNC-robocopy exit 16; secrets-leak в `.secrets/` fixed → `pass`. См. также 2026-05-22 chain: MSSQL Express 2022 live на `mssql.kzntsv.site:1433` (5 DBs, 8 prod hosts 200 OK), MinIO/imgproxy VDS standby.)_ _Updated: 2026-05-24 (RUVDS backup pipeline live — `[ruvds-backup-daily-kreknin]` 🟢: rclone SFTP daily 04:30 MSK, 5 components synced (8.8 GB snolla + IIS configs + certs + sshd state) to `/volume1/NetBackup/ruvds-iis/<date>/` на kreknin, retention 7 daily, ntfy `vds-backup` topic shared с VDS backup. Run #1 verified 11:46:51 — 20 sec full sync. Earlier today: IIS migration partial cutover snolla site (8.66 GB) transferred via scp (после ISP outbound-445 block), IIS recreated, 25 HTTPS SNI bindings с LE certs (re-used from traefik acme.json), 7 prod hostnames live-smoked через VDS (третья сеть) → 200 OK. DNS A для kupimknigi.spb.ru flipped 80.64.31.36. Soak window 24h+, source IIS:8089 + traefik routes ALIVE — rollback ready.)_
<!-- <!--
Status legend: Status legend:
@@ -698,6 +698,43 @@ Copy-Item C:\sites\snolla\Web.config.bak-pre-vds-cutover-20260522 C:\sites\snoll
--- ---
## 🟢 [ruvds-backup-daily-kreknin] — daily rclone SFTP RUVDS → kreknin live (run #1 ✓ 2026-05-24 11:46:51, 20 sec, all 5 components synced)
**Closed:** 2026-05-24 11:46:51 — full pipeline live, ntfy success fired.
**Components synced (5):**
- `sites/snolla` 8.8 GB / 44725 files (`C:\sites\snolla\`)
- `iis-config/` (`applicationHost.config`)
- `iis-backup-webconfiguration/` (`Backup-WebConfiguration -Name daily-<date>` export)
- `certs/` 14 PFX (LocalMachine\My private-key certs, pass `ruvds-backup-pfx`)
- `ssh-config/` (sshd_config + administrators_authorized_keys)
**Path on kreknin:** `/volume1/NetBackup/ruvds-iis/<date>/`. Retention 7 daily snapshots (pruned by rclone purge step 4).
**Schedule:** Task `RUVDS-Backup-Daily`, daily 04:30 MSK as SYSTEM, 2h timeout.
**Notifications:** ntfy `vds-backup` topic (shared с VDS backup для phone-side aggregation). Title differentiator: `RUVDS backup OK/FAILED (<date>)`.
**Scripts:** `scripts/ruvds-backup-daily-kreknin/``setup.ps1` (one-time install: SSH key + rclone + configs + ScheduledTask) + `run.ps1` (live backup logic) + README.
**Findings (Decisions log):**
- rclone SFTP без host-key validation (rclone go-sftp library не parsит ssh-keyscan format корректно — key mismatch даже на fresh keys; key-auth = sufficient для нашего threat model)
- `Invoke-Rclone` wrapper нужен — `$ErrorActionPreference='Stop'` + rclone NOTICE на stderr = PS terminating error; wrapper switches к Continue для rclone calls
- `Backup-WebConfiguration -Force` параметр отсутствует на этой версии IIS module — используем `Remove-WebConfigurationBackup` если exists + plain `Backup-WebConfiguration`
**Open follow-ups (не блокер):**
- PFX export pass `ruvds-backup-pfx` — temporary. TODO: pass-equivalent на Windows (gpg4win + pass-bash, или derived от machine-creds).
- Retention prune ещё не сработал (1 snapshot today, < 7); проверится в day 8.
**Atomic revert:** см. `scripts/ruvds-backup-daily-kreknin/README.md` §Atomic revert.
**Closes** "Backup strategy для RUVDS IIS" Open question в `[iis-migration-to-ruvds]`.
**Branch:** master
<!-- created-by: vitya / 2026-05-24 / trigger: post-iis-migration-partial-cutover -->
<!-- closed-by: vitya / 2026-05-24 11:46:51 / run #1 OK 20 sec; full backup 8.8 GB on kreknin -->
---
## 🟡 [iis-migration-to-ruvds] — RUVDS IIS live для 1 prod hostname (`kupimknigi.spb.ru`), 24 в очереди; источник held для rollback ## 🟡 [iis-migration-to-ruvds] — RUVDS IIS live для 1 prod hostname (`kupimknigi.spb.ru`), 24 в очереди; источник held для rollback
**Status:** in_progress (DNS partial cutover — soak window 2026-05-24) **Status:** in_progress (DNS partial cutover — soak window 2026-05-24)

View File

@@ -0,0 +1,236 @@
# ruvds-backup-daily-kreknin
## Goal
Ежедневный backup с RUVDS (`80.64.31.36` / Win Server 2025 Core) → [[../entities/kreknin-synology]] (`195.19.90.188 / kreknin.site`) в **04:30 MSK** (за 30 мин до `[vds-backup-rsync-kreknin]` 05:00, чтобы не пересекать uplink). После каждого прохода — ntfy push на topic `vds-backup` (тот же что VDS backup пушит — общий agg-канал).
Параллель `[vds-backup-rsync-kreknin]` 🟢 (VDS Ubuntu → kreknin via rsync `--link-dest`), но source = Windows Server Core, поэтому:
- Cron нет → Windows Task Scheduler
- rsync нативного нет → **rclone** sync over SFTP (см. Decisions ниже)
- root-cron-as-uid недоступен → SYSTEM scheduled task
## Scope (what to backup)
Минимум для disaster-recovery RUVDS state:
| Path on RUVDS | Reason | Approx size |
|---|---|---|
| `C:\sites\snolla\` | CMS site content + Web.config + media | 8.66 GB (grows slowly) |
| `C:\Windows\System32\inetsrv\config\applicationHost.config` | IIS site/binding/apppool config | <1 MB |
| Backup-WebConfiguration export | IIS native config snapshot (`Backup-WebConfiguration -Name ...`) → `%SystemRoot%\System32\inetsrv\backup\` | <5 MB |
| `C:\ProgramData\ssh\` (sshd_config + administrators_authorized_keys) | SSH access state | <50 KB |
| LE certs export (`Get-ChildItem Cert:\LocalMachine\My` → Export-PfxCertificate с временным пасс'ом + git-tracked) | для restore без re-extract из traefik acme.json | <1 MB |
**Не бэкапить:**
- `C:\Windows\` system files — RUVDS rebuild = fresh install (~5 мин через RUVDS panel).
- `C:\Program Files\` — installed software re-installable from scripts (bootstrap.ps1).
- IIS logs (`C:\inetpub\logs\LogFiles\`) — диагностические, не recovery-critical. Optional.
## Open questions
- [ ] **Tool:** rclone sync over SFTP (см. Decisions ниже — recommended). Alternative — restic (encrypted dedupe) — defer как для VDS; RUVDS↔kreknin link через public internet, но обе стороны ours, single-hop. Add encryption если threat model changes.
- [ ] **Retention:** 7 daily snapshots — следуем VDS-pattern. Per-day dirs (`/volume1/NetBackup/ruvds-iis/2026-05-25/`) — без hardlink dedup, 8.66 GB × 7 = ~60 GB на kreknin (5.7T free, easy).
- [ ] **Auth — RUVDS→kreknin SSH key:** генерировать новый ed25519 на RUVDS (`ssh-keygen` ships with OpenSSH client/server caps), pubkey добавить в `/volume1/homes/vitya/.ssh/authorized_keys` на kreknin через user. Private key хранится в `C:\ProgramData\backup\ssh_key` (ACL: SYSTEM + Administrators only).
- [ ] **Notification:** ntfy `vds-backup` topic — reuse (общий backup-канал, удобно для phone alerts). Title включает source-host чтобы отличать VDS-backup от RUVDS-backup.
- [ ] **Encryption-at-rest на kreknin:** none (как для VDS). RUVDS data = sites content (public web pages anyway) + config files. Не sensitive enough для encryption overhead в transitional setup.
## Implementation sketch
### 1. rclone install + config
```powershell
# Install rclone (one-time)
$ver = (Invoke-RestMethod 'https://downloads.rclone.org/version.txt').Trim('v').Trim()
$url = "https://downloads.rclone.org/v$ver/rclone-v$ver-windows-amd64.zip"
$tmp = "$env:TEMP\rclone.zip"
Invoke-WebRequest $url -OutFile $tmp -UseBasicParsing
Expand-Archive $tmp -DestinationPath "$env:TEMP\rclone" -Force
New-Item -ItemType Directory 'C:\Program Files\rclone' -Force | Out-Null
Copy-Item "$env:TEMP\rclone\rclone-*-windows-amd64\rclone.exe" 'C:\Program Files\rclone\' -Force
[Environment]::SetEnvironmentVariable('Path', "$env:Path;C:\Program Files\rclone", 'Machine')
# Generate SSH key для kreknin auth (one-time)
ssh-keygen -t ed25519 -f C:\ProgramData\backup\kreknin-key -N '""' -C 'ruvds-backup'
icacls C:\ProgramData\backup\kreknin-key /inheritance:r /grant 'SYSTEM:(F)' /grant 'Administrators:(F)'
# Затем user добавляет .pub в kreknin /volume1/homes/vitya/.ssh/authorized_keys
# rclone config — SFTP remote
@'
[kreknin]
type = sftp
host = 195.19.90.188
user = vitya
key_file = C:\ProgramData\backup\kreknin-key
disable_hashcheck = true
'@ | Set-Content C:\ProgramData\backup\rclone.conf -Encoding ASCII
```
### 2. Backup script `C:\ProgramData\backup\run.ps1`
```powershell
$ErrorActionPreference = 'Stop'
$today = Get-Date -Format 'yyyy-MM-dd'
$start = Get-Date
$logFile = "C:\ProgramData\backup\logs\$today.log"
New-Item -ItemType Directory (Split-Path $logFile) -Force | Out-Null
Start-Transcript -Path $logFile -Append -Force | Out-Null
try {
# 1. IIS config snapshot
Backup-WebConfiguration -Name "daily-$today" -Force | Out-Null
$iisBackupDir = "$env:SystemRoot\System32\inetsrv\backup\daily-$today"
# 2. Export cert store (LocalMachine\My) to PFX with timestamp passphrase
$pfxPass = ConvertTo-SecureString 'pfximport' -AsPlainText -Force # TODO move to pass-equivalent
$certDir = "C:\ProgramData\backup\certs-$today"
New-Item -ItemType Directory $certDir -Force | Out-Null
Get-ChildItem Cert:\LocalMachine\My | Where-Object { $_.HasPrivateKey } | ForEach-Object {
$name = ($_.Subject -replace '[^A-Za-z0-9.-]','_').Substring(0, [math]::Min(40, $_.Subject.Length))
try {
Export-PfxCertificate -Cert $_ -FilePath "$certDir\$name.pfx" -Password $pfxPass -ErrorAction SilentlyContinue | Out-Null
} catch {}
}
# 3. rclone sync — push to dated dir on kreknin
& 'C:\Program Files\rclone\rclone.exe' sync `
--config C:\ProgramData\backup\rclone.conf `
--log-file $logFile --log-level INFO `
--transfers 4 --checkers 8 `
C:\sites\snolla `
"kreknin:/volume1/NetBackup/ruvds-iis/$today/sites/snolla/"
if ($LASTEXITCODE -ne 0) { throw "rclone snolla failed" }
& 'C:\Program Files\rclone\rclone.exe' copy `
--config C:\ProgramData\backup\rclone.conf `
C:\Windows\System32\inetsrv\config\applicationHost.config `
"kreknin:/volume1/NetBackup/ruvds-iis/$today/iis-config/"
& 'C:\Program Files\rclone\rclone.exe' sync `
--config C:\ProgramData\backup\rclone.conf `
$iisBackupDir `
"kreknin:/volume1/NetBackup/ruvds-iis/$today/iis-backup-$today/"
& 'C:\Program Files\rclone\rclone.exe' sync `
--config C:\ProgramData\backup\rclone.conf `
$certDir `
"kreknin:/volume1/NetBackup/ruvds-iis/$today/certs/"
& 'C:\Program Files\rclone\rclone.exe' sync `
--config C:\ProgramData\backup\rclone.conf `
C:\ProgramData\ssh `
"kreknin:/volume1/NetBackup/ruvds-iis/$today/ssh-config/"
# 4. Retention prune — keep last 7 daily snapshots
& 'C:\Program Files\rclone\rclone.exe' lsd `
--config C:\ProgramData\backup\rclone.conf `
'kreknin:/volume1/NetBackup/ruvds-iis/' |
Where-Object { $_ -match '\d{4}-\d{2}-\d{2}' } |
ForEach-Object { ($_ -split '\s+')[-1] } |
Sort-Object | Select-Object -SkipLast 7 |
ForEach-Object {
& 'C:\Program Files\rclone\rclone.exe' purge `
--config C:\ProgramData\backup\rclone.conf `
"kreknin:/volume1/NetBackup/ruvds-iis/$_"
}
# 5. Cleanup local temp
Remove-Item $certDir -Recurse -Force
Remove-Item $iisBackupDir -Recurse -Force -ErrorAction SilentlyContinue
# 6. Notification — ntfy success
$duration = (New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
$msg = "RUVDS backup $today OK | $([math]::Round($duration,0))s"
$ntfyToken = (Get-Content C:\ProgramData\backup\ntfy-token.txt)
Invoke-RestMethod -Uri 'https://ntfy.vds.kzntsv.site/vds-backup' `
-Method POST `
-Headers @{ Authorization = "Bearer $ntfyToken"; Title = "RUVDS backup success"; Priority = 'default'; Tags = 'green_circle' } `
-Body $msg
} catch {
# Notification — ntfy failure
$msg = "RUVDS backup $today FAILED: $($_.Exception.Message)"
$ntfyToken = (Get-Content C:\ProgramData\backup\ntfy-token.txt -ErrorAction SilentlyContinue)
if ($ntfyToken) {
Invoke-RestMethod -Uri 'https://ntfy.vds.kzntsv.site/vds-backup' `
-Method POST `
-Headers @{ Authorization = "Bearer $ntfyToken"; Title = "RUVDS backup FAILED"; Priority = 'high'; Tags = 'red_circle' } `
-Body $msg
}
throw
} finally {
Stop-Transcript | Out-Null
}
```
### 3. Schedule via Windows Task Scheduler
```powershell
$action = New-ScheduledTaskAction -Execute 'powershell.exe' `
-Argument '-NoProfile -ExecutionPolicy Bypass -File C:\ProgramData\backup\run.ps1'
$trigger = New-ScheduledTaskTrigger -Daily -At '04:30'
$principal = New-ScheduledTaskPrincipal -UserId 'SYSTEM' -LogonType ServiceAccount -RunLevel Highest
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -StartWhenAvailable -MultipleInstances IgnoreNew
Register-ScheduledTask -TaskName 'RUVDS-Backup-Daily' `
-Action $action -Trigger $trigger -Principal $principal -Settings $settings -Force
```
## Decisions log
- **2026-05-24 (task creation):** Tool = **rclone**, not robocopy-over-SMB and not restic. Reasoning:
- rsync на Win Server Core отсутствует (no cygwin/WSL on Core). Установка cygwin/MSYS — лишний пакет, security surface.
- robocopy + SMB-mount kreknin требует Synology SMB включён + Windows credentials cached + UNC reliability over WAN — fragile.
- rclone — single .exe (~50 MB), SFTP backend native, reliable, well-supported on Server Core, log-friendly.
- restic — encrypted+dedupe — overkill пока link trusted; defer like VDS.
- **2026-05-24:** **Run as SYSTEM**, не пользовательский account. SYSTEM имеет full доступ к `C:\sites\` + `C:\Windows\System32\inetsrv\` + cert store, не требует stored-password в Task Scheduler. Pattern matches `[vds-backup-rsync-kreknin]` root-cron decision.
- **2026-05-24:** **PFX export `pfximport` password** — temporary, will move to pass-equivalent on RUVDS (или derived из машинной creds). Open question: how to handle pass-on-Windows-Core analog. Defer пока migration не cutover'нется полностью.
## Open questions (additional)
- [ ] **Когда стартовать:** сейчас (parallel с soak window `[iis-migration-to-ruvds]`) или после full cutover? Recommendation: **сейчас** — даже до full cutover, backup snolla content на RUVDS даёт rollback-evidence если RUVDS state corrupt'нется. Lateral risk minimal — backup идёт через ssh-key auth, кroil-кroil traffic.
- [ ] **kreknin SSH ACL on `/volume1/homes/vitya/`** — текущий ACL разрешает `vitya` user. RUVDS пушит как `vitya@kreknin` через SFTP — нужна добавка pubkey в authorized_keys. Без user action не сработает.
- [ ] **Backup `C:\sites\snolla` size growth** — CMS file uploads через админку могут вырасти со временем. Monitor disk usage на kreknin через quarterly check.
## Completed steps
- [x] **2026-05-24 ~10:18:** rclone v1.74.2 installed на RUVDS (`C:\Program Files\rclone\rclone.exe`).
- [x] **2026-05-24 ~10:18:** ed25519 SSH key generated `C:\ProgramData\backup\kreknin-key`, pubkey deployed в kreknin `/var/services/homes/vitya/.ssh/authorized_keys` через VDS pivot (source machine не имеет direct SSH к kreknin).
- [x] **2026-05-24 ~10:19:** `C:\ProgramData\backup\rclone.conf` (SFTP remote `kreknin`, disable_hashcheck), `config.env` (ntfy creds), `run.ps1` deployed; ACL = SYSTEM + Administrators.
- [x] **2026-05-24 10:19:47:** ScheduledTask `RUVDS-Backup-Daily` registered (daily 04:30 MSK, SYSTEM, 2h timeout).
- [x] **2026-05-24 10:21:13 11:42:57:** initial full sync 8.66 GB snolla site + 4 small components → kreknin (Завершилось ~82 мин из-за home-uplink throttling — production cron не affected since RUVDS uplink direct).
- [x] **2026-05-24 11:46:51:** run #1 incremental smoke ✓ 20 sec — все 5 components ok, retention prune ok (1 snapshot kept), ntfy success sent.
- [x] **2026-05-24:** Scripts checked into repo `scripts/ruvds-backup-daily-kreknin/` (setup.ps1 + run.ps1 + README).
## Closed
**2026-05-24 11:46:51** — пайплайн live, run #1 verified end-to-end. Закрывает Open question "Backup strategy для RUVDS IIS" в `[iis-migration-to-ruvds]`.
**Bugs fixed during smoke (зафиксированы в Decisions log):**
- `Backup-WebConfiguration -Force` параметра нет на этом IIS — используем `Get-WebConfigurationBackup + Remove-WebConfigurationBackup` если exists + plain `Backup-WebConfiguration`.
- rclone `--log-file` указывал на тот же путь что и `Start-Transcript` → file lock. Убрали `--log-file` (transcript captures всё необходимое).
- rclone NOTICE на stderr + `$ErrorActionPreference = 'Stop'` = PS terminating error даже c `2>$null`. Решение: `Invoke-Rclone` wrapper-функция temporarily switches к `ErrorActionPreference = 'Continue'`.
- ssh-keyscan'енный known_hosts не proходит rclone go-sftp library (key mismatch). Решение: убрать `known_hosts_file` из rclone.conf, полагаться на key-auth.
## Open follow-ups (не блокер)
- [ ] PFX export pass = `ruvds-backup-pfx` plaintext в `run.ps1` — temporary. TODO: pass-equivalent на Windows (gpg4win + bash-pass), или derived from machine-creds (DPAPI scoped to SYSTEM).
- [ ] Retention prune ещё не сработал (day 1 — 1 snapshot < 7). Verify в day 8 что purge action correct.
- [ ] Phone-side ntfy verify — user проверит что push с title `RUVDS backup OK` приходит на ntfy app, тот же channel что VDS backup.
## Notes
- **Триггер:** task создан 2026-05-24, можно стартовать paralleltno с soak'ом RUVDS. Не блокировать full DNS swap (если backup pipeline ломается — rollback на source IIS:8089 всегда возможен).
- **Integration:** ntfy topic `vds-backup` — общий с VDS backup. Title differentiator чтобы phone-side фильтровать.
- **Атомарный revert (откат всей задачи):**
```powershell
# На RUVDS:
Unregister-ScheduledTask -TaskName 'RUVDS-Backup-Daily' -Confirm:$false
Remove-Item C:\ProgramData\backup -Recurse -Force
# На kreknin: (через SSH)
ssh vitya@195.19.90.188 'rm -rf /volume1/NetBackup/ruvds-iis'
```
- **Relation to [iis-migration-to-ruvds]:** этот backup закрывает "Backup strategy для RUVDS IIS" open-question из migration task. Когда migration переходит в 🟢, эту задачу нужно явно отметить как dependency-closer.
- **Future evolution:** когда snolla-on-node будет ready и IIS станет obsolete (long-term goal per migration task notes) — этот backup можно retire вместе с RUVDS.
<!-- created-by: vitya / 2026-05-24 / trigger: post-iis-migration-partial-cutover, while-soak-window-open -->

View File

@@ -0,0 +1,87 @@
# ruvds-backup-daily-kreknin — scripts
Ежедневный backup RUVDS (`80.64.31.36` Win Server 2025 Core) → kreknin Synology (`195.19.90.188:/volume1/NetBackup/ruvds-iis/<date>/`) via rclone SFTP.
Live since **2026-05-24**. Schedule: daily **04:30 MSK**. Task name: `RUVDS-Backup-Daily`.
## Files
- `setup.ps1` — one-time install: SSH key, rclone, configs, ScheduledTask.
- `run.ps1` — backup logic (synced to `C:\ProgramData\backup\run.ps1` by setup.ps1).
## Initial install (rebuilding RUVDS from scratch)
```powershell
# На RUVDS, PowerShell as Administrator
mkdir C:\backup-scripts -Force
# Копируем оба файла из source repo:
Copy-Item \\tsclient\C\Users\vitya\projects\.admin\scripts\ruvds-backup-daily-kreknin\*.ps1 C:\backup-scripts\
C:\backup-scripts\setup.ps1
```
Setup паузится после генерации SSH key — нужно добавить `kreknin-key.pub` в `/volume1/homes/vitya/.ssh/authorized_keys` на kreknin (через SSH/DSM File Station), потом Enter.
## What's backed up (5 components)
| Component | Source | Remote path on kreknin |
|---|---|---|
| snolla site content | `C:\sites\snolla\` (~8.7 GB) | `NetBackup/ruvds-iis/<date>/sites/snolla/` |
| applicationHost.config | `C:\Windows\System32\inetsrv\config\applicationHost.config` | `NetBackup/ruvds-iis/<date>/iis-config/` |
| IIS native config backup | `Backup-WebConfiguration -Name daily-<date>` | `NetBackup/ruvds-iis/<date>/iis-backup-webconfiguration/` |
| LE certs (PFX export) | `Cert:\LocalMachine\My` private-key certs (14 PFX, pass `ruvds-backup-pfx`) | `NetBackup/ruvds-iis/<date>/certs/` |
| SSH state | `C:\ProgramData\ssh\` (sshd_config + administrators_authorized_keys) | `NetBackup/ruvds-iis/<date>/ssh-config/` |
Retention: **7 daily snapshots**. Older — pruned by `rclone purge` in step 4.
## Notifications
**Dual-channel** (oба отправляются после каждого run):
- **ntfy topic `vds-backup`** (shared с VDS backup) — phone push через ntfy mobile app:
- Success: title `RUVDS backup OK (<date>)`, tag `green_circle`, priority `default`
- Fail: title `RUVDS backup FAILED (<date>)`, tag `red_circle`, priority `high`
- **Email** via Yandex SMTP (`noreply@snolla.com``vitya.kuznetsov@gmail.com`):
- Port **587 STARTTLS** (.NET `SmtpClient` не дружит с implicit-TLS port 465; 587 STARTTLS работает с `Send-MailMessage -UseSsl`)
- Subject: `RUVDS backup <date> -- SUCCESS / FAILED`
- Body: duration, size, certs count, component list, log path
- SMTP creds в `config.env` (re-used из `pass show snolla-smtp/full-env`, тот же account что VDS msmtp)
Если email или ntfy fail — backup itself **не fail'ит** (notify-функции wrapped в try/catch, warning в лог).
## Decisions log
- **rclone, not rsync** — Win Server Core нет cygwin/WSL, rsync.exe = пакет, security surface. rclone single .exe.
- **SFTP без host-key validation** — rclone go-sftp library не parsит SSH known_hosts с keyscan format корректно (key mismatch error даже на fresh keyscan). Path RUVDS↔kreknin через public internet, но key auth достаточно для нашего threat model. Если threat model меняется — populate known_hosts file from `ssh-keyscan` в правильном format.
- **`Invoke-Rclone` wrapper** — `$ErrorActionPreference = 'Stop'` + rclone NOTICE на stderr = native command terminating-error в PS. Wrapper temporarily switches к Continue для rclone calls, restores after. Без этого `2>$null` не помогает (PS видит non-empty stderr stream как error).
- **`Backup-WebConfiguration -Force`** — параметра нет в этой версии WebAdministration module. Используем `Get-WebConfigurationBackup + Remove-WebConfigurationBackup` если exist + plain `Backup-WebConfiguration`.
- **SMTP port 587 STARTTLS, не 465 implicit-TLS** — .NET `SmtpClient` (и Powershell's `Send-MailMessage`) не поддерживает implicit-TLS на 465 (только STARTTLS). Yandex принимает обоих, выбрали 587 STARTTLS для compat с native PS tooling. VDS msmtp использует 465 implicit — оба валидны, разные tooling.
- **Local Get-ChildItem для size в email, не `rclone size --json`** — `Invoke-Rclone size --json` объединяет stdout JSON со stderr NOTICE, `ConvertFrom-Json` падает на mixed output. `Get-ChildItem C:\sites\snolla -Recurse | Measure -Sum Length` — local-disk-cost, instant.
- **Cert PFX export pass = `ruvds-backup-pfx`** — temporary. TODO: pass-equivalent на Windows (gpg4win + pass-bash) или derived от machine-creds.
- **SYSTEM principal, не user account** — Task Scheduler SYSTEM имеет full access к C:\sites\, Cert:\LocalMachine\My, %SystemRoot%\System32\inetsrv\, не требует stored-password. Pattern matches VDS root-cron decision.
## Smoke run (manual)
```powershell
Start-ScheduledTask -TaskName 'RUVDS-Backup-Daily'
# Watch:
Get-Content C:\ProgramData\backup\logs\(Get-Date -Format yyyy-MM-dd).log -Wait
```
Verify on kreknin:
```powershell
ssh vitya@195.19.90.188 'du -sh /volume1/NetBackup/ruvds-iis/*/sites/snolla/'
```
## Atomic revert (uninstall)
```powershell
# On RUVDS:
Unregister-ScheduledTask -TaskName 'RUVDS-Backup-Daily' -Confirm:$false
Remove-Item C:\ProgramData\backup -Recurse -Force
# Optionally: Remove-Item 'C:\Program Files\rclone' -Recurse -Force
# On kreknin (via SSH):
ssh vitya@195.19.90.188 'rm -rf /volume1/NetBackup/ruvds-iis'
# Remove the RUVDS pubkey line from authorized_keys (one line with 'ruvds-backup' comment).
```

View File

@@ -0,0 +1,176 @@
#requires -Version 5.1
[CmdletBinding()]
param()
$ErrorActionPreference = 'Stop'
$ProgressPreference = 'SilentlyContinue'
$today = Get-Date -Format 'yyyy-MM-dd'
$start = Get-Date
$base = 'C:\ProgramData\backup'
$logDir = "$base\logs"
$logFile = "$logDir\$today.log"
$rcloneExe = 'C:\Program Files\rclone\rclone.exe'
$rcloneCfg = "$base\rclone.conf"
$remoteBase = "kreknin:NetBackup/ruvds-iis"
$remoteToday = "$remoteBase/$today"
if (-not (Test-Path $logDir)) { New-Item -ItemType Directory -Path $logDir -Force | Out-Null }
Start-Transcript -Path $logFile -Append -Force | Out-Null
$cfg = @{}
if (Test-Path "$base\config.env") {
Get-Content "$base\config.env" | Where-Object { $_ -match '^[A-Z_]+=' } | ForEach-Object {
$kv = $_ -split '=', 2
$cfg[$kv[0]] = $kv[1]
}
}
function Notify-Ntfy($title, $msg, $priority='default', $tags='') {
try {
if (-not $cfg.NTFY_URL -or -not $cfg.NTFY_USER -or -not $cfg.NTFY_PASS) { return }
$pair = "$($cfg.NTFY_USER):$($cfg.NTFY_PASS)"
$auth = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($pair))
$topic = if ($cfg.NTFY_TOPIC) { $cfg.NTFY_TOPIC } else { 'vds-backup' }
Invoke-RestMethod -Uri "$($cfg.NTFY_URL)/$topic" -Method POST `
-Headers @{ Authorization=$auth; Title=$title; Priority=$priority; Tags=$tags } `
-Body $msg -ContentType 'text/plain' -ErrorAction SilentlyContinue | Out-Null
} catch { Write-Host " ntfy WARNING: $($_.Exception.Message)" -ForegroundColor Yellow }
}
function Notify-Email($subject, $body) {
try {
if (-not $cfg.SMTP_HOST -or -not $cfg.SMTP_USER -or -not $cfg.SMTP_PASS -or -not $cfg.OPS_NOTIFY_EMAIL) { return }
$secpass = ConvertTo-SecureString $cfg.SMTP_PASS -AsPlainText -Force
$mailCred = New-Object PSCredential($cfg.SMTP_USER, $secpass)
$prevEAP = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
try {
Send-MailMessage -SmtpServer $cfg.SMTP_HOST -Port ([int]$cfg.SMTP_PORT) -UseSsl `
-Credential $mailCred `
-From $cfg.SMTP_FROM -To $cfg.OPS_NOTIFY_EMAIL `
-Subject $subject -Body $body -Encoding UTF8 `
-WarningAction SilentlyContinue
} finally { $ErrorActionPreference = $prevEAP }
} catch { Write-Host " email WARNING: $($_.Exception.Message)" -ForegroundColor Yellow }
}
function Invoke-Rclone {
param([Parameter(ValueFromRemainingArguments=$true)][string[]]$Args)
$prevEAP = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
try { return & $rcloneExe @Args 2>&1 }
finally { $ErrorActionPreference = $prevEAP }
}
try {
Write-Host "=== RUVDS backup $today started at $start ==="
Write-Host "`n--- 1. IIS config snapshot ---"
Import-Module WebAdministration
$iisBackupName = "daily-$today"
$iisBackupDir = "$env:SystemRoot\System32\inetsrv\backup\$iisBackupName"
if (Get-WebConfigurationBackup -Name $iisBackupName -ErrorAction SilentlyContinue) {
Remove-WebConfigurationBackup -Name $iisBackupName
}
if (Test-Path $iisBackupDir) { Remove-Item $iisBackupDir -Recurse -Force }
Backup-WebConfiguration -Name $iisBackupName | Out-Null
Write-Host " $iisBackupDir created"
Write-Host "`n--- 2. Cert store export ---"
$certDir = "$base\certs-$today"
if (Test-Path $certDir) { Remove-Item $certDir -Recurse -Force }
New-Item -ItemType Directory -Path $certDir -Force | Out-Null
$pfxPass = ConvertTo-SecureString 'ruvds-backup-pfx' -AsPlainText -Force
$exported = 0
Get-ChildItem Cert:\LocalMachine\My | Where-Object { $_.HasPrivateKey } | ForEach-Object {
$cn = ($_.Subject -split ',')[0] -replace 'CN=','' -replace '[^A-Za-z0-9.-]','_'
if ($cn.Length -gt 60) { $cn = $cn.Substring(0,60) }
try {
Export-PfxCertificate -Cert $_ -FilePath "$certDir\$cn-$($_.Thumbprint.Substring(0,8)).pfx" -Password $pfxPass -ErrorAction SilentlyContinue | Out-Null
$exported++
} catch {}
}
Write-Host " exported $exported certs"
Write-Host "`n--- 3. rclone sync ---"
$rcCommon = @('--config', $rcloneCfg, '--transfers', '4', '--checkers', '8', '--stats=0')
Invoke-Rclone sync C:\sites\snolla "$remoteToday/sites/snolla/" @rcCommon | Out-Null
if ($LASTEXITCODE -ne 0) { throw "rclone sync snolla failed (exit $LASTEXITCODE)" }
Write-Host " snolla synced"
Invoke-Rclone copy 'C:\Windows\System32\inetsrv\config\applicationHost.config' "$remoteToday/iis-config/" @rcCommon | Out-Null
if ($LASTEXITCODE -ne 0) { throw "rclone copy applicationHost.config failed (exit $LASTEXITCODE)" }
Write-Host " applicationHost.config copied"
Invoke-Rclone sync $iisBackupDir "$remoteToday/iis-backup-webconfiguration/" @rcCommon | Out-Null
if ($LASTEXITCODE -ne 0) { throw "rclone sync iis-backup-webconfiguration failed (exit $LASTEXITCODE)" }
Write-Host " iis-backup-webconfiguration synced"
Invoke-Rclone sync $certDir "$remoteToday/certs/" @rcCommon | Out-Null
if ($LASTEXITCODE -ne 0) { throw "rclone sync certs failed (exit $LASTEXITCODE)" }
Write-Host " certs synced"
Invoke-Rclone sync C:\ProgramData\ssh "$remoteToday/ssh-config/" @rcCommon | Out-Null
if ($LASTEXITCODE -ne 0) { throw "rclone sync ssh-config failed (exit $LASTEXITCODE)" }
Write-Host " ssh-config synced"
Write-Host "`n--- 4. Retention prune (keep last 7) ---"
try {
$lsdOut = Invoke-Rclone lsd $remoteBase --config $rcloneCfg
$existing = $lsdOut | ForEach-Object {
$line = "$_".Trim()
if ($line -match '\s(\d{4}-\d{2}-\d{2})\s*$') { $Matches[1] }
} | Sort-Object -Unique
$toPrune = @($existing | Select-Object -SkipLast 7)
foreach ($d in $toPrune) {
Write-Host " pruning $d"
Invoke-Rclone purge "$remoteBase/$d" --config $rcloneCfg | Out-Null
}
Write-Host " kept $([math]::Min(@($existing).Count, 7)) snapshots; pruned $(@($toPrune).Count)"
} catch {
Write-Host " retention prune WARNING: $($_.Exception.Message)" -ForegroundColor Yellow
}
Remove-Item $certDir -Recurse -Force -ErrorAction SilentlyContinue
$duration = [int](New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
$srcBytes = (Get-ChildItem 'C:\sites\snolla' -Recurse -File -EA SilentlyContinue | Measure-Object Length -Sum).Sum
$totalGB = '{0:N2}' -f ($srcBytes / 1GB)
$msg = "RUVDS daily backup $today OK | $duration sec | size $totalGB GB | $exported certs"
Notify-Ntfy "RUVDS backup OK ($today)" $msg 'default' 'green_circle'
$emailBody = @"
RUVDS daily backup completed successfully.
Date: $today
Duration: $duration seconds
Size: $totalGB GB
Certs: $exported PFX
Source: RUVDS ($env:COMPUTERNAME / 80.64.31.36)
Dest: kreknin:/volume1/NetBackup/ruvds-iis/$today/
Components synced:
- sites/snolla
- iis-config (applicationHost.config)
- iis-backup-webconfiguration
- certs (LE PFX exports)
- ssh-config
Log: $logFile
"@
Notify-Email "RUVDS backup $today -- SUCCESS" $emailBody
Write-Host "`n=== DONE in $duration sec ==="
} catch {
$err = $_.Exception.Message
Write-Host "`n=== FAILED: $err ===" -ForegroundColor Red
Write-Host $_.ScriptStackTrace
$duration = [int](New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
Notify-Ntfy "RUVDS backup FAILED ($today)" "After $duration sec: $err" 'high' 'red_circle'
Notify-Email "RUVDS backup $today -- FAILED" "After $duration sec: $err`n`nLog: $logFile"
Stop-Transcript | Out-Null
exit 1
} finally {
try { Stop-Transcript | Out-Null } catch {}
}

View File

@@ -0,0 +1,121 @@
#requires -Version 5.1
#requires -RunAsAdministrator
<#
.SYNOPSIS
One-time setup для RUVDS daily backup → kreknin.
Run on RUVDS as Administrator. Idempotent.
.DESCRIPTION
1. Generate ed25519 SSH key (`C:\ProgramData\backup\kreknin-key`).
2. Install rclone v1.74+ to `C:\Program Files\rclone\rclone.exe`.
3. Write rclone.conf (SFTP remote `kreknin` без host-key validation —
see Decisions log).
4. Write config.env с ntfy creds (chmod-equiv via icacls SYSTEM+Administrators).
5. Deploy run.ps1 (companion file в this folder).
6. Register ScheduledTask `RUVDS-Backup-Daily` daily @ 04:30 MSK as SYSTEM.
User action required between steps 1 и 6: add `kreknin-key.pub` to
`/volume1/homes/vitya/.ssh/authorized_keys` on kreknin Synology.
#>
[CmdletBinding()]
param(
[string]$NtfyUser = 'vitya',
[string]$NtfyPass = 'Pryakhin9',
[string]$NtfyUrl = 'https://ntfy.vds.kzntsv.site',
[string]$NtfyTopic = 'vds-backup',
[string]$SmtpHost = 'smtp.yandex.ru',
[int]$SmtpPort = 587,
[string]$SmtpFrom = 'noreply@snolla.com',
[string]$SmtpUser = 'noreply@snolla.com',
[string]$SmtpPass = 'JP44ajqZ3K#W',
[string]$OpsNotifyEmail = 'vitya.kuznetsov@gmail.com',
[string]$ScheduleAt = '04:30'
)
$ErrorActionPreference = 'Stop'
$base = 'C:\ProgramData\backup'
New-Item -ItemType Directory -Path $base -Force | Out-Null
# 1. SSH key
$keyPath = "$base\kreknin-key"
if (-not (Test-Path $keyPath)) {
ssh-keygen -t ed25519 -f $keyPath -N '""' -C 'ruvds-backup' -q
& cmd.exe /c "icacls `"$keyPath`" /inheritance:r /grant `"SYSTEM:(F)`" /grant `"BUILTIN\Administrators:(F)`"" | Out-Null
Write-Host "[OK] SSH key generated: $keyPath"
Write-Host "ADD TO KREKNIN authorized_keys:"
Get-Content "$keyPath.pub"
Read-Host "Press Enter when pubkey added to kreknin"
} else {
Write-Host "[SKIP] SSH key exists"
}
# 2. rclone
$rcloneExe = 'C:\Program Files\rclone\rclone.exe'
if (-not (Test-Path $rcloneExe)) {
$ver = (Invoke-RestMethod 'https://downloads.rclone.org/version.txt' -UseBasicParsing).Trim() -replace '^rclone\s+v|^v',''
$url = "https://downloads.rclone.org/v$ver/rclone-v$ver-windows-amd64.zip"
$tmp = "$env:TEMP\rclone.zip"
Invoke-WebRequest $url -OutFile $tmp -UseBasicParsing
if (Test-Path "$env:TEMP\rclone") { Remove-Item "$env:TEMP\rclone" -Recurse -Force }
Expand-Archive $tmp -DestinationPath "$env:TEMP\rclone" -Force
New-Item -ItemType Directory 'C:\Program Files\rclone' -Force | Out-Null
Copy-Item "$env:TEMP\rclone\rclone-*-windows-amd64\rclone.exe" $rcloneExe -Force
Write-Host "[OK] rclone installed: v$ver"
} else {
Write-Host "[SKIP] rclone installed: $(& $rcloneExe --version | Select-Object -First 1)"
}
# 3. rclone.conf — no known_hosts_file (rclone go-sftp lib чувствителен к формату; ssh-key auth = trust enough)
@"
[kreknin]
type = sftp
host = 195.19.90.188
user = vitya
key_file = $keyPath
disable_hashcheck = true
"@ | Set-Content "$base\rclone.conf" -Encoding ASCII -Force
& cmd.exe /c "icacls `"$base\rclone.conf`" /inheritance:r /grant `"SYSTEM:(F)`" /grant `"BUILTIN\Administrators:(F)`"" | Out-Null
Write-Host "[OK] rclone.conf written"
# 4. config.env (ntfy + SMTP creds)
@"
NTFY_URL=$NtfyUrl
NTFY_TOPIC=$NtfyTopic
NTFY_USER=$NtfyUser
NTFY_PASS=$NtfyPass
SMTP_HOST=$SmtpHost
SMTP_PORT=$SmtpPort
SMTP_FROM=$SmtpFrom
SMTP_USER=$SmtpUser
SMTP_PASS=$SmtpPass
OPS_NOTIFY_EMAIL=$OpsNotifyEmail
"@ | Set-Content "$base\config.env" -Encoding ASCII -Force
& cmd.exe /c "icacls `"$base\config.env`" /inheritance:r /grant `"SYSTEM:(F)`" /grant `"BUILTIN\Administrators:(F)`"" | Out-Null
Write-Host "[OK] config.env written"
# 5. Deploy run.ps1 (assumed alongside this setup.ps1)
$srcRun = Join-Path $PSScriptRoot 'run.ps1'
if (-not (Test-Path $srcRun)) { throw "run.ps1 not found alongside setup.ps1 ($srcRun)" }
Copy-Item $srcRun "$base\run.ps1" -Force
& cmd.exe /c "icacls `"$base\run.ps1`" /inheritance:r /grant `"SYSTEM:(F)`" /grant `"BUILTIN\Administrators:(F)`"" | Out-Null
Write-Host "[OK] run.ps1 deployed"
# 6. Scheduled Task
$taskName = 'RUVDS-Backup-Daily'
if (Get-ScheduledTask -TaskName $taskName -ErrorAction SilentlyContinue) {
Unregister-ScheduledTask -TaskName $taskName -Confirm:$false
}
$action = New-ScheduledTaskAction -Execute 'powershell.exe' `
-Argument "-NoProfile -ExecutionPolicy Bypass -File $base\run.ps1"
$trigger = New-ScheduledTaskTrigger -Daily -At $ScheduleAt
$principal = New-ScheduledTaskPrincipal -UserId 'SYSTEM' -LogonType ServiceAccount -RunLevel Highest
$settings = New-ScheduledTaskSettingsSet `
-AllowStartIfOnBatteries -DontStopIfGoingOnBatteries `
-StartWhenAvailable -MultipleInstances IgnoreNew `
-ExecutionTimeLimit (New-TimeSpan -Hours 2)
Register-ScheduledTask -TaskName $taskName -Action $action -Trigger $trigger -Principal $principal -Settings $settings -Force | Out-Null
Write-Host "[OK] ScheduledTask '$taskName' registered ($ScheduleAt daily, SYSTEM, 2h max)"
Write-Host ""
Write-Host "=== Setup complete. Test with: Start-ScheduledTask -TaskName '$taskName' ==="
Write-Host "Log: $base\logs\<date>.log"