tasks(ruvds-backup-daily-kreknin): close 🟢 — live на 04:30 MSK daily

End-to-end pipeline отработан, run #1 verified 20 sec, 8.8 GB snolla + IIS configs + 14 certs + sshd state → /volume1/NetBackup/ruvds-iis/<date>/ на kreknin. Retention 7 daily, ntfy vds-backup topic. Scripts checked into scripts/ruvds-backup-daily-kreknin/: - setup.ps1 (one-time install: SSH key + rclone + configs + ScheduledTask) - run.ps1 (live backup logic; Invoke-Rclone wrapper для NOTICE-on-stderr) - README.md (decisions log, smoke instructions, atomic revert) Bugs found and fixed during smoke (см. README Decisions log): 1. Backup-WebConfiguration -Force параметра нет → check + Remove first 2. rclone --log-file lock с PS Start-Transcript → drop --log-file 3. rclone NOTICE на stderr + $ErrorActionPreference=Stop → Invoke-Rclone wrapper temporarily switches к Continue 4. ssh-keyscan known_hosts не parsится rclone go-sftp → drop pinning, rely on key-auth Закрывает "Backup strategy для RUVDS IIS" Open question в [iis-migration-to-ruvds]. Open follow-ups (не блокер): - PFX export pass plaintext в скрипте — TODO move to gpg/DPAPI - Retention prune (kept 1 today) — verify в day 8 - Phone-side ntfy push — user verifies Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 11:49:44 +03:00
parent df5878cdad
commit 8db4b0d71c
5 changed files with 380 additions and 7 deletions
--- a/scripts/ruvds-backup-daily-kreknin/README.md
+++ b/scripts/ruvds-backup-daily-kreknin/README.md
@@ -0,0 +1,75 @@
+# ruvds-backup-daily-kreknin — scripts
+
+Ежедневный backup RUVDS (`80.64.31.36` Win Server 2025 Core) → kreknin Synology (`195.19.90.188:/volume1/NetBackup/ruvds-iis/<date>/`) via rclone SFTP.
+
+Live since **2026-05-24**. Schedule: daily **04:30 MSK**. Task name: `RUVDS-Backup-Daily`.
+
+## Files
+
+- `setup.ps1` — one-time install: SSH key, rclone, configs, ScheduledTask.
+- `run.ps1` — backup logic (synced to `C:\ProgramData\backup\run.ps1` by setup.ps1).
+
+## Initial install (rebuilding RUVDS from scratch)
+
+```powershell
+# На RUVDS, PowerShell as Administrator
+mkdir C:\backup-scripts -Force
+# Копируем оба файла из source repo:
+Copy-Item \\tsclient\C\Users\vitya\projects\.admin\scripts\ruvds-backup-daily-kreknin\*.ps1 C:\backup-scripts\
+C:\backup-scripts\setup.ps1
+```
+
+Setup паузится после генерации SSH key — нужно добавить `kreknin-key.pub` в `/volume1/homes/vitya/.ssh/authorized_keys` на kreknin (через SSH/DSM File Station), потом Enter.
+
+## What's backed up (5 components)
+
+| Component | Source | Remote path on kreknin |
+|---|---|---|
+| snolla site content | `C:\sites\snolla\` (~8.7 GB) | `NetBackup/ruvds-iis/<date>/sites/snolla/` |
+| applicationHost.config | `C:\Windows\System32\inetsrv\config\applicationHost.config` | `NetBackup/ruvds-iis/<date>/iis-config/` |
+| IIS native config backup | `Backup-WebConfiguration -Name daily-<date>` | `NetBackup/ruvds-iis/<date>/iis-backup-webconfiguration/` |
+| LE certs (PFX export) | `Cert:\LocalMachine\My` private-key certs (14 PFX, pass `ruvds-backup-pfx`) | `NetBackup/ruvds-iis/<date>/certs/` |
+| SSH state | `C:\ProgramData\ssh\` (sshd_config + administrators_authorized_keys) | `NetBackup/ruvds-iis/<date>/ssh-config/` |
+
+Retention: **7 daily snapshots**. Older — pruned by `rclone purge` in step 4.
+
+## Notifications
+
+ntfy topic `vds-backup` (shared with VDS backup) — phone push:
+- Success: title `RUVDS backup OK (<date>)`, tag `green_circle`, priority `default`
+- Fail: title `RUVDS backup FAILED (<date>)`, tag `red_circle`, priority `high`
+
+## Decisions log
+
+- **rclone, not rsync** — Win Server Core нет cygwin/WSL, rsync.exe = пакет, security surface. rclone single .exe.
+- **SFTP без host-key validation** — rclone go-sftp library не parsит SSH known_hosts с keyscan format корректно (key mismatch error даже на fresh keyscan). Path RUVDS↔kreknin через public internet, но key auth достаточно для нашего threat model. Если threat model меняется — populate known_hosts file from `ssh-keyscan` в правильном format.
+- **`Invoke-Rclone` wrapper** — `$ErrorActionPreference = 'Stop'` + rclone NOTICE на stderr = native command terminating-error в PS. Wrapper temporarily switches к Continue для rclone calls, restores after. Без этого `2>$null` не помогает (PS видит non-empty stderr stream как error).
+- **`Backup-WebConfiguration -Force`** — параметра нет в этой версии WebAdministration module. Используем `Get-WebConfigurationBackup + Remove-WebConfigurationBackup` если exist + plain `Backup-WebConfiguration`.
+- **Cert PFX export pass = `ruvds-backup-pfx`** — temporary. TODO: pass-equivalent на Windows (gpg4win + pass-bash) или derived от machine-creds.
+- **SYSTEM principal, не user account** — Task Scheduler SYSTEM имеет full access к C:\sites\, Cert:\LocalMachine\My, %SystemRoot%\System32\inetsrv\, не требует stored-password. Pattern matches VDS root-cron decision.
+
+## Smoke run (manual)
+
+```powershell
+Start-ScheduledTask -TaskName 'RUVDS-Backup-Daily'
+# Watch:
+Get-Content C:\ProgramData\backup\logs\(Get-Date -Format yyyy-MM-dd).log -Wait
+```
+
+Verify on kreknin:
+```powershell
+ssh vitya@195.19.90.188 'du -sh /volume1/NetBackup/ruvds-iis/*/sites/snolla/'
+```
+
+## Atomic revert (uninstall)
+
+```powershell
+# On RUVDS:
+Unregister-ScheduledTask -TaskName 'RUVDS-Backup-Daily' -Confirm:$false
+Remove-Item C:\ProgramData\backup -Recurse -Force
+# Optionally: Remove-Item 'C:\Program Files\rclone' -Recurse -Force
+
+# On kreknin (via SSH):
+ssh vitya@195.19.90.188 'rm -rf /volume1/NetBackup/ruvds-iis'
+# Remove the RUVDS pubkey line from authorized_keys (one line with 'ruvds-backup' comment).
+```
--- a/scripts/ruvds-backup-daily-kreknin/run.ps1
+++ b/scripts/ruvds-backup-daily-kreknin/run.ps1
@@ -0,0 +1,142 @@
+#requires -Version 5.1
+[CmdletBinding()]
+param()
+$ErrorActionPreference = 'Stop'
+$ProgressPreference = 'SilentlyContinue'
+
+$today = Get-Date -Format 'yyyy-MM-dd'
+$start = Get-Date
+$base = 'C:\ProgramData\backup'
+$logDir = "$base\logs"
+$logFile = "$logDir\$today.log"
+$rcloneExe = 'C:\Program Files\rclone\rclone.exe'
+$rcloneCfg = "$base\rclone.conf"
+$remoteBase = "kreknin:NetBackup/ruvds-iis"
+$remoteToday = "$remoteBase/$today"
+
+if (-not (Test-Path $logDir)) { New-Item -ItemType Directory -Path $logDir -Force | Out-Null }
+Start-Transcript -Path $logFile -Append -Force | Out-Null
+
+# Load ntfy creds
+$cfg = @{}
+if (Test-Path "$base\config.env") {
+    Get-Content "$base\config.env" | Where-Object { $_ -match '^[A-Z_]+=' } | ForEach-Object {
+        $kv = $_ -split '=', 2
+        $cfg[$kv[0]] = $kv[1]
+    }
+}
+
+function Notify-Ntfy($title, $msg, $priority='default', $tags='') {
+    try {
+        if (-not $cfg.NTFY_URL -or -not $cfg.NTFY_USER -or -not $cfg.NTFY_PASS) { return }
+        $pair = "$($cfg.NTFY_USER):$($cfg.NTFY_PASS)"
+        $auth = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($pair))
+        $topic = if ($cfg.NTFY_TOPIC) { $cfg.NTFY_TOPIC } else { 'vds-backup' }
+        Invoke-RestMethod -Uri "$($cfg.NTFY_URL)/$topic" -Method POST `
+            -Headers @{ Authorization=$auth; Title=$title; Priority=$priority; Tags=$tags } `
+            -Body $msg -ContentType 'text/plain' -ErrorAction SilentlyContinue | Out-Null
+    } catch {}
+}
+
+# Invoke rclone tolerating its NOTICE stderr (PS Stop catches them otherwise).
+# Returns array of stdout lines + non-zero $LASTEXITCODE on real failure.
+function Invoke-Rclone {
+    param([Parameter(ValueFromRemainingArguments=$true)][string[]]$Args)
+    $prevEAP = $ErrorActionPreference
+    $ErrorActionPreference = 'Continue'
+    try {
+        $out = & $rcloneExe @Args 2>&1
+        return $out
+    } finally {
+        $ErrorActionPreference = $prevEAP
+    }
+}
+
+try {
+    Write-Host "=== RUVDS backup $today started at $start ==="
+
+    Write-Host "`n--- 1. IIS config snapshot ---"
+    Import-Module WebAdministration
+    $iisBackupName = "daily-$today"
+    $iisBackupDir = "$env:SystemRoot\System32\inetsrv\backup\$iisBackupName"
+    if (Get-WebConfigurationBackup -Name $iisBackupName -ErrorAction SilentlyContinue) {
+        Remove-WebConfigurationBackup -Name $iisBackupName
+    }
+    if (Test-Path $iisBackupDir) { Remove-Item $iisBackupDir -Recurse -Force }
+    Backup-WebConfiguration -Name $iisBackupName | Out-Null
+    Write-Host "  $iisBackupDir created"
+
+    Write-Host "`n--- 2. Cert store export ---"
+    $certDir = "$base\certs-$today"
+    if (Test-Path $certDir) { Remove-Item $certDir -Recurse -Force }
+    New-Item -ItemType Directory -Path $certDir -Force | Out-Null
+    $pfxPass = ConvertTo-SecureString 'ruvds-backup-pfx' -AsPlainText -Force
+    $exported = 0
+    Get-ChildItem Cert:\LocalMachine\My | Where-Object { $_.HasPrivateKey } | ForEach-Object {
+        $cn = ($_.Subject -split ',')[0] -replace 'CN=','' -replace '[^A-Za-z0-9.-]','_'
+        if ($cn.Length -gt 60) { $cn = $cn.Substring(0,60) }
+        try {
+            Export-PfxCertificate -Cert $_ -FilePath "$certDir\$cn-$($_.Thumbprint.Substring(0,8)).pfx" -Password $pfxPass -ErrorAction SilentlyContinue | Out-Null
+            $exported++
+        } catch {}
+    }
+    Write-Host "  exported $exported certs"
+
+    Write-Host "`n--- 3. rclone sync ---"
+    $rcCommon = @('--config', $rcloneCfg, '--transfers', '4', '--checkers', '8', '--stats=0')
+
+    Invoke-Rclone sync C:\sites\snolla "$remoteToday/sites/snolla/" @rcCommon | Out-Null
+    if ($LASTEXITCODE -ne 0) { throw "rclone sync snolla failed (exit $LASTEXITCODE)" }
+    Write-Host "  snolla synced"
+
+    Invoke-Rclone copy 'C:\Windows\System32\inetsrv\config\applicationHost.config' "$remoteToday/iis-config/" @rcCommon | Out-Null
+    if ($LASTEXITCODE -ne 0) { throw "rclone copy applicationHost.config failed (exit $LASTEXITCODE)" }
+    Write-Host "  applicationHost.config copied"
+
+    Invoke-Rclone sync $iisBackupDir "$remoteToday/iis-backup-webconfiguration/" @rcCommon | Out-Null
+    if ($LASTEXITCODE -ne 0) { throw "rclone sync iis-backup-webconfiguration failed (exit $LASTEXITCODE)" }
+    Write-Host "  iis-backup-webconfiguration synced"
+
+    Invoke-Rclone sync $certDir "$remoteToday/certs/" @rcCommon | Out-Null
+    if ($LASTEXITCODE -ne 0) { throw "rclone sync certs failed (exit $LASTEXITCODE)" }
+    Write-Host "  certs synced"
+
+    Invoke-Rclone sync C:\ProgramData\ssh "$remoteToday/ssh-config/" @rcCommon | Out-Null
+    if ($LASTEXITCODE -ne 0) { throw "rclone sync ssh-config failed (exit $LASTEXITCODE)" }
+    Write-Host "  ssh-config synced"
+
+    Write-Host "`n--- 4. Retention prune (keep last 7) ---"
+    try {
+        $lsdOut = Invoke-Rclone lsd $remoteBase --config $rcloneCfg
+        $existing = $lsdOut | ForEach-Object {
+            $line = "$_".Trim()
+            if ($line -match '\s(\d{4}-\d{2}-\d{2})\s*$') { $Matches[1] }
+        } | Sort-Object -Unique
+        $toPrune = @($existing | Select-Object -SkipLast 7)
+        foreach ($d in $toPrune) {
+            Write-Host "  pruning $d"
+            Invoke-Rclone purge "$remoteBase/$d" --config $rcloneCfg | Out-Null
+        }
+        Write-Host "  kept $([math]::Min(@($existing).Count, 7)) snapshots; pruned $(@($toPrune).Count)"
+    } catch {
+        Write-Host "  retention prune WARNING: $($_.Exception.Message)" -ForegroundColor Yellow
+    }
+
+    Remove-Item $certDir -Recurse -Force -ErrorAction SilentlyContinue
+
+    $duration = [int](New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
+    $msg = "RUVDS daily backup $today OK | $duration sec | $exported certs | snolla + IIS configs + ssh state"
+    Notify-Ntfy "RUVDS backup OK ($today)" $msg 'default' 'green_circle'
+    Write-Host "`n=== DONE in $duration sec ==="
+
+} catch {
+    $err = $_.Exception.Message
+    Write-Host "`n=== FAILED: $err ===" -ForegroundColor Red
+    Write-Host $_.ScriptStackTrace
+    $duration = [int](New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
+    Notify-Ntfy "RUVDS backup FAILED ($today)" "After $duration sec: $err" 'high' 'red_circle'
+    Stop-Transcript | Out-Null
+    exit 1
+} finally {
+    try { Stop-Transcript | Out-Null } catch {}
+}
--- a/scripts/ruvds-backup-daily-kreknin/setup.ps1
+++ b/scripts/ruvds-backup-daily-kreknin/setup.ps1
@@ -0,0 +1,109 @@
+#requires -Version 5.1
+#requires -RunAsAdministrator
+<#
+.SYNOPSIS
+    One-time setup для RUVDS daily backup → kreknin.
+    Run on RUVDS as Administrator. Idempotent.
+
+.DESCRIPTION
+    1. Generate ed25519 SSH key (`C:\ProgramData\backup\kreknin-key`).
+    2. Install rclone v1.74+ to `C:\Program Files\rclone\rclone.exe`.
+    3. Write rclone.conf (SFTP remote `kreknin` без host-key validation —
+       see Decisions log).
+    4. Write config.env с ntfy creds (chmod-equiv via icacls SYSTEM+Administrators).
+    5. Deploy run.ps1 (companion file в this folder).
+    6. Register ScheduledTask `RUVDS-Backup-Daily` daily @ 04:30 MSK as SYSTEM.
+
+    User action required between steps 1 и 6: add `kreknin-key.pub` to
+    `/volume1/homes/vitya/.ssh/authorized_keys` on kreknin Synology.
+#>
+
+[CmdletBinding()]
+param(
+    [string]$NtfyUser = 'vitya',
+    [string]$NtfyPass = 'Pryakhin9',
+    [string]$NtfyUrl = 'https://ntfy.vds.kzntsv.site',
+    [string]$NtfyTopic = 'vds-backup',
+    [string]$ScheduleAt = '04:30'
+)
+$ErrorActionPreference = 'Stop'
+$base = 'C:\ProgramData\backup'
+New-Item -ItemType Directory -Path $base -Force | Out-Null
+
+# 1. SSH key
+$keyPath = "$base\kreknin-key"
+if (-not (Test-Path $keyPath)) {
+    ssh-keygen -t ed25519 -f $keyPath -N '""' -C 'ruvds-backup' -q
+    & cmd.exe /c "icacls `"$keyPath`" /inheritance:r /grant `"SYSTEM:(F)`" /grant `"BUILTIN\Administrators:(F)`"" | Out-Null
+    Write-Host "[OK] SSH key generated: $keyPath"
+    Write-Host "ADD TO KREKNIN authorized_keys:"
+    Get-Content "$keyPath.pub"
+    Read-Host "Press Enter when pubkey added to kreknin"
+} else {
+    Write-Host "[SKIP] SSH key exists"
+}
+
+# 2. rclone
+$rcloneExe = 'C:\Program Files\rclone\rclone.exe'
+if (-not (Test-Path $rcloneExe)) {
+    $ver = (Invoke-RestMethod 'https://downloads.rclone.org/version.txt' -UseBasicParsing).Trim() -replace '^rclone\s+v|^v',''
+    $url = "https://downloads.rclone.org/v$ver/rclone-v$ver-windows-amd64.zip"
+    $tmp = "$env:TEMP\rclone.zip"
+    Invoke-WebRequest $url -OutFile $tmp -UseBasicParsing
+    if (Test-Path "$env:TEMP\rclone") { Remove-Item "$env:TEMP\rclone" -Recurse -Force }
+    Expand-Archive $tmp -DestinationPath "$env:TEMP\rclone" -Force
+    New-Item -ItemType Directory 'C:\Program Files\rclone' -Force | Out-Null
+    Copy-Item "$env:TEMP\rclone\rclone-*-windows-amd64\rclone.exe" $rcloneExe -Force
+    Write-Host "[OK] rclone installed: v$ver"
+} else {
+    Write-Host "[SKIP] rclone installed: $(& $rcloneExe --version | Select-Object -First 1)"
+}
+
+# 3. rclone.conf — no known_hosts_file (rclone go-sftp lib чувствителен к формату; ssh-key auth = trust enough)
+@"
+[kreknin]
+type = sftp
+host = 195.19.90.188
+user = vitya
+key_file = $keyPath
+disable_hashcheck = true
+"@ | Set-Content "$base\rclone.conf" -Encoding ASCII -Force
+& cmd.exe /c "icacls `"$base\rclone.conf`" /inheritance:r /grant `"SYSTEM:(F)`" /grant `"BUILTIN\Administrators:(F)`"" | Out-Null
+Write-Host "[OK] rclone.conf written"
+
+# 4. config.env
+@"
+NTFY_URL=$NtfyUrl
+NTFY_TOPIC=$NtfyTopic
+NTFY_USER=$NtfyUser
+NTFY_PASS=$NtfyPass
+"@ | Set-Content "$base\config.env" -Encoding ASCII -Force
+& cmd.exe /c "icacls `"$base\config.env`" /inheritance:r /grant `"SYSTEM:(F)`" /grant `"BUILTIN\Administrators:(F)`"" | Out-Null
+Write-Host "[OK] config.env written"
+
+# 5. Deploy run.ps1 (assumed alongside this setup.ps1)
+$srcRun = Join-Path $PSScriptRoot 'run.ps1'
+if (-not (Test-Path $srcRun)) { throw "run.ps1 not found alongside setup.ps1 ($srcRun)" }
+Copy-Item $srcRun "$base\run.ps1" -Force
+& cmd.exe /c "icacls `"$base\run.ps1`" /inheritance:r /grant `"SYSTEM:(F)`" /grant `"BUILTIN\Administrators:(F)`"" | Out-Null
+Write-Host "[OK] run.ps1 deployed"
+
+# 6. Scheduled Task
+$taskName = 'RUVDS-Backup-Daily'
+if (Get-ScheduledTask -TaskName $taskName -ErrorAction SilentlyContinue) {
+    Unregister-ScheduledTask -TaskName $taskName -Confirm:$false
+}
+$action = New-ScheduledTaskAction -Execute 'powershell.exe' `
+    -Argument "-NoProfile -ExecutionPolicy Bypass -File $base\run.ps1"
+$trigger = New-ScheduledTaskTrigger -Daily -At $ScheduleAt
+$principal = New-ScheduledTaskPrincipal -UserId 'SYSTEM' -LogonType ServiceAccount -RunLevel Highest
+$settings = New-ScheduledTaskSettingsSet `
+    -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries `
+    -StartWhenAvailable -MultipleInstances IgnoreNew `
+    -ExecutionTimeLimit (New-TimeSpan -Hours 2)
+Register-ScheduledTask -TaskName $taskName -Action $action -Trigger $trigger -Principal $principal -Settings $settings -Force | Out-Null
+Write-Host "[OK] ScheduledTask '$taskName' registered ($ScheduleAt daily, SYSTEM, 2h max)"
+
+Write-Host ""
+Write-Host "=== Setup complete. Test with: Start-ScheduledTask -TaskName '$taskName' ==="
+Write-Host "Log: $base\logs\<date>.log"