admin/.tasks/ruvds-backup-daily-kreknin.md

ruvds-backup-daily-kreknin

Goal

Ежедневный backup с RUVDS (80.64.31.36 / Win Server 2025 Core) → ../entities/kreknin-synology (195.19.90.188 / kreknin.site) в 04:30 MSK (за 30 мин до [vds-backup-rsync-kreknin] 05:00, чтобы не пересекать uplink). После каждого прохода — ntfy push на topic vds-backup (тот же что VDS backup пушит — общий agg-канал).

Параллель [vds-backup-rsync-kreknin] 🟢 (VDS Ubuntu → kreknin via rsync --link-dest), но source = Windows Server Core, поэтому:

• Cron нет → Windows Task Scheduler
• rsync нативного нет → rclone sync over SFTP (см. Decisions ниже)
• root-cron-as-uid недоступен → SYSTEM scheduled task

Scope (what to backup)

Минимум для disaster-recovery RUVDS state:

Path on RUVDS	Reason	Approx size
C:\sites\snolla\	CMS site content + Web.config + media	8.66 GB (grows slowly)
C:\Windows\System32\inetsrv\config\applicationHost.config	IIS site/binding/apppool config	<1 MB
Backup-WebConfiguration export	IIS native config snapshot (Backup-WebConfiguration -Name ...) → %SystemRoot%\System32\inetsrv\backup\	<5 MB
C:\ProgramData\ssh\ (sshd_config + administrators_authorized_keys)	SSH access state	<50 KB
LE certs export (Get-ChildItem Cert:\LocalMachine\My → Export-PfxCertificate с временным пасс'ом + git-tracked)	для restore без re-extract из traefik acme.json	<1 MB

Не бэкапить:

• C:\Windows\ system files — RUVDS rebuild = fresh install (~5 мин через RUVDS panel).
• C:\Program Files\ — installed software re-installable from scripts (bootstrap.ps1).
• IIS logs (C:\inetpub\logs\LogFiles\) — диагностические, не recovery-critical. Optional.

Open questions

• Tool: rclone sync over SFTP (см. Decisions ниже — recommended). Alternative — restic (encrypted dedupe) — defer как для VDS; RUVDS↔kreknin link через public internet, но обе стороны ours, single-hop. Add encryption если threat model changes.
• Retention: 7 daily snapshots — следуем VDS-pattern. Per-day dirs (/volume1/NetBackup/ruvds-iis/2026-05-25/) — без hardlink dedup, 8.66 GB × 7 = ~60 GB на kreknin (5.7T free, easy).
• Auth — RUVDS→kreknin SSH key: генерировать новый ed25519 на RUVDS (ssh-keygen ships with OpenSSH client/server caps), pubkey добавить в /volume1/homes/vitya/.ssh/authorized_keys на kreknin через user. Private key хранится в C:\ProgramData\backup\ssh_key (ACL: SYSTEM + Administrators only).
• Notification: ntfy vds-backup topic — reuse (общий backup-канал, удобно для phone alerts). Title включает source-host чтобы отличать VDS-backup от RUVDS-backup.
• Encryption-at-rest на kreknin: none (как для VDS). RUVDS data = sites content (public web pages anyway) + config files. Не sensitive enough для encryption overhead в transitional setup.

Implementation sketch

1. rclone install + config

# Install rclone (one-time)
$ver = (Invoke-RestMethod 'https://downloads.rclone.org/version.txt').Trim('v').Trim()
$url = "https://downloads.rclone.org/v$ver/rclone-v$ver-windows-amd64.zip"
$tmp = "$env:TEMP\rclone.zip"
Invoke-WebRequest $url -OutFile $tmp -UseBasicParsing
Expand-Archive $tmp -DestinationPath "$env:TEMP\rclone" -Force
New-Item -ItemType Directory 'C:\Program Files\rclone' -Force | Out-Null
Copy-Item "$env:TEMP\rclone\rclone-*-windows-amd64\rclone.exe" 'C:\Program Files\rclone\' -Force
[Environment]::SetEnvironmentVariable('Path', "$env:Path;C:\Program Files\rclone", 'Machine')

# Generate SSH key для kreknin auth (one-time)
ssh-keygen -t ed25519 -f C:\ProgramData\backup\kreknin-key -N '""' -C 'ruvds-backup'
icacls C:\ProgramData\backup\kreknin-key /inheritance:r /grant 'SYSTEM:(F)' /grant 'Administrators:(F)'
# Затем user добавляет .pub в kreknin /volume1/homes/vitya/.ssh/authorized_keys

# rclone config — SFTP remote
@'
[kreknin]
type = sftp
host = 195.19.90.188
user = vitya
key_file = C:\ProgramData\backup\kreknin-key
disable_hashcheck = true
'@ | Set-Content C:\ProgramData\backup\rclone.conf -Encoding ASCII

2. Backup script C:\ProgramData\backup\run.ps1

$ErrorActionPreference = 'Stop'
$today = Get-Date -Format 'yyyy-MM-dd'
$start = Get-Date
$logFile = "C:\ProgramData\backup\logs\$today.log"
New-Item -ItemType Directory (Split-Path $logFile) -Force | Out-Null
Start-Transcript -Path $logFile -Append -Force | Out-Null

try {
    # 1. IIS config snapshot
    Backup-WebConfiguration -Name "daily-$today" -Force | Out-Null
    $iisBackupDir = "$env:SystemRoot\System32\inetsrv\backup\daily-$today"

    # 2. Export cert store (LocalMachine\My) to PFX with timestamp passphrase
    $pfxPass = ConvertTo-SecureString 'pfximport' -AsPlainText -Force  # TODO move to pass-equivalent
    $certDir = "C:\ProgramData\backup\certs-$today"
    New-Item -ItemType Directory $certDir -Force | Out-Null
    Get-ChildItem Cert:\LocalMachine\My | Where-Object { $_.HasPrivateKey } | ForEach-Object {
        $name = ($_.Subject -replace '[^A-Za-z0-9.-]','_').Substring(0, [math]::Min(40, $_.Subject.Length))
        try {
            Export-PfxCertificate -Cert $_ -FilePath "$certDir\$name.pfx" -Password $pfxPass -ErrorAction SilentlyContinue | Out-Null
        } catch {}
    }

    # 3. rclone sync — push to dated dir on kreknin
    & 'C:\Program Files\rclone\rclone.exe' sync `
        --config C:\ProgramData\backup\rclone.conf `
        --log-file $logFile --log-level INFO `
        --transfers 4 --checkers 8 `
        C:\sites\snolla `
        "kreknin:/volume1/NetBackup/ruvds-iis/$today/sites/snolla/"
    if ($LASTEXITCODE -ne 0) { throw "rclone snolla failed" }

    & 'C:\Program Files\rclone\rclone.exe' copy `
        --config C:\ProgramData\backup\rclone.conf `
        C:\Windows\System32\inetsrv\config\applicationHost.config `
        "kreknin:/volume1/NetBackup/ruvds-iis/$today/iis-config/"

    & 'C:\Program Files\rclone\rclone.exe' sync `
        --config C:\ProgramData\backup\rclone.conf `
        $iisBackupDir `
        "kreknin:/volume1/NetBackup/ruvds-iis/$today/iis-backup-$today/"

    & 'C:\Program Files\rclone\rclone.exe' sync `
        --config C:\ProgramData\backup\rclone.conf `
        $certDir `
        "kreknin:/volume1/NetBackup/ruvds-iis/$today/certs/"

    & 'C:\Program Files\rclone\rclone.exe' sync `
        --config C:\ProgramData\backup\rclone.conf `
        C:\ProgramData\ssh `
        "kreknin:/volume1/NetBackup/ruvds-iis/$today/ssh-config/"

    # 4. Retention prune — keep last 7 daily snapshots
    & 'C:\Program Files\rclone\rclone.exe' lsd `
        --config C:\ProgramData\backup\rclone.conf `
        'kreknin:/volume1/NetBackup/ruvds-iis/' |
        Where-Object { $_ -match '\d{4}-\d{2}-\d{2}' } |
        ForEach-Object { ($_ -split '\s+')[-1] } |
        Sort-Object | Select-Object -SkipLast 7 |
        ForEach-Object {
            & 'C:\Program Files\rclone\rclone.exe' purge `
                --config C:\ProgramData\backup\rclone.conf `
                "kreknin:/volume1/NetBackup/ruvds-iis/$_"
        }

    # 5. Cleanup local temp
    Remove-Item $certDir -Recurse -Force
    Remove-Item $iisBackupDir -Recurse -Force -ErrorAction SilentlyContinue

    # 6. Notification — ntfy success
    $duration = (New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
    $msg = "RUVDS backup $today OK | $([math]::Round($duration,0))s"
    $ntfyToken = (Get-Content C:\ProgramData\backup\ntfy-token.txt)
    Invoke-RestMethod -Uri 'https://ntfy.vds.kzntsv.site/vds-backup' `
        -Method POST `
        -Headers @{ Authorization = "Bearer $ntfyToken"; Title = "RUVDS backup success"; Priority = 'default'; Tags = 'green_circle' } `
        -Body $msg

} catch {
    # Notification — ntfy failure
    $msg = "RUVDS backup $today FAILED: $($_.Exception.Message)"
    $ntfyToken = (Get-Content C:\ProgramData\backup\ntfy-token.txt -ErrorAction SilentlyContinue)
    if ($ntfyToken) {
        Invoke-RestMethod -Uri 'https://ntfy.vds.kzntsv.site/vds-backup' `
            -Method POST `
            -Headers @{ Authorization = "Bearer $ntfyToken"; Title = "RUVDS backup FAILED"; Priority = 'high'; Tags = 'red_circle' } `
            -Body $msg
    }
    throw
} finally {
    Stop-Transcript | Out-Null
}

3. Schedule via Windows Task Scheduler

$action = New-ScheduledTaskAction -Execute 'powershell.exe' `
    -Argument '-NoProfile -ExecutionPolicy Bypass -File C:\ProgramData\backup\run.ps1'
$trigger = New-ScheduledTaskTrigger -Daily -At '04:30'
$principal = New-ScheduledTaskPrincipal -UserId 'SYSTEM' -LogonType ServiceAccount -RunLevel Highest
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -StartWhenAvailable -MultipleInstances IgnoreNew
Register-ScheduledTask -TaskName 'RUVDS-Backup-Daily' `
    -Action $action -Trigger $trigger -Principal $principal -Settings $settings -Force

Decisions log

• 2026-05-24 (task creation): Tool = rclone, not robocopy-over-SMB and not restic. Reasoning:

  • rsync на Win Server Core отсутствует (no cygwin/WSL on Core). Установка cygwin/MSYS — лишний пакет, security surface.
  • robocopy + SMB-mount kreknin требует Synology SMB включён + Windows credentials cached + UNC reliability over WAN — fragile.
  • rclone — single .exe (~50 MB), SFTP backend native, reliable, well-supported on Server Core, log-friendly.
  • restic — encrypted+dedupe — overkill пока link trusted; defer like VDS.

• 2026-05-24: Run as SYSTEM, не пользовательский account. SYSTEM имеет full доступ к C:\sites\ + C:\Windows\System32\inetsrv\ + cert store, не требует stored-password в Task Scheduler. Pattern matches [vds-backup-rsync-kreknin] root-cron decision.

• 2026-05-24: PFX export pfximport password — temporary, will move to pass-equivalent on RUVDS (или derived из машинной creds). Open question: how to handle pass-on-Windows-Core analog. Defer пока migration не cutover'нется полностью.

Open questions (additional)

• Когда стартовать: сейчас (parallel с soak window [iis-migration-to-ruvds]) или после full cutover? Recommendation: сейчас — даже до full cutover, backup snolla content на RUVDS даёт rollback-evidence если RUVDS state corrupt'нется. Lateral risk minimal — backup идёт через ssh-key auth, кroil-кroil traffic.
• kreknin SSH ACL on /volume1/homes/vitya/ — текущий ACL разрешает vitya user. RUVDS пушит как vitya@kreknin через SFTP — нужна добавка pubkey в authorized_keys. Без user action не сработает.
• Backup C:\sites\snolla size growth — CMS file uploads через админку могут вырасти со временем. Monitor disk usage на kreknin через quarterly check.

Completed steps

• 2026-05-24 ~10:18: rclone v1.74.2 installed на RUVDS (C:\Program Files\rclone\rclone.exe).
• 2026-05-24 ~10:18: ed25519 SSH key generated C:\ProgramData\backup\kreknin-key, pubkey deployed в kreknin /var/services/homes/vitya/.ssh/authorized_keys через VDS pivot (source machine не имеет direct SSH к kreknin).
• 2026-05-24 ~10:19: C:\ProgramData\backup\rclone.conf (SFTP remote kreknin, disable_hashcheck), config.env (ntfy creds), run.ps1 deployed; ACL = SYSTEM + Administrators.
• 2026-05-24 10:19:47: ScheduledTask RUVDS-Backup-Daily registered (daily 04:30 MSK, SYSTEM, 2h timeout).
• 2026-05-24 10:21:13 – 11:42:57: initial full sync 8.66 GB snolla site + 4 small components → kreknin (Завершилось ~82 мин из-за home-uplink throttling — production cron не affected since RUVDS uplink direct).
• 2026-05-24 11:46:51: run #1 incremental smoke ✓ 20 sec — все 5 components ok, retention prune ok (1 snapshot kept), ntfy success sent.
• 2026-05-24: Scripts checked into repo scripts/ruvds-backup-daily-kreknin/ (setup.ps1 + run.ps1 + README).

Closed

2026-05-24 11:46:51 — пайплайн live, run #1 verified end-to-end. Закрывает Open question "Backup strategy для RUVDS IIS" в [iis-migration-to-ruvds].

Bugs fixed during smoke (зафиксированы в Decisions log):

• Backup-WebConfiguration -Force параметра нет на этом IIS — используем Get-WebConfigurationBackup + Remove-WebConfigurationBackup если exists + plain Backup-WebConfiguration.
• rclone --log-file указывал на тот же путь что и Start-Transcript → file lock. Убрали --log-file (transcript captures всё необходимое).
• rclone NOTICE на stderr + $ErrorActionPreference = 'Stop' = PS terminating error даже c 2>$null. Решение: Invoke-Rclone wrapper-функция temporarily switches к ErrorActionPreference = 'Continue'.
• ssh-keyscan'енный known_hosts не proходит rclone go-sftp library (key mismatch). Решение: убрать known_hosts_file из rclone.conf, полагаться на key-auth.

Open follow-ups (не блокер)

• PFX export pass = ruvds-backup-pfx plaintext в run.ps1 — temporary. TODO: pass-equivalent на Windows (gpg4win + bash-pass), или derived from machine-creds (DPAPI scoped to SYSTEM).
• Retention prune ещё не сработал (day 1 — 1 snapshot < 7). Verify в day 8 что purge action correct.
• Phone-side ntfy verify — user проверит что push с title RUVDS backup OK приходит на ntfy app, тот же channel что VDS backup.

Notes

• Триггер: task создан 2026-05-24, можно стартовать paralleltno с soak'ом RUVDS. Не блокировать full DNS swap (если backup pipeline ломается — rollback на source IIS:8089 всегда возможен).
• Integration: ntfy topic vds-backup — общий с VDS backup. Title differentiator чтобы phone-side фильтровать.
• Атомарный revert (откат всей задачи):

  # На RUVDS:
  Unregister-ScheduledTask -TaskName 'RUVDS-Backup-Daily' -Confirm:$false
  Remove-Item C:\ProgramData\backup -Recurse -Force
  # На kreknin: (через SSH)
  ssh vitya@195.19.90.188 'rm -rf /volume1/NetBackup/ruvds-iis'
  

• Relation to [iis-migration-to-ruvds]: этот backup закрывает "Backup strategy для RUVDS IIS" open-question из migration task. Когда migration переходит в 🟢, эту задачу нужно явно отметить как dependency-closer.
• Future evolution: когда snolla-on-node будет ready и IIS станет obsolete (long-term goal per migration task notes) — этот backup можно retire вместе с RUVDS.