Commit Graph

12 Commits

Author SHA1 Message Date
063910e265 incident(books-vds-es): true RCA — ransom-бот через открытый :9200, не оператор
Рецидив вчерашнего ES-инцидента вскрыл истинную причину. Вчерашняя
гипотеза «оператор в cutover попал на canonical» опровергнута.

Root cause: ES (stack 33) публиковал 0.0.0.0:9200 мимо traefik. Free-ES
7.10 без auth → порт открыт всему интернету. Ransom-бот сносил индексы
by-name (мимо Control #1 destructive_requires_name), оставлял read_me с
BTC-выкупом. accessLog (Control #2) пуст — бот шёл прямо в порт, не через
traefik. firewalld бесполезен (docker-publish обходит INPUT-зоны).

Fix (Control #3): убрана публикация host-порта из stack 33 (Portainer
PUT), дыра закрыта; re-restore epz/products/artmone из daily-2026-05-25.

Отдельный баг: epz-поиск падал у ОБОИХ тенантов — getTenantIdSeller(
config.get("tenant")) через node-config, а tenant не задан ни в
default.json, ни в env-маппинге (TENANT env = мёртвый груз). Добавлен
tenant в overlay default.json (slovo/bookva). products работал —
отдельный код-путь.

accessLog откатан (сторожил не ту дверь).

- wiki concept: correction-блок + секция «Рецидив 2026-05-29» + exposure-audit
- tasks: restore-es reopened+reclosed; new  harden-books-vds-exposed-ports
- NEXT_SESSION handoff

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 09:31:25 +03:00
48a5cf397e tasks(restore-es-indices-books-vds): closed — snapshot restore + 2 preventive controls
snapshot restore из kreknin:daily-2026-05-25 за 46 сек (epz=820604,
products=105922, artmone=2621, counts == source).

RCA: 5 индексов (включая system .tasks) удалены через ES API DELETE _all
за 1 сек на 2026-05-26 10:21 UTC — 1ч 11мин после создания bookva-es в
cutover-prep. Каноничный endpoint elasticsearch.kzntsv.site попал под
команду которая предназначалась bookva-es:9200 (internal-only, без
traefik route). Caller identity unrecoverable: ES audit = X-Pack платный,
traefik accessLog был выключен, Portainer CE без audit.

Preventive controls applied + verified:
1. ES env action.destructive_requires_name=true (stack 33) — DELETE _all
   и wildcard теперь 400 BadRequest; by-name DELETE работает (нужно для
   reindex). Pattern удаления что случился физически невозможен.
2. Traefik JSON accessLog в /letsencrypt/access.log — будущие DELETE
   оставят forensic след с IP/user/method/path.

Wiki concept: .wiki/concepts/es-destructive-delete-incident-2026-05-26.md
с recovery runbook + preventive controls + cross-refs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 20:11:07 +03:00
11554d45ac wiki(ingest): vds-kzntsv network-stack mismatch RCA + ifupdown anti-pattern
Incident 2026-05-28 ~05:45–13:52 MSK на vds-kzntsv. ~2.5ч активного outage
+ ~5.5ч на эфемерной статике до окончательного fix хостером.

Revised RCA: наш netplan+systemd-networkd конфликтовал с provider's expected
ifupdown stack. Их start/ipadd procedure ожидает чистый ifupdown и не может
auto-recover когда networkd «держит» eth0. 8 дней работало потому что
networkd сам тянул DHCP. Когда что-то на стороне Rusonyx разорвало
DHCP-binding — auto-recovery не сработала.

Fix: systemctl mask netplan + systemd-networkd* (на running system без stop —
IP и SSH сохранились), Rusonyx ребутнул VM и положил чистый
/etc/network/interfaces.d/ifcfg-eth0 через свой start/ipadd. Netmask /18,
gw 89.253.192.1, чистый ifupdown.

Wiki:
- NEW concepts/vds-kzntsv-dhcp-outage-2026-05-28 — full RCA + recovery
  runbook (эфемерная статика + permanent-fix via ifupdown) + diagnostic
  dot-graph + revised lessons-learned + anti-pattern
- NEW sources/vds-kzntsv-incident-2026-05-28 — timeline 05:25 backup OK →
  08:43 statics → 13:52 final reset; provider's ifcfg-eth0 content; ticket
  text reference
- UPDATE entities/vds-kzntsv — mask /18, ifupdown stack, kernel cmdline
  net.ifnames=0 объясняет eth0 naming, hypervisor hw80, pass-store путь,
  Known issues §
- UPDATE concepts/rusonyx-vps-onboarding-quirks — quirk #9 переписан про
  /18 layout + canonical ifcfg, quirk #10 NEW про ifupdown vs netplan
  stack choice + bootstrap mask commands
- UPDATE index.md + log.md

Tasks:
- STATUS header: incident RESOLVED summary
- NEXT_SESSION: следующая сессия — cleanup netplan-artifacts (optional),
  registry GC (~20G pending), board-viewer-build unhealthy разбор

Memory (out-of-repo): vds-kzntsv-rusonyx-network-recovery переписан с
revised RCA — canonical bootstrap step «mask netplan/networkd» для всех
Rusonyx Ubuntu VDS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 14:12:14 +03:00
22786e1865 wiki(ingest): RUVDS IIS migration + daily backup pipeline
- entities/ruvds-iis-host (NEW) — 80.64.31.36, Win Server 2025 Core,
  25 SNI HTTPS bindings, 2/24 hostnames DNS-flipped
- sources/iis-migration-to-ruvds-2026-05-23 (NEW) — chronology,
  SSH/scp pivot после home-ISP outbound 445 block
- sources/ruvds-backup-daily-kreknin-2026-05-24 (NEW) — rclone+SFTP
  SYSTEM task daily 04:30, ntfy общий канал
- concepts/traefik-acme-json-to-iis-cert-import (NEW) — PFX + SNI
  recipe
- concepts/windows-server-2025-core-bootstrap — SMB deprecate,
  HTTP middlebox warning, HTTP/2 note; backup/cert open-Qs закрыты
- entities/windows-recovery-host — partial-cutover state,
  imgproxy SPOF carve-out, tandemmebel indefinitely здесь
- overview / index / log — catalog refresh

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 23:31:14 +03:00
b7b8e27a27 tasks(books-ssh-audit-shared-vds): close 🟢 — audit clean (1 retained, 0 revoked)
SSH audit на shared VDS vds-kzntsv (89.253.255.94, hosts books Slovo +
Bookva в shared compose-стеке) перед Phase 3 cutover'ом tenant-split.

Findings:
- 1 retained key: vitya@DESKTOP-NSEF0UK (core dev, sole admin)
- 0 keys to revoke — никаких analyst / former employee / unknown keys
- 1 cosmetic cleanup: removed dead root authorized_keys entry (was
  duplicate of vitya's key, dead из-за `permitrootlogin no`)
- sshd hardening verified via `sshd -T` (effective config: root-no,
  password-no, kbd-no). **Gotcha** noted: raw grep of /etc/ssh/sshd_config
  shows defaults; sshd_config.d/ overrides делают effective. Future
  audits use `sshd -T`, not raw grep.
- fail2ban active, 2670 failed / 37 banned hist, currently 0
- last 7 days journalctl ssh: только vitya@94.19.247.14 (мой home IP)

Acceptance per spec (Фаза 3, шаг 2 tenant-split):
 authorized_keys reviewed
 Non-core keys revoked (N/A — none existed)
 Analyst keys revoked (N/A — never issued)
 Documented in .wiki/concepts/books-ssh-access.md

Ingest: .wiki/concepts/books-ssh-access.md — retained keys table +
sshd state + fail2ban + login history + add/revoke processes +
cross-refs. Logged + indexed.

STATUS.md  block → ARCHIVE.md (per-task file kept in .tasks/).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 15:59:50 +03:00
fdefe96d6f tasks(iis-migration-to-ruvds): pause после failed-robocopy + ingest SMB-default finding
Temp admin (2026-05-23 09:37-18:08) начал импл iis-migration-to-ruvds,
не закончил bootstrap, упал на UNC robocopy (exit 16) и dropped tree dirty.

Session recovery:
- secrets leak fix: `.secrets/ruvds-iis.env` → `pass show ruvds-iis/full-env`
  (etap-2 discipline restored). `.secrets/` + `*.env` + `*-log.txt` + `*-size.txt`
  added to `.gitignore` чтобы не повторилось.
- root cause зафиксирован: TCP/445 closed по дефолту на fresh Win Server 2025
  Core + SMB share не создан → UNC robocopy не работает без RUVDS bootstrap.
- new concept `windows-server-2025-core-bootstrap.md` — default-blockers
  table + transfer-методов матрица (RDP-redirect / SMB / WinRM / SFTP) +
  bootstrap-чеклист 12 шагов. Recommendation = SMB inbound с source-IP
  whitelist.
- task 🟡 paused с concrete next-step (capacity audit, transfer-method
  confirm, RUVDS bootstrap, backup source, recreate IIS sites + conn-string
  swap, pilot kupimknigi + DNS swap).
- Open question raised: source 11 sites сумма vs RUVDS 30 GB HDD — capacity
  blocker possible (snolla одна 8.66 GB).

Не push'нуто — ждёт user grant per project-discipline Rule 4.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 22:07:34 +03:00
68706d0bee tasks(owncloud-vds-deploy): close 🟢 — 26 GB live + 60s timeout finding
Import complete: 25 044 objs / 26 GB. 99.97% залито rclone'ом (22.2 GB
за ~2.5 ч), оставшиеся 6 файлов (4.04 GB) — 4× .pat ~221 MB + 2× .seospider
1.55+1.7 GB — упали с 502/500 на каждом rclone PUT из-за 60-секундного
HTTP timeout в reva v2.27 datagateway (hardcoded в Go http.Client.Timeout,
не env-configurable).

Workaround: VDS-side curl PUT loopback через throwaway sftp key.
Local network 142 MB/s → 4.04 GB за 28 секунд, все PROPFIND size match.

Wiki: §Gotcha 5 в concepts/ocis-on-vds-deploy-recipe.md документирует
finding + recipe для workaround. Traefik buffering middleware пробован,
не помог (likely vulcand/oxy buffer bug на больших телах) — откатил.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 07:26:07 +03:00
74cf219533 wiki(concepts): ingest ocis-on-vds-deploy-recipe + entity refresh
New concept page documents the oCIS-on-VDS deploy pattern with the 4
non-obvious gotchas surfaced during owncloud-vds-deploy:

  1. UID mismatch — image ocis-user is 1000, host vitya is 1001 →
     compose `user: "1001:1001"` override (alt chown rejected as orphan-UID)
  2. PROXY_TLS=false — explicit per docs when reverse proxy terminates HTTPS
  3. PROXY_ENABLE_BASIC_AUTH=true — required for WebDAV/LibreGraph API
  4. LibreGraph POST /graph/v1.0/users — only way to create users (no CLI)

Plus decomposedfs storage layout, atomic-revert recipe, backup integration
note. Source: .tasks/owncloud-vds-deploy.md.

entities/vds-kzntsv.md updated: stack table + hostnames + file layout
include owncloud row.

index.md catalog + log.md ingest entry refreshed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 19:29:43 +03:00
38f1d3358a wiki(index): refresh catalog after subtree import — 6 entities, 18 concepts, 3 sources
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:52:32 +03:00
d558ccfaed chore: mark admin-infra-project-pointers done, unblock morecms-subtree-split
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:37:10 +03:00
4c35e2a925 meta(wiki): log += ingest concepts/admin-infra-project 2026-05-21 10:21:29 +00:00
34e779e002 bootstrap: admin project skeleton (CLAUDE.md, .wiki/, .tasks/) 2026-05-21 13:17:45 +03:00