Commit Graph

332 Commits

Author SHA1 Message Date
1eb58aa940 task(decommission): windows-recovery-host cleanup complete (C: 62->275GB)
Elevated finale ran clean: stostayer.old repointed to mssql.kzntsv.site
(orphaned snolla user fixed agent-side as sa), smoke 200; removed snolla
IIS site + C:\sites\snolla + local MSSQL. Both kept sites verified 200
(stostayer :8090 external DB, stostayer.old :8091 on VDS).

Updated entities/windows-recovery-host with post-decommission state
(old hosting sections marked historical); image-pipeline SPOF retired.
Remaining: WSL vhdx compact (~176GB, outside session).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:15:12 +03:00
b686ac132f fix(decommission): smoke :8091 via curl --noproxy; idempotent F2
The F3 smoke false-failed: system VPN proxy swallows Invoke-WebRequest
to localhost (wiki bug #7), returning empty -> gate aborted before E/M.
Verified out-of-band: :8091 -> HTTP 200 (real stostayer.old page), so
the repoint actually succeeded. Switched smoke to curl.exe --noproxy.
Made F2 idempotent (only repoint if still localhost) so re-runs don't
clobber the original localhost backup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:15:12 +03:00
52098c821b fix(decommission): orphan fix done agent-side; drop F1/sa from script
Verified `pass` decrypts in the agent bash env; applied the VDS
orphaned-user fix (snolla@stostayer -> mapped) directly as sa, password
passed via env var (never echoed). Verified snolla can now open VDS
stostayer (128 tables). finish-elevated.ps1 no longer needs sa/pass or
any prompt - it only runs the elevated IIS/web.config/docker steps with
a snolla->stostayer pre-gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:15:12 +03:00
eca4f24e11 fix(decommission): prompt for sa password instead of bash/pass in elevated PS
In an elevated shell `bash` resolves to WSL bash (not Git bash where
`pass` lives); its stderr proxy warning + ErrorActionPreference=Stop
aborted the script at the pass-fetch line. Replaced with a hidden
Read-Host prompt; set ErrorActionPreference=Continue around the docker
teardown so native stderr can't abort it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:15:12 +03:00
21e3d4266c fix(decommission): rewrite finish-elevated.ps1 ASCII-only (PS5.1 codepage)
Cyrillic comments saved as UTF-8-no-BOM were mis-parsed by Windows
PowerShell 5.1 (read as cp1251) -> parser error at "(~28 GB)". Rewrote
the script with ASCII-only text; verified 0 non-ASCII bytes + clean AST
parse. Also removed traefik teardown step from the script (local traefik
already removed per user decision); kept lightrag + mutable-dev VM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:15:12 +03:00
e6e2479970 task(decommission): clean up windows-recovery-host post-Synology, phases A-D
Audited the local recovery host: services migrated to VDS, content left
behind. Reclaimed phases (non-elevated, done):
- docker image+builder prune: images 118->7.1GB, cache 65->0GB (~176GB
  logical; physical reclaim pending WSL vhdx compact)
- removed redundant local CMS-backend containers (minio/imgproxy/es)
- removed 21 empty husk stacks + leftover data dirs
- deleted snolla-recovery VBox VM (95GB) + C:\nas-recovery (~150GB)
- deleted C:\sites snolla archives + identity-manager
=> C: free 62GB -> 214GB

Elevated finale (E+F) scripted in finish-elevated.ps1 for the user to run:
stostayer.old repoint to mssql.kzntsv.site (with VDS orphaned-user fix),
smoke-gated; then snolla IIS site + content + local MSSQL teardown (~38GB).
Keeps IIS stostayer + stostayer.old per user.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:15:12 +03:00
2811b5a180 fix(iis-migration): maljarka.tandemmebel.ru 502-on-HTTPS resolved
Root cause (NOT a migration defect): MoreThenCms tenant `maljarka`
had `dbo.Sites.SettingsData = NULL` (no `httpSecure` block) -> on a
real HTTPS request SnollaMiddleware throws KeyNotFoundException ->
502. HTTP served 200 fine. DNS/TLS/IIS binding all correct.

Fix applied to shared MSSQL (mssql.kzntsv.site): UPDATE Sites set
SettingsData with httpSecure{enableHttps:true,...} + recycle snolla
pool. Verified 443->200 server-local and external via 80.64.31.36;
kupimknigi untouched. Audit: maljarka was the only NULL-settings
site with a :443 binding (of 25); rimiz degraded for another reason.

- new concept: morethencms-null-settingsdata-https-502
- entities/ruvds-iis-host: maljarka moved from Degraded -> fixed
- index.md + log.md + STATUS.md + NEXT_SESSION.md updated

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:15:12 +03:00
67fb97fbba meta(tasks): update [agents-task-runner-vds-deploy] in OpeItcLoc03/admin 2026-06-08 12:06:21 +00:00
99dba5138f meta(tasks): update [agents-task-runner-vds-deploy] in OpeItcLoc03/admin 2026-06-08 10:25:19 +00:00
a704f9d313 meta(tasks): create [agents-task-runner-vds-deploy] in OpeItcLoc03/admin 2026-06-08 10:10:17 +00:00
2094f90e31 wiki(nl-vds-3xui): +3 Shadowsocks inbounds for Outline-app
3 classic chacha20-ietf-poly1305 SS inbounds (ports 32031/32/33),
one ss:// key each, per-key revocable. Protocol e2e-tested OK;
RF reachability unverified (SS DPI-blocked in RF on this node).
Keys/passwords in pass nl-vds-3xui/full-env.
Also flagged undocumented VLESS:13027 inbound found in DB.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 09:34:01 +03:00
3c7ca3416b feat(iis-migration): win-acme HTTP-01 auto-renewal on RUVDS IIS
Stood up a permanent self-renewing Let's Encrypt pipeline on the RUVDS
IIS host, replacing the manual traefik acme.json -> PFX import and
closing the 2026-07-22 cert-expiry deadline (new 25-SAN cert valid to
2026-09-03, SYSTEM scheduled task renews 55 days before expiry).

Key obstacle: the MoreThenCms OWIN catch-all (owin:HandleAllRequests)
swallowed /.well-known/acme-challenge/. Solved by carving the challenge
path into a separate IIS application in a No-Managed-Code app pool, plus
patching win-acme's Web_Config.xml template to remove the inherited Owin
handler. Staging + prod validation green for all 25 hostnames; live TLS
smoke confirms the new cert is served (incl degraded maljarka/rimiz).

- scripts/iis-migration-to-ruvds/03-ruvds-winacme.ps1 (idempotent setup)
- scripts/iis-migration-to-ruvds/winacme-Web_Config.xml (patched template)
- .wiki/concepts/winacme-iis-owin-catchall-http01.md (recipe + gotchas)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 21:08:41 +03:00
9cd6d4e22c task(iis-migration): full DNS cutover reached 2026-06-05
All 25 hostnames (tandemmebel + entire snolla.com zone) now resolve to
RUVDS (80.64.31.36) authoritatively on ns1.reg.ru; nothing left on
windows-source authoritatively. Google cache tail draining for
on./tandemmebel.snolla.com. Source kept as warm rollback per user
decision; decommission + win-acme LE renewal pending soak.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 20:29:45 +03:00
7bba8cf853 meta(handoff): tandemmebel DNS cutover 2026-06-05
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 20:11:53 +03:00
539f1a07ed task(iis-migration): tandemmebel DNS cutover 2026-06-05
tandemmebel.ru/www/maljarka resolve to RUVDS (80.64.31.36) on
authoritative + all public resolvers. RUVDS serves 200 OK; maljarka
502 is pre-existing CMS defect, not a cutover regression. Supersedes
the 2026-05-24 scope-exception. Source left as warm rollback per
user decision (shared catch-all, 11 *.snolla.com still resolve here).
Balance: 14 hostnames on RUVDS / 11 on windows-source.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 20:11:18 +03:00
5cdedb376a wiki(nl-vds-3xui): ingest session 2026-06-05 — honest final state + lessons
- new sources/nl-vds-3xui-setup-2026-06-05.md (full chronicle; HONEST outcome:
  у реальных клиентов из РФ работает только plain VLESS 32030; Reality/MTProto/
  SOCKS не поднялись)
- new concepts/proxy-debugging-test-the-real-client.md (anti-pattern: own curl/
  standalone tests passed while user's real clients failed; overclaim + bad
  MSS-clamp fix that broke things)
- rewrote entities/nl-vds-3xui.md — removed false "Reality verified/fixed" &
  "mtg works" claims; honest status table; MSS-clamp removed
- caveat added to reality-pq concept (disabling PQ != working Reality for GUI
  clients); index.md + log.md updated

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 14:36:36 +03:00
9359f9ec4f fix(nl-vds-3xui): move Reality 443 -> 2053 (RKN blocking :443)
Confirmed by experiment: Reality on :443 worked ~12min then died (all
timeouts via live v2rayN); moved inbound to :2053 and verified from same
RU PC/core -> exit NL, google 200, 10MB @5.3MB/s. RKN port-blocks :443
dynamically. New X25519 share link (port 2053) in pass. Caveat: 2053 may
also get flagged under heavy use; rotate port if it dies.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 13:06:26 +03:00
cacd582303 fix(nl-vds-3xui): disable ML-DSA-65 PQ on Reality 443 — GUI clients can't connect
Root cause of "Reality fails on client": v2rayN 7.19.5 + phone apps do NOT emit
mldsa65Verify to the xray core, so they can't auth to a PQ-mandated REALITY
inbound (only plain VLESS 32030 worked). Replica matrix: noPQ+no-verify=204,
PQ+no-verify=000, noPQ+with-verify=000. Removed mldsa65Seed from inbound id=1 ->
standard Reality X25519. Verified verify-less client through real :443 -> 204.
Fresh X25519 share-link generated for re-import. Wiki concept updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 12:31:31 +03:00
b23ca3a890 feat(nl-vds-3xui): add MTProto proxy (mtg) on :8443, FakeTLS
SOCKS5 works server-side but DPI on user's path blocks it -> stood up mtg
v2.2.8 as systemd service (DynamicUser, Restart=always), FakeTLS domain
www.cloudflare.com. Verified: openssl probe -> valid TLS1.3 CN=www.cloudflare.com
(indistinguishable from real HTTPS). Secret + tg link in pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 12:00:39 +03:00
c87d7e7442 wiki(nl-vds-3xui): add SOCKS5 inbound (port 47020) for Telegram
socks5 auth=password inbound added via x-ui.db insert + restart (self-rollback
guard, not triggered). Verified external: auth enforced, no-auth rejected,
exits via NL node. Creds in pass nl-vds-3xui/full-env.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:50:36 +03:00
fc245c23c7 fix(nl-vds-3xui): apply Reality dest fix + rotate creds (server-side done)
Per user "и то и другое":
- Reality 443 dest www.intel.com -> www.microsoft.com (x-ui.db + restart),
  verified end-to-end tunnel through real :443 (SNI microsoft) -> HTTP 204.
  ML-DSA-65 kept. Plain VLESS 32030 unaffected (xray restarted, listening).
- Rotated leaked creds: root SSH password, panel username+password (bcrypt),
  panel JWT secret (kills install API token + sessions). New creds in
  pass nl-vds-3xui/full-env; old SSH pass now rejected; db backed up on server.
- Remaining client-side: user sets v2rayN profile pqmkayaxo2 SNI -> microsoft.

Task fix-nl-vds-reality-pq-dest -> 🟡 (awaiting client-side + confirm).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:45:44 +03:00
5116abae07 tasks(fix-nl-vds-reality-pq-dest): open — Reality 443 PQ×dest diagnosed, fix awaits user
Read-only diagnosis complete, server untouched. Root cause: ML-DSA-65 (PQ)
incompatible with dest www.intel.com (Akamai). Fix (intel->microsoft) staged
on board, not applied. Plain VLESS 32030 works.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:27:10 +03:00
4fc7cbf688 wiki(nl-vds-3xui): new NL 3x-UI node + Reality PQ×dest root-cause
Reality 443 inbound silently fails: ML-DSA-65 (post-quantum) ClientHello
key-share X25519MLKEM768 relayed to dest www.intel.com (Akamai) -> HRR ->
borrowed-TLS handshake never completes. Plain VLESS 32030 unaffected.
Isolated via replica xray pair (matrix: intel+PQ is the only failing cell).
Fix (NOT applied, awaiting user): switch dest/SNI -> www.microsoft.com
(PQ-capable, verified) on both inbound and client profile.

- new entities/nl-vds-3xui.md
- new concepts/reality-pq-mldsa65-dest-incompatibility.md
- index.md + log.md updated
- creds saved to pass nl-vds-3xui/full-env (not in repo)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:26:16 +03:00
b04033d15a tasks(NEXT_SESSION): handoff — books-vds disk-full closed, Bookva->slovo scoping, key-revocation guard
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 12:29:19 +03:00
b0559fdbed wiki(books-vds): disk-full incident 2026-06-04 + Bookva->slovo job scoping
Provider Rusonyx alert (96% used). Reclaimed ~64G: truncated runaway
task-runner json-logs (38G) + removed orphan BuildKit builders/volumes
(26G). Added logrotate copytruncate guard.

Root cause: books-task-runner (slovo) error-looped on Ozon seller Bookva
(50542) whose key was intentionally swapped to slovo's as an access
revocation. Spam came from unscoped legacy seller-iterating agenda jobs.
Fixed: idSeller:2 on 5 jobs (deploy-durable, non-_fromConfig). Verified
6448 errors/min -> 0.

Group-V (channel-driven) confirmed non-issue: Bookva channel is type=ym,
no active Bookva-ozon channel exists.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 12:28:30 +03:00
adecf36ffd tasks(vehicles-loader-progress-deploy): 01.06 prod-run verified — importRun id=5 ok, email delivered to gmail
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 09:50:25 +03:00
e3f9d24621 tasks(NEXT_SESSION): handoff — loader 0.4.0 verified + email fixed, task closed
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 09:31:49 +03:00
c588326c28 tasks(vehicles-loader-progress-deploy): close 🟢 — verify ok + email-recipient fix
Live-verify прогона Sun 31.05 06:21→07:53 MSK прошёл: progress-logging
показал движение по всем фазам, НЕ тишина. exit 0, importRun id=4 ok.

Попутно найден+починен email-баг: STOSTAYER_MAIL_TO=site@stostayer.ru
(отчёт слался сам себе) → vitya.kuznetsov@gmail.com в host env
(бэкап .bak.20260531). Доставка на gmail подтверждена тест-письмом.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 09:31:17 +03:00
fea9fd4f53 tasks(NEXT_SESSION): handoff — 0.4.0 deployed, journalctl verify after 06:21 Sun
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 09:03:48 +03:00
b8a07fee85 tasks(vehicles-loader-progress-deploy): 0.4.0 deployed, verify pending 06:21
Build здесь → push docker.stostayer.ru/vehicles-loader:0.4.0 → на хосте
клиента pull + re-tag :latest=0.4.0 (0.3.0 retained для rollback).
Acceptance #1 закрыт. Verify journalctl ждёт natural-прогона
Sun 2026-05-31 06:21 MSK (user выбрал natural-окно, без baseline-reset).
Open Q #1 снят: a74ef73 уже в origin/master.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 09:03:13 +03:00
7ec6200901 tasks(vehicles-loader-progress-deploy): new task — deploy 0.4.0 + live-verify journalctl
Handoff from stostayer.new (code task vehicles-loader-progress-logging closed
by-inspection, commit a74ef73). Ops follow-up: rebuild image 0.4.0 on the same
channel (build here -> push docker.stostayer.ru -> host pull + re-tag) and
capture journalctl from a live run to verify phase/batch/delete-sweep progress.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 08:51:41 +03:00
f102d9ec19 @
docs(CLAUDE): add secrets rule — pass-first, stostayer server map

Credentials live in pass (password-store); check `pass ls`/`pass show` before
grepping the wiki or ~/.ssh/config. Lists the СТО Стайер pass entries
(stostayer/client, rusonyx, client-wireguard).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@
2026-05-30 08:35:12 +03:00
129b75a018 @
tasks(vehicles-loader-image-distribution): first prod run + email/DNS fix

First scheduled run (2026-05-30 06:20) synced OK (importRun id=3) but failed on
the email step: container DNS cannot resolve mail.stostayer.ru (rewritten
resolv.conf + firewalled :53; --add-host ignored under --network host). Data
landed fine. Fixed by mounting a custom /etc/hosts; verified via
nodemailer.verify() = SMTP_VERIFY_OK. stostayer.new 8e5997c. Logging task filed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@
2026-05-30 07:54:21 +03:00
76c261cc4f @
tasks(vehicles-loader-image-distribution): prod deploy done + MariaDB IPv6 gotcha

vehicles-loader deployed live on new.stostayer.ru: image 0.3.0 in client
registry, env-file placed, systemd timer enabled (next 06:20 MSK). Records the
MariaDB-IPv4 / host-net /etc/hosts ::1 gotcha hit during the dry-run (fixed via
STOSTAYER_DB_HOST=127.0.0.1 + --network host), now in stostayer.new wiki.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@
2026-05-29 23:36:30 +03:00
2cf0b536de @
tasks(vehicles-loader-image-distribution): closed — client-registry + build-here + no-Portainer

Decision channel resolved: maintainers build locally, push to the client's
own registry docker.stostayer.ru, host pulls + re-tags :latest. No Portainer
for the oneshot run. Deploy docs finalized in stostayer.new e55cfba.
Actual prod deploy on new.stostayer.ru:20435 is a follow-up pending grant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@
2026-05-29 22:42:38 +03:00
91fac7014a docs(tasks): vehicles-loader-image-distribution — B requires verdaccio access; A build-proven; CMD finalized
Из stostayer.new session 29.05: lite core зашипан, Dockerfile CMD финализирован
(блокер сузился до канала поставки + README distribution-секции). Дописано:
B (build-from-git) требует verdaccio-доступ у клиента (.yarn/cache не коммитится),
что усиливает рекомендацию A. A технически подтверждён (docker build + run --help ок).
2026-05-29 20:43:30 +03:00
2a153d573c meta(tasks): create [vehicles-loader-image-distribution] in OpeItcLoc03/admin 2026-05-29 10:17:30 +00:00
59cb5c2fbd tasks(NEXT_SESSION): session-close — labtools.ru slow = non-issue (local DNS artifact)
Прод RUVDS отдаётся быстро (~0.35s); медленность была локальной stale-копией
(LAN-DNS → home traefik :8089), не дефектом прода. traefik не трогал.
Open ES-incident треки/guards перенесены вперёд.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 09:56:42 +03:00
063910e265 incident(books-vds-es): true RCA — ransom-бот через открытый :9200, не оператор
Рецидив вчерашнего ES-инцидента вскрыл истинную причину. Вчерашняя
гипотеза «оператор в cutover попал на canonical» опровергнута.

Root cause: ES (stack 33) публиковал 0.0.0.0:9200 мимо traefik. Free-ES
7.10 без auth → порт открыт всему интернету. Ransom-бот сносил индексы
by-name (мимо Control #1 destructive_requires_name), оставлял read_me с
BTC-выкупом. accessLog (Control #2) пуст — бот шёл прямо в порт, не через
traefik. firewalld бесполезен (docker-publish обходит INPUT-зоны).

Fix (Control #3): убрана публикация host-порта из stack 33 (Portainer
PUT), дыра закрыта; re-restore epz/products/artmone из daily-2026-05-25.

Отдельный баг: epz-поиск падал у ОБОИХ тенантов — getTenantIdSeller(
config.get("tenant")) через node-config, а tenant не задан ни в
default.json, ни в env-маппинге (TENANT env = мёртвый груз). Добавлен
tenant в overlay default.json (slovo/bookva). products работал —
отдельный код-путь.

accessLog откатан (сторожил не ту дверь).

- wiki concept: correction-блок + секция «Рецидив 2026-05-29» + exposure-audit
- tasks: restore-es reopened+reclosed; new  harden-books-vds-exposed-ports
- NEXT_SESSION handoff

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 09:31:25 +03:00
63708fe659 tasks(NEXT_SESSION): session-close — ES restore + 2 preventive controls live
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 20:12:11 +03:00
48a5cf397e tasks(restore-es-indices-books-vds): closed — snapshot restore + 2 preventive controls
snapshot restore из kreknin:daily-2026-05-25 за 46 сек (epz=820604,
products=105922, artmone=2621, counts == source).

RCA: 5 индексов (включая system .tasks) удалены через ES API DELETE _all
за 1 сек на 2026-05-26 10:21 UTC — 1ч 11мин после создания bookva-es в
cutover-prep. Каноничный endpoint elasticsearch.kzntsv.site попал под
команду которая предназначалась bookva-es:9200 (internal-only, без
traefik route). Caller identity unrecoverable: ES audit = X-Pack платный,
traefik accessLog был выключен, Portainer CE без audit.

Preventive controls applied + verified:
1. ES env action.destructive_requires_name=true (stack 33) — DELETE _all
   и wildcard теперь 400 BadRequest; by-name DELETE работает (нужно для
   reindex). Pattern удаления что случился физически невозможен.
2. Traefik JSON accessLog в /letsencrypt/access.log — будущие DELETE
   оставят forensic след с IP/user/method/path.

Wiki concept: .wiki/concepts/es-destructive-delete-incident-2026-05-26.md
с recovery runbook + preventive controls + cross-refs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 20:11:07 +03:00
e9b0a9edbc docs(tasks): restore-elasticsearch-indices-books-vds — books VDS ES stack 33 пуст
Prod incident в репо books: slovo поиск товаров+EPZ лежит. `elasticsearch.kzntsv.site`
отвечает 200 на _cluster/health (green, 0 active shards, 0 docs); source
`elasticold.kzntsv.site` (Windows rollback) — полный набор: epz 820604 /
products 105922 / artmone 2621. Окно поломки 27.05 10:24 → 28.05 13:59, в нём
шла bookva-tenant-cutover-prep (bookva-es stack 37 создавался) — возможный
конфликт.

Playbook в task-файле: forensics (Portainer stack 33 audit + docker volume
inspect) → snapshot restore из kreknin repo если post-25.05 snapshot с
непустыми индексами → иначе reindex повторно по migrate-elasticsearch-to-books-vds
procedure (730MB ~5-10 мин). После — restart 4 consumer'ов.

NEXT_SESSION.md перезаписан, restore поставлен как приоритет 1, carried
items (registry GC, netplan artifacts, board-viewer-build) сохранены.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 19:50:13 +03:00
068f7b2691 docs(iis-migration): owner DNS-flip instructions (labtools, tandemmebel)
— labtools.ru: Я.360 (admin.yandex.ru/domains), apex+www A → 80.64.31.36
— tandemmebel.ru: reg.ru, apex A → 80.64.31.36; www/maljarka follow via CNAME

Server-side prep for tandemmebel done same session: cert tandemmebel.ru
(LE R13, NotAfter 2026-07-21) extracted from local traefik acme.json,
imported to RUVDS LocalMachine\My, attached to existing IIS bindings for
apex + www. Wiki entity ruvds-iis-host.md stale on this point — bindings
were already created during initial migration, just lacked cert.
2026-05-28 15:41:23 +03:00
11554d45ac wiki(ingest): vds-kzntsv network-stack mismatch RCA + ifupdown anti-pattern
Incident 2026-05-28 ~05:45–13:52 MSK на vds-kzntsv. ~2.5ч активного outage
+ ~5.5ч на эфемерной статике до окончательного fix хостером.

Revised RCA: наш netplan+systemd-networkd конфликтовал с provider's expected
ifupdown stack. Их start/ipadd procedure ожидает чистый ifupdown и не может
auto-recover когда networkd «держит» eth0. 8 дней работало потому что
networkd сам тянул DHCP. Когда что-то на стороне Rusonyx разорвало
DHCP-binding — auto-recovery не сработала.

Fix: systemctl mask netplan + systemd-networkd* (на running system без stop —
IP и SSH сохранились), Rusonyx ребутнул VM и положил чистый
/etc/network/interfaces.d/ifcfg-eth0 через свой start/ipadd. Netmask /18,
gw 89.253.192.1, чистый ifupdown.

Wiki:
- NEW concepts/vds-kzntsv-dhcp-outage-2026-05-28 — full RCA + recovery
  runbook (эфемерная статика + permanent-fix via ifupdown) + diagnostic
  dot-graph + revised lessons-learned + anti-pattern
- NEW sources/vds-kzntsv-incident-2026-05-28 — timeline 05:25 backup OK →
  08:43 statics → 13:52 final reset; provider's ifcfg-eth0 content; ticket
  text reference
- UPDATE entities/vds-kzntsv — mask /18, ifupdown stack, kernel cmdline
  net.ifnames=0 объясняет eth0 naming, hypervisor hw80, pass-store путь,
  Known issues §
- UPDATE concepts/rusonyx-vps-onboarding-quirks — quirk #9 переписан про
  /18 layout + canonical ifcfg, quirk #10 NEW про ifupdown vs netplan
  stack choice + bootstrap mask commands
- UPDATE index.md + log.md

Tasks:
- STATUS header: incident RESOLVED summary
- NEXT_SESSION: следующая сессия — cleanup netplan-artifacts (optional),
  registry GC (~20G pending), board-viewer-build unhealthy разбор

Memory (out-of-repo): vds-kzntsv-rusonyx-network-recovery переписан с
revised RCA — canonical bootstrap step «mask netplan/networkd» для всех
Rusonyx Ubuntu VDS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 14:12:14 +03:00
dacc39a113 tasks(modulair-rag-vds-redeploy): follow-ups #1-3 done — compose/lightrag fixes verified
entrypoint https->websecure, lightrag env mapped to native LLM_/EMBEDDING_
names (EMBEDDING_DIM=4096), ollama tag qwen3-next:80b-cloud. End-to-end
ingest verified (4 entities, 3 relations). modulair-rag fc68a16 (unpushed).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 09:50:48 +03:00
c895cf52e6 tasks(modulair-rag-vds-redeploy): 🟢 closed — 4-container stack live on VDS
Deployed modulair-rag (Portainer stack 15) on VDS 89.253.255.94, acceptance 6/6.
Pre-flight verify found the task was filed on stale premises; corrected in-flight:
- registry images absent (registry reinstalled 2026-05-20) → rebuilt all 3 ON the
  VDS (3.36GB tier1 layer 499'd pushing through traefik from home; local push works)
- MINIO_ENDPOINT=minio.vds.kzntsv.site (minio.kzntsv.site is books VDS)
- traefik entrypoint https→websecure (NAS-era label) fixed in live stack
- created DB modulair_rag, minio bucket + scoped svcacct, ROUTERAI key to pass
- DNS modulair-mcp.kzntsv.site→VDS (user)

Follow-ups (consumer repo modulair-rag): compose.yml entrypoint commit,
portainer-stack.md rewrite, lightrag embedding binding=ollama/model=None check.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 09:28:48 +03:00
feeba28433 tasks(NEXT_SESSION): session-close — tenant scheduler seller-pinning live
Scheduler seller-pinning выкачен на slovo+bookva (16 джоб, idSeller/ch
per-tenant), task-runner добавлен в auto-deploy, wiki ingested. Открытые
треки: slovo-overlay deploy не активен (legacy bundled compose), docs-commit
триггерит лишний redeploy, bookva compose без DEPLOY_AT.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 22:22:52 +03:00
8238ef46a8 tasks: ops-mcp-multi-tenant activated same session
Stack 26 (books-ops-mcp) PUT с pullImage:true + добавлен BOOKVA_MARIADB_PASSWORD
env (= MARIADB_PASSWORD, bookva-db ops_ro password identical после cp-a).
Container books-ops-mcp@master-f5f295b running.

Smoke:
- ops.mariadb.query tenant=slovo → АФО2/Главная26/НК11 (slovo warehouses)
- ops.mariadb.query tenant=bookva → Главная26/Ира/НК11 (bookva warehouses)
  (Ира появилась в bookva post-cutover, нет в slovo — данные genuinely разные)
- ops.mongo.count tenant=bookva collection=agendaJobs → 2818

Один host-level ops-mcp видит обе tenant DBs через per-tenant pool maps.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 18:40:33 +03:00
12c7dc5ab9 tasks(NEXT_SESSION): session-close — bookva tenant deep-debug + 6-bug post-mortem
Multi-fix session: bookva tenant works correctly после 6 fix'ов одной природы
(incomplete cutover-prep без e2e smoke). См. NEXT_SESSION.md § "What's LIVE"
+ § "Memory updates" + shared wiki concept tenant-overlay-config-volume-mount-path-pitfall.

Changes:
- host-stacks/books-vds/ops-mcp.compose.yml: add BOOKVA_MARIADB_PASSWORD env
  declaration + multi-tenant docs note. Stack 26 ещё не пере-PUT'нут на это,
  открытая микро-таска ops-mcp-multi-tenant-stack-activate в NEXT_SESSION.
- STATUS.md: 6-bug summary в latest update line.
- NEXT_SESSION.md: full rewrite с 14 recent commits, open треками, asks к user'у,
  memory updates про agenda.db.collection / composite jobId / scheduler envs /
  bookva auth tokens.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 18:32:29 +03:00
10e80dc036 tasks(books-ops-mcp-host-promote): 🟢 closed + host-stacks compose canonical
Stack 26 (books-ops-mcp) in-place PUT в host-level compose без container churn.
Stack 43 (bookva-ops-mcp) deleted. 2 Gitea secrets revoked.

New canonical: `host-stacks/books-vds/ops-mcp.compose.yml` (management-plane,
manual Portainer updates как traefik/portainer).

Design locks: location=.admin/host-stacks, deploy=manual Portainer,
MariaDB scope=slovo's only (Option A), config path unchanged.

Companion commits: `victor/books ae3ab14`, `victor/bookva-overlay 50f5bbb`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 14:04:15 +03:00