From c895cf52e69f7a34c1ca03acc49b4b3350ec7628 Mon Sep 17 00:00:00 2001 From: vitya Date: Wed, 27 May 2026 09:28:48 +0300 Subject: [PATCH] =?UTF-8?q?tasks(modulair-rag-vds-redeploy):=20?= =?UTF-8?q?=F0=9F=9F=A2=20closed=20=E2=80=94=204-container=20stack=20live?= =?UTF-8?q?=20on=20VDS?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Deployed modulair-rag (Portainer stack 15) on VDS 89.253.255.94, acceptance 6/6. Pre-flight verify found the task was filed on stale premises; corrected in-flight: - registry images absent (registry reinstalled 2026-05-20) → rebuilt all 3 ON the VDS (3.36GB tier1 layer 499'd pushing through traefik from home; local push works) - MINIO_ENDPOINT=minio.vds.kzntsv.site (minio.kzntsv.site is books VDS) - traefik entrypoint https→websecure (NAS-era label) fixed in live stack - created DB modulair_rag, minio bucket + scoped svcacct, ROUTERAI key to pass - DNS modulair-mcp.kzntsv.site→VDS (user) Follow-ups (consumer repo modulair-rag): compose.yml entrypoint commit, portainer-stack.md rewrite, lightrag embedding binding=ollama/model=None check. Co-Authored-By: Claude Opus 4.7 (1M context) --- .tasks/STATUS.md | 16 ++++++ .tasks/modulair-rag-vds-redeploy.md | 84 +++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+) create mode 100644 .tasks/modulair-rag-vds-redeploy.md diff --git a/.tasks/STATUS.md b/.tasks/STATUS.md index f19e1bd..1fe079b 100644 --- a/.tasks/STATUS.md +++ b/.tasks/STATUS.md @@ -1,4 +1,6 @@ # Admin Task Board +_Updated: 2026-05-27 — `modulair-rag-vds-redeploy` 🟢 **closed** в ту же сессию: 4-контейнерный стек развёрнут на VDS (Portainer stack 15), acceptance 6/6. Образы пересобраны на самом VDS (push 3.36GB через traefik с дома падал 499); env/entrypoint/minio-host скорректированы под VDS-реальность. 3 follow-up'а переданы в modulair-rag handoff._ +_Updated: 2026-05-27 — заведена `modulair-rag-vds-redeploy` ⚪ (handoff из modulair-rag session). NAS-loss redeploy 4 контейнеров на VDS. Блокер MinIO снят — verified up на VDS 2026-05-27. postgres `proxy`-network reachability подтверждён (`postgres:5432` резолвится из pipeline/mcp). Спека: modulair-rag concept `nas-loss-vds-redeploy-context` + compose.yml as-is._ _Updated: 2026-05-27 (ночь, после re-open) — ops-mcp multi-tenant **activated** (stack 26 PUT + BOOKVA_MARIADB_PASSWORD env). Smoke verified: slovo/bookva DB queries возвращают tenant-specific data (АФО2 vs Ира warehouses), bookva agendaJobs count=2818. Один host-level books-ops-mcp видит обе tenant DBs._ _Updated: 2026-05-27 (ночь) — bookva-tenant deep-debug session. 6 багов одной природы (incomplete cutover-prep): (1) bookva-web без config volume → slovo DB leak; (2) bookva-{api,scheduler,task-runner} mount path /app→/usr/src/app; (3) NITRO/NUXT env не reference'или в compose; (4) bookva-scheduler без RECONCILER_ENABLED → dry-run handlers; (5) bookva-scheduler без healthcheck → CI timeout; (6) deploy.yml svc=job-scheduler vs secret=SCHEDULER mismatch. + composite agendaJobId display fix + books-web config mount (slovo also missed) + jobs.post.js agenda.db.collection→MongoClient. Ingested wiki concept tenant-overlay-config-volume-mount-path-pitfall (shared). ops-mcp multi-tenant код в image готов но stack 26 ещё не пере-PUT'нут (next session)._ @@ -15,6 +17,20 @@ _Updated: 2026-05-26 — заведена `bookva-tenant-cutover-prep` ⚪ (7-st _Updated: 2026-05-25 (`iis-migration-to-ruvds` 🟢 closed Phase 1 per user decision — 9/24 hostnames live на RUVDS, source IIS оставлен running. `migrate-elasticsearch-to-books-vds` 🟢 closed ранее сегодня. 16 IIS hostnames + LE renewal pipeline + decommission — descoped в Closure note, не отдельные tracker tasks.)_ +## 🟢 [modulair-rag-vds-redeploy] — closed 2026-05-27 — 4-контейнерный стек развёрнут на VDS, acceptance 6/6 + +NAS помер → 4 контейнера пропали → fresh redeploy на VDS (89.253.255.94). Portainer stack id=15. **Acceptance 6/6 green:** 4 контейнера up без restart-loop; lightrag Uvicorn :9621; `https://modulair-mcp.kzntsv.site`→200; pipeline+tier1 0% CPU; pipeline-лог без postgres/minio ошибок. + +Таска писалась по stale-предпосылкам — по ходу исправлено (см. [modulair-rag-vds-redeploy.md](modulair-rag-vds-redeploy.md) Decisions): registry-образы отсутствовали (registry переустановлен 2026-05-20) → **rebuild на VDS** (push 3.36GB tier1-слоя с дома падал 499 через traefik → собрал на самом VDS, push локальный); `MINIO_ENDPOINT`=`minio.vds.kzntsv.site` (не `minio.kzntsv.site`=books VDS); traefik entrypoint `https`→`websecure` (NAS-имя в compose-лейбле); DB+bucket+scoped-svcacct созданы; ROUTERAI key в pass; DNS поправил user. + +**Follow-ups (не блокеры acceptance):** modulair-rag/compose.yml entrypoint-фикс в репо; portainer-stack.md под VDS; lightrag embedding binding=ollama/model=None — проверить ROUTERAI_* маппинг. Все переданы в handoff (consumer repo / parallel session). +**Branch:** master + + + + +--- + ## 🟢 [migrate-elasticsearch-to-books-vds] — closed 2026-05-25 — ES indices с Windows (elasticold.kzntsv.site, 7.10.1) → books VDS (elasticsearch.kzntsv.site, 7.10.0). 3 indices, 928616 docs. Reindex-from-remote через `reindex.remote.whitelist` env, добавленный в Portainer stack 33. Pre-migration snapshot в kreknin repo как rollback. Consumer configs sed'd, 3 containers restarted, smoke green. Source НЕ выключен. См. [migrate-elasticsearch-to-books-vds.md](migrate-elasticsearch-to-books-vds.md) § Closure note. diff --git a/.tasks/modulair-rag-vds-redeploy.md b/.tasks/modulair-rag-vds-redeploy.md new file mode 100644 index 0000000..2044c67 --- /dev/null +++ b/.tasks/modulair-rag-vds-redeploy.md @@ -0,0 +1,84 @@ +# modulair-rag-vds-redeploy + +## Goal +Re-deploy the lost `modulair-rag` stack (4 containers) from the dead Synology NAS onto the VDS (`89.253.255.94`, rusonyx). Fresh start — knowledge base + pipeline cache were on lost named volumes, not critical (deploy was only ~1 week old). This `.admin` project owns the ops; `modulair-rag` repo stays the consumer. + +## Source of truth (read before acting) +- **Full context-freeze (canonical):** `~/projects/modulair-rag/.wiki/concepts/nas-loss-vds-redeploy-context.md` — pre-flight, deploy steps, acceptance, post-deploy. *(Note: currently uncommitted in the modulair-rag repo, but present on disk.)* +- **Compose file (deploy as-is):** `~/projects/modulair-rag/compose.yml` — 4 services, named volumes, `proxy` external network. Do not edit; paste into Portainer. +- **Design spec (architecture):** `mcp__projects-meta__knowledge_get slug="concepts/modulair-rag-design"` (shared wiki). +- **Stack identity (NAS-era, now stale):** `~/projects/modulair-rag/.wiki/entities/portainer-stack.md`. + +## VDS state — VERIFIED 2026-05-27 (via vds-ops `ops_docker_ps` / `ops_docker_inspect`) +- ✅ **MinIO** container `minio/minio:latest` running 4d on VDS — the original blocker is CLEARED. +- ✅ **postgres:16** running; on networks `proxy` (172.18.x) **and** `shared-dbs`, aliases `postgres`. → hostname `postgres:5432` from `modulair-pipeline`/`modulair-mcp` (which sit on `proxy`) **will resolve**. Compose's cross-stack DB reference is sound, no extra network wiring needed. + - Minor: postgres runs with `ssl=on`; libpq default `sslmode=prefer` negotiates automatically — not a blocker. +- ✅ `registry:2.8.3` (`registry.kzntsv.site`) running — images `modulair-pipeline:latest`, `tier1-converter:latest`, `modulair-mcp:latest` were pushed 2026-05-13 (per context-freeze). Verify they're still present before deploy. +- ✅ `traefik:v2.11` running — compose labels target `modulair-mcp.kzntsv.site`. +- ⛔ None of the 4 modulair containers exist yet (`lightrag-modulair`, `modulair-pipeline`, `tier1-converter`, `modulair-mcp`) — clean deploy. + +## Pre-flight — admin must verify/create (NOT visible to read-only vds-ops) +1. [ ] **MinIO bucket** `modulair-rag` exists. MinIO container is up but bucket membership is internal — create if missing (`mc mb`). +2. [ ] **Postgres DB** `modulair_rag` exists → `CREATE DATABASE modulair_rag;` if not. lightrag self-creates its schema on first start — do NOT pre-init tables. +3. [ ] **DNS** `modulair-mcp.kzntsv.site` → VDS IP `89.253.255.94` (likely still points at the dead NAS — repoint via regru/cloudflare). +4. [ ] **MinIO endpoint sanity:** compose env uses `MINIO_ENDPOINT=minio.kzntsv.site:443` (SSL, hairpin via traefik back to the VDS minio). Confirm traefik route `minio.kzntsv.site` now points at the VDS minio container, and DNS resolves to VDS. (Alternative: direct `minio:9000` on a shared network if both end up on `proxy` — only if the external route is problematic.) +5. [ ] **Secrets via pass** (pattern from `[secrets-manager-adopt]`): + ``` + pass insert modulair-rag/routerai-key + pass insert modulair-rag/ollama-api-key + pass insert modulair-rag/minio-access-key + pass insert modulair-rag/minio-secret-key + pass insert modulair-rag/postgres-password + ``` + +## Deploy (Portainer — per `.admin` VDS-ops rule, NOT ssh+compose) +1. Open Portainer `https://portainer.vds.kzntsv.site`. +2. New Stack → paste `~/projects/modulair-rag/compose.yml` as-is. +3. Environment vars — fill from `pass show` + statics: + - `ROUTERAI_API_KEY` (pass), `ROUTERAI_BASE_URL=https://routerai.ru/api/v1`, `ROUTERAI_EMBED_MODEL=qwen3-embedding-8b` + - `OLLAMA_API_KEY` (pass), `OLLAMA_HOST=https://ollama.com`, `OLLAMA_LLM_MODEL=qwen3-next:latest` + - `MINIO_ENDPOINT=minio.kzntsv.site`, `MINIO_PORT=443`, `MINIO_USE_SSL=true`, `MINIO_FORCE_PATH_STYLE=true`, `MINIO_BUCKET=modulair-rag`, `MINIO_REGION=us-east-1`, `MINIO_ACCESS_KEY`/`MINIO_SECRET_KEY` (pass) + - `POSTGRES_USER`, `POSTGRES_PASSWORD` (pass), `POSTGRES_DB=modulair_rag` + - `IMAGE_TAG=latest` +4. Pre-pull images, Deploy. + +## Acceptance +- [ ] 4 containers up, no restart-loops (`ops_docker_ps` on VDS). +- [ ] `lightrag-modulair :9621` healthcheck responds (internal network, not exposed). +- [ ] `https://modulair-mcp.kzntsv.site` → 200 on root. +- [ ] `modulair-pipeline` + `tier1-converter` NOT in CPU-spin — fix `[pipeline-tier1-cpu-fix]` already in `:latest` (commits `24c9bd5` + `ceb525e`). +- [ ] `modulair-pipeline` log: no `connection refused` to postgres. +- [ ] pipeline log: no `NoSuchBucket` / `AccessDenied` from MinIO. + +## Post-deploy +1. Rewrite `~/projects/modulair-rag/.wiki/entities/portainer-stack.md` for VDS (Docker host, Portainer URL, MinIO endpoint). +2. Close this 🟢 task with note. +3. `[future-resilient-architecture-goals]` — add modulair-rag to RTO/RPO matrix (low priority, non-client prod, but host-loss recovery should be documented). + +## Decisions log +- 2026-05-27 (exec session, .admin): pre-flight verify вскрыл, что таска писалась по stale-предпосылкам. Исправления: + - **Registry образы ОТСУТСТВУЮТ** (open-q #1 = NO). Registry на VDS переустановлен с нуля 2026-05-20 (`pass vds-kzntsv/full-env` коммент: "не мигрировали с kreknin — user accepts loss"). Образы push'ились 2026-05-13 на kreknin-registry → потеряны. Каталог `registry.kzntsv.site/v2/_catalog`: только books/board/vds-ops. → **rebuild всех 3 обязателен.** + - **MINIO_ENDPOINT исправлен** (open-q #2): `minio.kzntsv.site` → 89.253.255.**133** (books VDS, чужой!). Правильный route на этом VDS = `minio.vds.kzntsv.site` (traefik label на minio-контейнере = `Host(minio.vds.kzntsv.site)`, health/live=200). tier1-converter сидит только на `modulair` bridge (НЕ на `proxy`) → внутренний `minio:9000` для него не резолвится → нужен внешний route для всех. Итог env: `MINIO_ENDPOINT=minio.vds.kzntsv.site`, PORT=443, SSL=true. Compose не редактирую (paste-as-is), это env-var. + - **MinIO creds:** создан scoped svcacct `modulair-rag-svc` (политика — только bucket `modulair-rag`, verified видит только свой bucket). Не root. → `pass modulair-rag/minio-{access,secret}-key`. + - **MinIO bucket** `modulair-rag` создан (mc, локальный alias на minio.vds.kzntsv.site root creds). + - **Postgres DB** `modulair_rag` создана на `postgres.vds.kzntsv.site:5432` (docker one-off psql, sslmode=require). POSTGRES_USER=postgres (shared), pwd из `pass vds-kzntsv/full-env`. + - **DNS** `modulair-mcp.kzntsv.site` → был 94.19.247.14 (мёртвый NAS = домашний IP). User поправил → 89.253.255.94 (verified public+authoritative). + - **ROUTERAI key** не было в pass — user дал, сохранён `pass modulair-rag/routerai-key`. OLLAMA key = `pass interns/ollama-cloud-api-key`. + - **Portainer API key протух** (401) → auth через admin user/pass (`pass vds-kzntsv/full-env`), endpoint=1 (local). + - **BUILD-БЛОКЕР: tier1-converter = 10.2GB (3.36GB сжатый).** Push с локальной машины падал HTTP 499 (home-канал/middlebox рвёт многогиг HTTPS-заливку через traefik; docker push монолитный, без resume). → **Решение: сборка НА VDS** (`ssh vitya@89.253.255.94` ключ `~/.ssh/id_ed25519`, vitya в docker-группе; клон приватного репо из локального gitea `git.kzntsv.site/cancel_music/modulair-rag` юзером OpeItcLoc03 + admin-token; push registry локально). services/ на origin/master идентичен локальному HEAD (cpu-fix ceb525e внутри). IMAGE_TAG=0.1.0. + +- 2026-05-27 (DEPLOY ✅): стек `modulair-rag` (Portainer stack id=15, endpoint=1) развёрнут. Образы 0.1.0 (pipeline/tier1-converter/modulair-mcp) собраны на VDS + lightrag ghcr.io/hkuds/lightrag:latest. Acceptance 6/6 green: 4 контейнера up без restart-loop; lightrag Uvicorn :9621 startup complete; https://modulair-mcp.kzntsv.site → 200 (uvicorn, /docs тоже 200); pipeline+tier1 0% CPU (cpu-fix ok); pipeline-лог без postgres/minio ошибок. + - **+1 compose-фикс при деплое:** лейбл `traefik...modulair-mcp.entrypoints=https` — NAS-имя; на этом VDS entrypoints = `web`/`websecure` (читал `/opt/stacks/traefik/data/traefik.yml`). Без фикса traefik default-404. Поправлено в live-стеке (PUT, https→websecure). **TODO в репо modulair-rag/compose.yml** — закоммитить тот же фикс, иначе следующий redeploy снова 404. + +## Post-deploy / follow-ups +- [ ] **modulair-rag/compose.yml** (consumer repo, parallel session активна — не трогал их дерево): (a) `entrypoints=https`→`websecure`; (b) опционально задокументировать что `MINIO_ENDPOINT` на VDS = `minio.vds.kzntsv.site` (а `minio.kzntsv.site` = books VDS). Передано в handoff. +- [ ] **modulair-rag/.wiki/entities/portainer-stack.md** — переписать под VDS (Docker host=vds-kzntsv 89.253.255.94, Portainer stack 15, MinIO=minio.vds.kzntsv.site, registry образы 0.1.0). Consumer repo — на modulair-rag сессию. +- [ ] **Наблюдение (не блокер):** lightrag-лог: `Embedding config: binding=ollama model=None embedding_dim=1024` — выбран ollama-binding для embeddings с `model=None`, хотя в env передан `ROUTERAI_EMBED_MODEL=qwen3-embedding-8b`. Контейнеры healthy, но первый реальный ingest может не заэмбедить как ожидается. Проверить env→config маппинг ROUTERAI_* в lightrag-образе на modulair-rag стороне. +- [ ] `[future-resilient-architecture-goals]` — добавить modulair-rag в RTO/RPO matrix (low prio). + +## Open questions +- [x] Registry images present? → NO, rebuilt on VDS (см. Decisions). +- [x] minio.kzntsv.site repointed? → NO, это books VDS; используем `minio.vds.kzntsv.site`. + +**Branch:** master +