admin/.tasks/modulair-rag-vds-redeploy.md

modulair-rag-vds-redeploy

Goal

Re-deploy the lost modulair-rag stack (4 containers) from the dead Synology NAS onto the VDS (89.253.255.94, rusonyx). Fresh start — knowledge base + pipeline cache were on lost named volumes, not critical (deploy was only ~1 week old). This .admin project owns the ops; modulair-rag repo stays the consumer.

Source of truth (read before acting)

• Full context-freeze (canonical): ~/projects/modulair-rag/.wiki/concepts/nas-loss-vds-redeploy-context.md — pre-flight, deploy steps, acceptance, post-deploy. (Note: currently uncommitted in the modulair-rag repo, but present on disk.)
• Compose file (deploy as-is): ~/projects/modulair-rag/compose.yml — 4 services, named volumes, proxy external network. Do not edit; paste into Portainer.
• Design spec (architecture): mcp__projects-meta__knowledge_get slug="concepts/modulair-rag-design" (shared wiki).
• Stack identity (NAS-era, now stale): ~/projects/modulair-rag/.wiki/entities/portainer-stack.md.

VDS state — VERIFIED 2026-05-27 (via vds-ops ops_docker_ps / ops_docker_inspect)

• ✅ MinIO container minio/minio:latest running 4d on VDS — the original blocker is CLEARED.
• ✅ postgres:16 running; on networks proxy (172.18.x) and shared-dbs, aliases postgres. → hostname postgres:5432 from modulair-pipeline/modulair-mcp (which sit on proxy) will resolve. Compose's cross-stack DB reference is sound, no extra network wiring needed.
  • Minor: postgres runs with ssl=on; libpq default sslmode=prefer negotiates automatically — not a blocker.
• ✅ registry:2.8.3 (registry.kzntsv.site) running — images modulair-pipeline:latest, tier1-converter:latest, modulair-mcp:latest were pushed 2026-05-13 (per context-freeze). Verify they're still present before deploy.
• ✅ traefik:v2.11 running — compose labels target modulair-mcp.kzntsv.site.
• ⛔ None of the 4 modulair containers exist yet (lightrag-modulair, modulair-pipeline, tier1-converter, modulair-mcp) — clean deploy.

Pre-flight — admin must verify/create (NOT visible to read-only vds-ops)

1. MinIO bucket modulair-rag exists. MinIO container is up but bucket membership is internal — create if missing (mc mb).
2. Postgres DB modulair_rag exists → CREATE DATABASE modulair_rag; if not. lightrag self-creates its schema on first start — do NOT pre-init tables.
3. DNS modulair-mcp.kzntsv.site → VDS IP 89.253.255.94 (likely still points at the dead NAS — repoint via regru/cloudflare).
4. MinIO endpoint sanity: compose env uses MINIO_ENDPOINT=minio.kzntsv.site:443 (SSL, hairpin via traefik back to the VDS minio). Confirm traefik route minio.kzntsv.site now points at the VDS minio container, and DNS resolves to VDS. (Alternative: direct minio:9000 on a shared network if both end up on proxy — only if the external route is problematic.)
5. Secrets via pass (pattern from [secrets-manager-adopt]):

   pass insert modulair-rag/routerai-key
   pass insert modulair-rag/ollama-api-key
   pass insert modulair-rag/minio-access-key
   pass insert modulair-rag/minio-secret-key
   pass insert modulair-rag/postgres-password
   

Deploy (Portainer — per .admin VDS-ops rule, NOT ssh+compose)

1. Open Portainer https://portainer.vds.kzntsv.site.
2. New Stack → paste ~/projects/modulair-rag/compose.yml as-is.
3. Environment vars — fill from pass show + statics:
   • ROUTERAI_API_KEY (pass), ROUTERAI_BASE_URL=https://routerai.ru/api/v1, ROUTERAI_EMBED_MODEL=qwen3-embedding-8b
   • OLLAMA_API_KEY (pass), OLLAMA_HOST=https://ollama.com, OLLAMA_LLM_MODEL=qwen3-next:latest
   • MINIO_ENDPOINT=minio.kzntsv.site, MINIO_PORT=443, MINIO_USE_SSL=true, MINIO_FORCE_PATH_STYLE=true, MINIO_BUCKET=modulair-rag, MINIO_REGION=us-east-1, MINIO_ACCESS_KEY/MINIO_SECRET_KEY (pass)
   • POSTGRES_USER, POSTGRES_PASSWORD (pass), POSTGRES_DB=modulair_rag
   • IMAGE_TAG=latest
4. Pre-pull images, Deploy.

Acceptance

• 4 containers up, no restart-loops (ops_docker_ps on VDS).
• lightrag-modulair :9621 healthcheck responds (internal network, not exposed).
• https://modulair-mcp.kzntsv.site → 200 on root.
• modulair-pipeline + tier1-converter NOT in CPU-spin — fix [pipeline-tier1-cpu-fix] already in :latest (commits 24c9bd5 + ceb525e).
• modulair-pipeline log: no connection refused to postgres.
• pipeline log: no NoSuchBucket / AccessDenied from MinIO.

Post-deploy

1. Rewrite ~/projects/modulair-rag/.wiki/entities/portainer-stack.md for VDS (Docker host, Portainer URL, MinIO endpoint).
2. Close this 🟢 task with note.
3. [future-resilient-architecture-goals] — add modulair-rag to RTO/RPO matrix (low priority, non-client prod, but host-loss recovery should be documented).

Decisions log

• 2026-05-27 (exec session, .admin): pre-flight verify вскрыл, что таска писалась по stale-предпосылкам. Исправления:

  • Registry образы ОТСУТСТВУЮТ (open-q #1 = NO). Registry на VDS переустановлен с нуля 2026-05-20 (pass vds-kzntsv/full-env коммент: "не мигрировали с kreknin — user accepts loss"). Образы push'ились 2026-05-13 на kreknin-registry → потеряны. Каталог registry.kzntsv.site/v2/_catalog: только books/board/vds-ops. → rebuild всех 3 обязателен.
  • MINIO_ENDPOINT исправлен (open-q #2): minio.kzntsv.site → 89.253.255.133 (books VDS, чужой!). Правильный route на этом VDS = minio.vds.kzntsv.site (traefik label на minio-контейнере = Host(minio.vds.kzntsv.site), health/live=200). tier1-converter сидит только на modulair bridge (НЕ на proxy) → внутренний minio:9000 для него не резолвится → нужен внешний route для всех. Итог env: MINIO_ENDPOINT=minio.vds.kzntsv.site, PORT=443, SSL=true. Compose не редактирую (paste-as-is), это env-var.
  • MinIO creds: создан scoped svcacct modulair-rag-svc (политика — только bucket modulair-rag, verified видит только свой bucket). Не root. → pass modulair-rag/minio-{access,secret}-key.
  • MinIO bucket modulair-rag создан (mc, локальный alias на minio.vds.kzntsv.site root creds).
  • Postgres DB modulair_rag создана на postgres.vds.kzntsv.site:5432 (docker one-off psql, sslmode=require). POSTGRES_USER=postgres (shared), pwd из pass vds-kzntsv/full-env.
  • DNS modulair-mcp.kzntsv.site → был 94.19.247.14 (мёртвый NAS = домашний IP). User поправил → 89.253.255.94 (verified public+authoritative).
  • ROUTERAI key не было в pass — user дал, сохранён pass modulair-rag/routerai-key. OLLAMA key = pass interns/ollama-cloud-api-key.
  • Portainer API key протух (401) → auth через admin user/pass (pass vds-kzntsv/full-env), endpoint=1 (local).
  • BUILD-БЛОКЕР: tier1-converter = 10.2GB (3.36GB сжатый). Push с локальной машины падал HTTP 499 (home-канал/middlebox рвёт многогиг HTTPS-заливку через traefik; docker push монолитный, без resume). → Решение: сборка НА VDS (ssh vitya@89.253.255.94 ключ ~/.ssh/id_ed25519, vitya в docker-группе; клон приватного репо из локального gitea git.kzntsv.site/cancel_music/modulair-rag юзером OpeItcLoc03 + admin-token; push registry локально). services/ на origin/master идентичен локальному HEAD (cpu-fix ceb525e внутри). IMAGE_TAG=0.1.0.

• 2026-05-27 (DEPLOY ✅): стек modulair-rag (Portainer stack id=15, endpoint=1) развёрнут. Образы 0.1.0 (pipeline/tier1-converter/modulair-mcp) собраны на VDS + lightrag ghcr.io/hkuds/lightrag:latest. Acceptance 6/6 green: 4 контейнера up без restart-loop; lightrag Uvicorn :9621 startup complete; https://modulair-mcp.kzntsv.site → 200 (uvicorn, /docs тоже 200); pipeline+tier1 0% CPU (cpu-fix ok); pipeline-лог без postgres/minio ошибок.

  • +1 compose-фикс при деплое: лейбл traefik...modulair-mcp.entrypoints=https — NAS-имя; на этом VDS entrypoints = web/websecure (читал /opt/stacks/traefik/data/traefik.yml). Без фикса traefik default-404. Поправлено в live-стеке (PUT, https→websecure). TODO в репо modulair-rag/compose.yml — закоммитить тот же фикс, иначе следующий redeploy снова 404.

Post-deploy / follow-ups

• modulair-rag/compose.yml — entrypoints=https→websecure (commit fc68a16). Параллельная сессия завершилась (tree clean), сделал сам.
• modulair-rag/.wiki/entities/portainer-stack.md — переписана под VDS (stack 15, websecure, minio.vds, build-on-VDS workflow, model-config table) + log.md entry (fc68a16).
• #3 lightrag embedding — RESOLVED & verified end-to-end. Был latent-баг (с NAS): stock-образ LightRAG читает LLM_*/EMBEDDING_*, а не ROUTERAI_*/OLLAMA_* → дефолты (binding=ollama model=None dim=1024). Замаплены stack-inputs на нативные имена в compose: EMBEDDING_BINDING=openai (RouterAI qwen/qwen3-embedding-8b, DIM=4096), LLM_BINDING=ollama (Ollama Cloud). LLM-тег qwen3-next:latest → 404 на Ollama Cloud → исправлен на qwen3-next:80b-cloud (user-confirmed; qwen3-next:80b тоже валиден). Live-smoke на VDS: insert → embedding(RouterAI)+extraction(Ollama) → 4 entities, 3 relations, потом workspace очищен. compose.yml+lightrag-models.md (fc68a16).
• [future-resilient-architecture-goals] — добавить modulair-rag в RTO/RPO matrix (low prio).
• PUSH: modulair-rag fc68a16 + .admin task-коммиты unpushed (автопуш не выдан в этой сессии).

Open questions

• Registry images present? → NO, rebuilt on VDS (см. Decisions).
• minio.kzntsv.site repointed? → NO, это books VDS; используем minio.vds.kzntsv.site.

Branch: master