vitya
dacc39a113
entrypoint https->websecure, lightrag env mapped to native LLM_/EMBEDDING_ names (EMBEDDING_DIM=4096), ollama tag qwen3-next:80b-cloud. End-to-end ingest verified (4 entities, 3 relations). modulair-rag fc68a16 (unpushed). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
86 lines
11 KiB
Markdown
86 lines
11 KiB
Markdown
# modulair-rag-vds-redeploy
|
||
|
||
## Goal
|
||
Re-deploy the lost `modulair-rag` stack (4 containers) from the dead Synology NAS onto the VDS (`89.253.255.94`, rusonyx). Fresh start — knowledge base + pipeline cache were on lost named volumes, not critical (deploy was only ~1 week old). This `.admin` project owns the ops; `modulair-rag` repo stays the consumer.
|
||
|
||
## Source of truth (read before acting)
|
||
- **Full context-freeze (canonical):** `~/projects/modulair-rag/.wiki/concepts/nas-loss-vds-redeploy-context.md` — pre-flight, deploy steps, acceptance, post-deploy. *(Note: currently uncommitted in the modulair-rag repo, but present on disk.)*
|
||
- **Compose file (deploy as-is):** `~/projects/modulair-rag/compose.yml` — 4 services, named volumes, `proxy` external network. Do not edit; paste into Portainer.
|
||
- **Design spec (architecture):** `mcp__projects-meta__knowledge_get slug="concepts/modulair-rag-design"` (shared wiki).
|
||
- **Stack identity (NAS-era, now stale):** `~/projects/modulair-rag/.wiki/entities/portainer-stack.md`.
|
||
|
||
## VDS state — VERIFIED 2026-05-27 (via vds-ops `ops_docker_ps` / `ops_docker_inspect`)
|
||
- ✅ **MinIO** container `minio/minio:latest` running 4d on VDS — the original blocker is CLEARED.
|
||
- ✅ **postgres:16** running; on networks `proxy` (172.18.x) **and** `shared-dbs`, aliases `postgres`. → hostname `postgres:5432` from `modulair-pipeline`/`modulair-mcp` (which sit on `proxy`) **will resolve**. Compose's cross-stack DB reference is sound, no extra network wiring needed.
|
||
- Minor: postgres runs with `ssl=on`; libpq default `sslmode=prefer` negotiates automatically — not a blocker.
|
||
- ✅ `registry:2.8.3` (`registry.kzntsv.site`) running — images `modulair-pipeline:latest`, `tier1-converter:latest`, `modulair-mcp:latest` were pushed 2026-05-13 (per context-freeze). Verify they're still present before deploy.
|
||
- ✅ `traefik:v2.11` running — compose labels target `modulair-mcp.kzntsv.site`.
|
||
- ⛔ None of the 4 modulair containers exist yet (`lightrag-modulair`, `modulair-pipeline`, `tier1-converter`, `modulair-mcp`) — clean deploy.
|
||
|
||
## Pre-flight — admin must verify/create (NOT visible to read-only vds-ops)
|
||
1. [ ] **MinIO bucket** `modulair-rag` exists. MinIO container is up but bucket membership is internal — create if missing (`mc mb`).
|
||
2. [ ] **Postgres DB** `modulair_rag` exists → `CREATE DATABASE modulair_rag;` if not. lightrag self-creates its schema on first start — do NOT pre-init tables.
|
||
3. [ ] **DNS** `modulair-mcp.kzntsv.site` → VDS IP `89.253.255.94` (likely still points at the dead NAS — repoint via regru/cloudflare).
|
||
4. [ ] **MinIO endpoint sanity:** compose env uses `MINIO_ENDPOINT=minio.kzntsv.site:443` (SSL, hairpin via traefik back to the VDS minio). Confirm traefik route `minio.kzntsv.site` now points at the VDS minio container, and DNS resolves to VDS. (Alternative: direct `minio:9000` on a shared network if both end up on `proxy` — only if the external route is problematic.)
|
||
5. [ ] **Secrets via pass** (pattern from `[secrets-manager-adopt]`):
|
||
```
|
||
pass insert modulair-rag/routerai-key
|
||
pass insert modulair-rag/ollama-api-key
|
||
pass insert modulair-rag/minio-access-key
|
||
pass insert modulair-rag/minio-secret-key
|
||
pass insert modulair-rag/postgres-password
|
||
```
|
||
|
||
## Deploy (Portainer — per `.admin` VDS-ops rule, NOT ssh+compose)
|
||
1. Open Portainer `https://portainer.vds.kzntsv.site`.
|
||
2. New Stack → paste `~/projects/modulair-rag/compose.yml` as-is.
|
||
3. Environment vars — fill from `pass show` + statics:
|
||
- `ROUTERAI_API_KEY` (pass), `ROUTERAI_BASE_URL=https://routerai.ru/api/v1`, `ROUTERAI_EMBED_MODEL=qwen3-embedding-8b`
|
||
- `OLLAMA_API_KEY` (pass), `OLLAMA_HOST=https://ollama.com`, `OLLAMA_LLM_MODEL=qwen3-next:latest`
|
||
- `MINIO_ENDPOINT=minio.kzntsv.site`, `MINIO_PORT=443`, `MINIO_USE_SSL=true`, `MINIO_FORCE_PATH_STYLE=true`, `MINIO_BUCKET=modulair-rag`, `MINIO_REGION=us-east-1`, `MINIO_ACCESS_KEY`/`MINIO_SECRET_KEY` (pass)
|
||
- `POSTGRES_USER`, `POSTGRES_PASSWORD` (pass), `POSTGRES_DB=modulair_rag`
|
||
- `IMAGE_TAG=latest`
|
||
4. Pre-pull images, Deploy.
|
||
|
||
## Acceptance
|
||
- [ ] 4 containers up, no restart-loops (`ops_docker_ps` on VDS).
|
||
- [ ] `lightrag-modulair :9621` healthcheck responds (internal network, not exposed).
|
||
- [ ] `https://modulair-mcp.kzntsv.site` → 200 on root.
|
||
- [ ] `modulair-pipeline` + `tier1-converter` NOT in CPU-spin — fix `[pipeline-tier1-cpu-fix]` already in `:latest` (commits `24c9bd5` + `ceb525e`).
|
||
- [ ] `modulair-pipeline` log: no `connection refused` to postgres.
|
||
- [ ] pipeline log: no `NoSuchBucket` / `AccessDenied` from MinIO.
|
||
|
||
## Post-deploy
|
||
1. Rewrite `~/projects/modulair-rag/.wiki/entities/portainer-stack.md` for VDS (Docker host, Portainer URL, MinIO endpoint).
|
||
2. Close this 🟢 task with note.
|
||
3. `[future-resilient-architecture-goals]` — add modulair-rag to RTO/RPO matrix (low priority, non-client prod, but host-loss recovery should be documented).
|
||
|
||
## Decisions log
|
||
- 2026-05-27 (exec session, .admin): pre-flight verify вскрыл, что таска писалась по stale-предпосылкам. Исправления:
|
||
- **Registry образы ОТСУТСТВУЮТ** (open-q #1 = NO). Registry на VDS переустановлен с нуля 2026-05-20 (`pass vds-kzntsv/full-env` коммент: "не мигрировали с kreknin — user accepts loss"). Образы push'ились 2026-05-13 на kreknin-registry → потеряны. Каталог `registry.kzntsv.site/v2/_catalog`: только books/board/vds-ops. → **rebuild всех 3 обязателен.**
|
||
- **MINIO_ENDPOINT исправлен** (open-q #2): `minio.kzntsv.site` → 89.253.255.**133** (books VDS, чужой!). Правильный route на этом VDS = `minio.vds.kzntsv.site` (traefik label на minio-контейнере = `Host(minio.vds.kzntsv.site)`, health/live=200). tier1-converter сидит только на `modulair` bridge (НЕ на `proxy`) → внутренний `minio:9000` для него не резолвится → нужен внешний route для всех. Итог env: `MINIO_ENDPOINT=minio.vds.kzntsv.site`, PORT=443, SSL=true. Compose не редактирую (paste-as-is), это env-var.
|
||
- **MinIO creds:** создан scoped svcacct `modulair-rag-svc` (политика — только bucket `modulair-rag`, verified видит только свой bucket). Не root. → `pass modulair-rag/minio-{access,secret}-key`.
|
||
- **MinIO bucket** `modulair-rag` создан (mc, локальный alias на minio.vds.kzntsv.site root creds).
|
||
- **Postgres DB** `modulair_rag` создана на `postgres.vds.kzntsv.site:5432` (docker one-off psql, sslmode=require). POSTGRES_USER=postgres (shared), pwd из `pass vds-kzntsv/full-env`.
|
||
- **DNS** `modulair-mcp.kzntsv.site` → был 94.19.247.14 (мёртвый NAS = домашний IP). User поправил → 89.253.255.94 (verified public+authoritative).
|
||
- **ROUTERAI key** не было в pass — user дал, сохранён `pass modulair-rag/routerai-key`. OLLAMA key = `pass interns/ollama-cloud-api-key`.
|
||
- **Portainer API key протух** (401) → auth через admin user/pass (`pass vds-kzntsv/full-env`), endpoint=1 (local).
|
||
- **BUILD-БЛОКЕР: tier1-converter = 10.2GB (3.36GB сжатый).** Push с локальной машины падал HTTP 499 (home-канал/middlebox рвёт многогиг HTTPS-заливку через traefik; docker push монолитный, без resume). → **Решение: сборка НА VDS** (`ssh vitya@89.253.255.94` ключ `~/.ssh/id_ed25519`, vitya в docker-группе; клон приватного репо из локального gitea `git.kzntsv.site/cancel_music/modulair-rag` юзером OpeItcLoc03 + admin-token; push registry локально). services/ на origin/master идентичен локальному HEAD (cpu-fix ceb525e внутри). IMAGE_TAG=0.1.0.
|
||
|
||
- 2026-05-27 (DEPLOY ✅): стек `modulair-rag` (Portainer stack id=15, endpoint=1) развёрнут. Образы 0.1.0 (pipeline/tier1-converter/modulair-mcp) собраны на VDS + lightrag ghcr.io/hkuds/lightrag:latest. Acceptance 6/6 green: 4 контейнера up без restart-loop; lightrag Uvicorn :9621 startup complete; https://modulair-mcp.kzntsv.site → 200 (uvicorn, /docs тоже 200); pipeline+tier1 0% CPU (cpu-fix ok); pipeline-лог без postgres/minio ошибок.
|
||
- **+1 compose-фикс при деплое:** лейбл `traefik...modulair-mcp.entrypoints=https` — NAS-имя; на этом VDS entrypoints = `web`/`websecure` (читал `/opt/stacks/traefik/data/traefik.yml`). Без фикса traefik default-404. Поправлено в live-стеке (PUT, https→websecure). **TODO в репо modulair-rag/compose.yml** — закоммитить тот же фикс, иначе следующий redeploy снова 404.
|
||
|
||
## Post-deploy / follow-ups
|
||
- [x] **modulair-rag/compose.yml** — `entrypoints=https`→`websecure` (commit `fc68a16`). Параллельная сессия завершилась (tree clean), сделал сам.
|
||
- [x] **modulair-rag/.wiki/entities/portainer-stack.md** — переписана под VDS (stack 15, websecure, minio.vds, build-on-VDS workflow, model-config table) + log.md entry (`fc68a16`).
|
||
- [x] **#3 lightrag embedding — RESOLVED & verified end-to-end.** Был latent-баг (с NAS): stock-образ LightRAG читает `LLM_*`/`EMBEDDING_*`, а не `ROUTERAI_*`/`OLLAMA_*` → дефолты (`binding=ollama model=None dim=1024`). Замаплены stack-inputs на нативные имена в compose: `EMBEDDING_BINDING=openai` (RouterAI qwen/qwen3-embedding-8b, **DIM=4096**), `LLM_BINDING=ollama` (Ollama Cloud). LLM-тег `qwen3-next:latest` → 404 на Ollama Cloud → исправлен на **`qwen3-next:80b-cloud`** (user-confirmed; `qwen3-next:80b` тоже валиден). Live-smoke на VDS: insert → embedding(RouterAI)+extraction(Ollama) → `4 entities, 3 relations`, потом workspace очищен. `compose.yml`+`lightrag-models.md` (`fc68a16`).
|
||
- [ ] `[future-resilient-architecture-goals]` — добавить modulair-rag в RTO/RPO matrix (low prio).
|
||
- [ ] **PUSH:** `modulair-rag fc68a16` + `.admin` task-коммиты unpushed (автопуш не выдан в этой сессии).
|
||
|
||
## Open questions
|
||
- [x] Registry images present? → NO, rebuilt on VDS (см. Decisions).
|
||
- [x] minio.kzntsv.site repointed? → NO, это books VDS; используем `minio.vds.kzntsv.site`.
|
||
|
||
**Branch:** master
|
||
<!-- created-by: vitya@modulair-rag-session / 2026-05-27 / from: modulair-rag nas-loss-vds-redeploy-context concept; user handed deploy to .admin (has VDS access) -->
|