From 12c7dc5ab9f1079fda225ae9464f15e2b95061db Mon Sep 17 00:00:00 2001 From: vitya Date: Tue, 26 May 2026 18:32:29 +0300 Subject: [PATCH] =?UTF-8?q?tasks(NEXT=5FSESSION):=20session-close=20?= =?UTF-8?q?=E2=80=94=20bookva=20tenant=20deep-debug=20+=206-bug=20post-mor?= =?UTF-8?q?tem?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Multi-fix session: bookva tenant works correctly после 6 fix'ов одной природы (incomplete cutover-prep без e2e smoke). См. NEXT_SESSION.md § "What's LIVE" + § "Memory updates" + shared wiki concept tenant-overlay-config-volume-mount-path-pitfall. Changes: - host-stacks/books-vds/ops-mcp.compose.yml: add BOOKVA_MARIADB_PASSWORD env declaration + multi-tenant docs note. Stack 26 ещё не пере-PUT'нут на это, открытая микро-таска ops-mcp-multi-tenant-stack-activate в NEXT_SESSION. - STATUS.md: 6-bug summary в latest update line. - NEXT_SESSION.md: full rewrite с 14 recent commits, open треками, asks к user'у, memory updates про agenda.db.collection / composite jobId / scheduler envs / bookva auth tokens. Co-Authored-By: Claude Opus 4.7 (1M context) --- .tasks/NEXT_SESSION.md | 122 +++++++++++----------- .tasks/STATUS.md | 2 + host-stacks/books-vds/ops-mcp.compose.yml | 8 +- 3 files changed, 67 insertions(+), 65 deletions(-) diff --git a/.tasks/NEXT_SESSION.md b/.tasks/NEXT_SESSION.md index 6a1f79e..88311d9 100644 --- a/.tasks/NEXT_SESSION.md +++ b/.tasks/NEXT_SESSION.md @@ -1,93 +1,89 @@ --- -_last_updated_: 2026-05-27T00:30:00+03:00 -session_id: 2026-05-26-ops-mcp-host-promote-CLOSED +_last_updated_: 2026-05-27T01:30:00+03:00 +session_id: 2026-05-26-bookva-tenant-deep-debug --- # Next session handoff ## Recent commits -- `victor/books ae3ab14` — feat(deploy): ops-mcp host-promote — drop tenant pipeline (deploy.yml + delete ops-mcp.compose.yml) -- `victor/bookva-overlay 50f5bbb` — chore(deploy): drop ops-mcp.compose.yml (host-promoted в .admin) -- `victor/bookva-overlay 437d024` — fix(deploy): bookva-ops-mcp idle-stub command (interim, теперь obsolete) -- `.admin × 2` — tasks `books-ops-mcp-host-promote` 🟢 closed + handoff refresh + new `host-stacks/books-vds/ops-mcp.compose.yml` -- `.wiki b2528fb` — wiki(concepts): ingest 3 generic ops-gotchas from bookva tenant-split (manual write, MCP commitFile bug) -- `victor/books c434230` — deploy.yml: auto-deploy default tenant=slovo → all (symmetric rollout active) -- `victor/bookva-overlay 2862279` — traefik labels: entrypoints=https + certresolver=letsEncrypt (books-VDS instance naming) -- `victor/bookva-overlay 5474b10` — elasticsearch.compose.yml: add reindex.remote.whitelist=elasticsearch:9200 -- `victor/bookva-overlay b324e12` — add bookva-minio + external port-bind для bookva-db -- `victor/bookva-overlay 8bffd68` — *.compose.yml: drop depends_on (per-stack model) -- `victor/bookva-overlay 2b20a3e` — api.compose.yml: remove traefik (internal-only per design v3) -- `victor/bookva-overlay 93c2ea2` — mongo.compose.yml: pin mongo:4.2 (WT format compat) -- `victor/bookva-overlay 23eb374` — db.compose.yml: pin mariadb:10.6 (cp -a parity) -- `victor/bookva-overlay 264373d` — .env.example: design v3 hostnames -- `victor/bookva-overlay 5b173de` — config-templates align to prod shape + design v3 hostname -- `victor/books 75ea320` — feat(deploy): web.compose.yml embed-api env switch -- `victor/books 2f7b539` — fix(ci): rename secrets.GITEA_TOKEN → BOOKVA_OVERLAY_TOKEN -- `.admin × 5` — task progress, closure, NEXT_SESSION refreshes +- `victor/books 1da3093` — fix(jobs.post): bypass agenda.db.collection → MongoClient (idempotency check fix for embed-api) +- `victor/books e083b71` — fix(files): composite agenda jobId — strip :timestamp before ObjectId.isValid (display fix) +- `victor/books 60382bf` — fix(deploy): books-web-config mount (slovo embed-api Mongo endpoint was stale external) +- `victor/books fc33ea5` — fix(deploy): scheduler — job-scheduler↔scheduler naming bridge tenant-aware +- `victor/books f5f295b` — feat(ops-mcp): multi-tenant pools — MariaDB/Mongo per tenant (0.0.1 → 0.1.0) +- `victor/books ae3ab14` — feat(deploy): ops-mcp host-promote — drop tenant pipeline +- `victor/bookva-overlay 8c104f7` — fix(scheduler): add RECONCILER_ENABLED/TASK_RUNNER_URL/DOCKER_HOST envs +- `victor/bookva-overlay 60e39e5` — fix(deploy): bookva-scheduler healthcheck (mirror slovo's 58b13e0) +- `victor/bookva-overlay a9063f0` — fix(deploy): bookva mount paths /app → /usr/src/app +- `victor/bookva-overlay 4635305` — fix(deploy): bookva-api/web pass NITRO_*/NUXT_* auth env in compose +- `victor/bookva-overlay b6f9144` — fix(web): bookva-web-config volume mount — DB/Mongo/S3 leak fix +- `victor/bookva-overlay 50f5bbb` — chore(deploy): drop ops-mcp.compose.yml (host-promoted) +- `victor/bookva-overlay 437d024` — fix(deploy): bookva-ops-mcp idle-stub (obsolete после delete) +- `.admin × 2` — host-stacks/books-vds/ops-mcp.compose.yml multi-tenant env + task closure + handoff +- `~/projects/.wiki fe3ae78` — ingest concepts/tenant-overlay-config-volume-mount-path-pitfall (shared) ## What's LIVE сейчас -- **bookseller.kzntsv.site** → bookva-web (embed-api), 200 OK, login-gate active (id=1=pobeda2003, id=2=vitya) -- **bookva.kzntsv.site** → books-web (slovo, embed-api тоже), 200 OK, fallback к books-api stack для legacy paths -- **books.kzntsv.site** → books-api (legacy stack), ещё running 24-48h soak до shutdown micro-task -- **10 bookva Portainer stacks** all ACTIVE (db/mongo/es/minio/api/web/scheduler/task-runner/ops-mcp/ntfy). bookva-ozon-mcp пока не создан (image отсутствует в registry). -- **External admin endpoints:** - - `mysql -h 89.253.255.133 -P 33306 -uroot -p${BOOKS_DB_ROOT_PASSWORD} books` - - S3 client → `http://89.253.255.133:9001`, AKIA same as books (см. pass `books-vds/full-env`) -- **Auto-deploy симметричный** — push в victor/books master → build → deploy.yml tenant=all → PUT slovo (22/23/24/26) + bookva (47/45/46/43) +**Both tenants полностью изолированы (final audit pass):** + +- `bookseller.kzntsv.site` → bookva-web@master-1da3093 → bookva-{db,mongo,es,minio,api,scheduler,task-runner}. Auth `2f2e96e2-25f5-4653-930d-fd307a170543`. +- `bookva.kzntsv.site` → books-web@master-1da3093 (slovo) → books-{db,job-scheduler-mongo,...}. Auth `81e123d0-8edd-...` (slovo's original, не менял). +- bookva-scheduler reconciler **registered** all handlers (`load products to warehouse` ровно processed файл 2674 в `2026-05-26 14:20:45`). +- books-web embed-api (`POST /api/jobs`) теперь без 500 — idempotency через MongoClient. +- 11 Portainer stacks LIVE на books VDS: books-{api,web,job-scheduler,task-runner,db,ntfy} (slovo) + bookva-{api,web,scheduler,task-runner,db,mongo,es,minio,ntfy} + books-ops-mcp host-level + books-docker-proxy{,-ro} + traefik/portainer/registry/mongo/minio/imgproxy и т.д. ## Open треки | Трек | Готовность | Entry-point | |---|---|---| -| `books-ops-mcp-host-promote` 🟢 **closed** — see Recent commits below | n/a | -| `books-api-shutdown` (NEW micro-task) | 24-48h soak ends ~2026-05-27/28. `docker stop books-api` через Portainer | TBD `.tasks/books-api-shutdown.md` | -| `bookva-ozon-mcp-image-build` (NEW micro-task) | image `books-ozon-mcp:master` не в registry — нужен build pipeline в victor/books | TBD `.tasks/bookva-ozon-mcp-image-build.md` | -| `bookva-tenant-cutover` task **не понадобилась** — cutover de-facto произошёл в этой сессии (Step 6 descoped по решению user'а) | n/a — manual heads-up учредителю Bookva | -| `bookva-minio-https-dns` (optional) | reg.ru A `bookva-minio.kzntsv.site` → 89.253.255.133 + `push.bookseller.kzntsv.site` → 89.253.255.133 для traefik HTTPS вместо port-bind | reg.ru DNS UI | -| `mariadb-mongo-upgrade-chain` (eventual post-cutover) | bookva-db 10.6 → 10.11, bookva-mongo 4.2 → 4.4 → 5 → 6 → 7 chain. Когда usage стабилизируется | TBD | -| `stateful-split-volume-copy` ⚪ | bookva-side done; slovo-side scope deferred (отдельный slovo-cutover) | `.tasks/stateful-split-volume-copy.md` | -| `infra-inventory` ⚪ | books VDS теперь 14 stacks (+ bookva-minio + 9 bookva) — момент удачный для ingest factsheet | `.tasks/infra-inventory.md` | -| `books-bookva-user-whitelist-gathering` ⚪ | login-gate уже applied (id=1+2 only) — таска осталась только для verification списка от учредителя Bookva post-cutover | `.tasks/books-bookva-user-whitelist-gathering.md` | -| `books-vds-bookva-bootstrap` 🔵 | blocked on Phase 4 (Rusonyx или другой VDS для Bookva billing). Long-term — bookva сейчас на shared books VDS | `.tasks/books-vds-bookva-bootstrap.md` | -| `books-dns-cutover-bookva` 🔵 | blocked on books-vds-bookva-bootstrap | `.tasks/books-dns-cutover-bookva.md` | +| **`ops-mcp-multi-tenant-stack-activate`** ⚪ NEW micro-task — image `books-ops-mcp:master` уже rebuilt (master-1da3093) с multi-tenant code + VDS host config `/opt/books/books-ops-mcp/config/default.json` уже на новой schema. **TODO:** PUT Portainer stack 26 с pullImage:true + добавить `BOOKVA_MARIADB_PASSWORD` env (same value as `MARIADB_PASSWORD` — bookva-db = cp -a of books-db, ops_ro password identical). Compose `.admin/host-stacks/books-vds/ops-mcp.compose.yml` уже declared в этом session'е. | TBD `.tasks/ops-mcp-multi-tenant-stack-activate.md` (или inline в session) | +| `books-api-shutdown` (ранее заведённая micro-task) | 24-48h soak with bookva LIVE — нужно проверить если still applicable (books-api всё ещё нужен для slovo на bookva.kzntsv.site fallback пути?). | TBD | +| `bookva-ozon-mcp-image-build` | image отсутствует в registry, bookva-ozon-mcp stack не создан. Если service не используется — skip навсегда. | TBD | +| `bookva-mongo-cleanup` ⚪ (user-deferred) | 2817 agendaJobs в bookva-mongo from cp-a — user сказал «8 не трогай не стирай». Когда будет готов rotate — отдельная таска. | TBD | +| `bookva-ozon-credentials-audit` ⚪ (user-deferred) | bookva ozonSeller.clientId=50542+94191 = slovo's. Финансовый риск. User сказал «сам подумаю об утечках». | TBD | +| `stateful-split-volume-copy` ⚪ | bookva-side done; slovo-side scope deferred. | `.tasks/stateful-split-volume-copy.md` | +| `infra-inventory` ⚪ | books VDS теперь 14+ stacks — момент удачный для ingest factsheet. | `.tasks/infra-inventory.md` | +| `books-bookva-user-whitelist-gathering` ⚪ | login-gate уже applied — таска только для verification списка post-cutover. | `.tasks/books-bookva-user-whitelist-gathering.md` | +| `books-vds-bookva-bootstrap` 🔵 blocked on Phase 4 (Rusonyx или другой VDS для Bookva billing) | long-term — bookva сейчас на shared books VDS. | `.tasks/books-vds-bookva-bootstrap.md` | +| `books-dns-cutover-bookva` 🔵 blocked on books-vds-bookva-bootstrap | | `.tasks/books-dns-cutover-bookva.md` | ## Спроси user'а -- **Multi-tenant DB observability через ops-mcp** (deferred design call) — нужно ли видеть bookva-db / bookva-mongo через MCP? Если да — extend lib (per-tenant pool array) ИЛИ drop DB tools оставив только docker. -- **Manual heads-up учредителю Bookva** (Step 6 substitute) — отправить ей сообщение «Bookva теперь на `https://bookseller.kzntsv.site`, login те же». Когда? -- **`books-api-shutdown`** через 24-48ч — ad-hoc через Portainer или tracker task? -- **`bookva-ozon-mcp-image-build`** — нужен build в victor/books CI? Сейчас image отсутствует, bookva-ozon-mcp stack не создан. Если service не используется — skip навсегда? +- **`ops-mcp-multi-tenant-stack-activate`** — finish сейчас (PUT stack 26 + add env) или взять отдельной таской? +- **Manual heads-up учредителю Bookva** — Bookva теперь на `https://bookseller.kzntsv.site`, login те же. Когда отправляешь? +- **`books-api-shutdown`** — поверка: books-api всё ещё нужен для legacy paths fallback на bookva.kzntsv.site? Если нет — shutdown candidate. +- **slovo-overlay parity audit** — slovo's compose файлы (book/deploy/) могут страдать от тех же багов что были у bookva (mount path, RECONCILER_ENABLED). Сейчас работает только потому что slovo's mount был через `/opt/...` bind (не named volume), который случайно совпал с правильным WORKDIR. Audit перед follow-up cutover? - **bookva-minio creds rotation** — сейчас AKIA same as books-minio. Rotate в отдельную task'у когда стабилизируется? -- **slovo-overlay parity audit** — `.env.example` и templates вероятно stale similarly to bookva pre-fix. Audit перед slovo-cutover task? ## Не делать (preemptive guards) -- **Не stop bookva-{db,mongo,es,minio,api,web,scheduler,task-runner,ops-mcp,ntfy}** — все LIVE на bookseller.kzntsv.site, любой stop = production outage Bookva tenant'а. -- **Не editing config volumes через alpine** — source of truth = `bookva-overlay/config-templates/`. Edit → commit → push → re-populate. -- **Не stop books-api** до явного soak end (~2026-05-27/28) + user sign-off. Внешние callers всё ещё на `https://books.kzntsv.site/*`. -- **Не push без grant в next session** — auto-push grant был session-only per Rule 4. В этой сессии user grant'нул на «делай авто-push» — следующая session требует нового grant. -- **Не trust DNS wildcard kzntsv.site** — wildcard → 94.19.247.14 (Windows IIS), НЕ books VDS 89.253.255.133. Новые subdomains требуют explicit A-records в reg.ru. -- **Не restore /backups/books-{db,mongo}-pre-split-20260526-*.tar.gz** unless rollback. Manual cleanup после 7+ дней. +- **Не trust деплой "успех" без end-to-end smoke** — этот session показал 6 багов одной природы которые passed CI но broke runtime. После любого tenant infra изменения: login + write + verify tenant DB + processing-job + UI status. +- **Не stop bookva-{db,mongo,es,minio,api,web,scheduler,task-runner,ntfy}** — LIVE на bookseller.kzntsv.site, любой stop = production outage. +- **Не editing config volumes через alpine напрямую** — source of truth = `bookva-overlay/config-templates/` (для api/scheduler/task-runner) + host-mounted (для web). Edit → commit → push → re-populate. +- **Не stop books-api** до явного soak end + user sign-off. +- **Не push без grant в next session** — auto-push grant был session-only. Этот session был grant'нут. +- **`ops_ro` password в обоих tenant'ах identical** (cp -a). Если будут rotate'ить — оба MariaDB одновременно. +- **bookva-mongo.agendaJobs 2817 docs from slovo cp-a НЕ ТРОГАТЬ** — user explicit "8 не трогай не стирай". ## Memory updates / wiki ingests Ingested в shared wiki `~/projects/.wiki/concepts/`: -- `gitea-reserved-secret-prefix` — GITEA_* / GITHUB_* prefix reserved (PUT → 400) -- `mongo-wt-format-major-version-incompat` — Mongo 4→7 crash exit 62, chain upgrade FCV -- `portainer-per-stack-depends-on-pitfall` — cross-stack depends_on "undefined service" +- `tenant-overlay-config-volume-mount-path-pitfall` — 6-bug post-mortem из этого session'а. Includes audit checklist + Lesson о mandatory end-to-end smoke. -Local memory updates за сессию (не ingested, books-specific): -- DNS wildcard kzntsv.site → 94.19.247.14 (Windows IIS, не books-vds) -- bookva passwords plaintext в DB (varchar(45), TODO bcrypt в login.post.js) -- books-vds traefik instance naming: `entrypoints=https` + `certresolver=letsEncrypt` (case-sensitive) -- Embed-api auth gate в books-web identical к books-api (401 без token, 401 invalid JWT) -- Multi-tenant MinIO design: separate container vs bucket-split. User выбрал separate container (zero-config-change в apps). +Local memory updates (books-specific, не ingested): +- `agenda.db.collection` populated только после `await agenda.start()`. В web/api контейнерах start не вызывается (worker logic в scheduler) → idempotency findOne fails. Используй отдельный MongoClient. +- `composite agenda jobId` (`:`) пишется через `job-scheduler/lib/handlers/http.js`. ObjectId.isValid возвращает false на 38-char composite — fix: split на `:` перед `.isValid`. +- bookva-scheduler нужны 3 envs для full operation: `RECONCILER_ENABLED=true`, `TASK_RUNNER_URL=http://bookva-task-runner:3000`, `DOCKER_HOST=tcp://books-docker-proxy:2375`. +- bookva auth: NITRO_AUTH_TOKEN/NUXT_AUTH_TOKEN/NITRO_JWT_SECRET_KEY/NUXT_JWT_SECRET_KEY = `2f2e96e2-25f5-4653-930d-fd307a170543` / `5e5ef0460549f36d9ab276882b490b2b77a266e41dc8a6478415869768c1ee36`. Stored in Portainer stack 45 (api) + 46 (web) Env. **Slovo's auth не трогали** (81e123d0/505172e5). +- `_loadFiles` cron в config.jobs больше **нигде не handle'ится** (deprecated после tasksConfig.json migration). File uploads requires manual trigger "Запустить обработку" UI button → POST /api/jobs. +- mongo-shell command на bookva-mongo container — image `mongo:4.2`, use `docker exec bookva-mongo mongo --quiet --eval '...'` (legacy mongo client, NOT mongosh). +- books-db root password в pass `books-vds/full-env` как `BOOKS_DB_ROOT_PASSWORD`. Same для bookva-db (cp-a). ## Pass / Reference -- pass entry `gitea/victor-books-ci-bookva-overlay` (PAT scope=read:repository) — backup для BOOKVA_OVERLAY_TOKEN secret в victor/books. -- pass entry `books-vds/full-env` — BOOKS_PORTAINER_URL+API_KEY, BOOKS_DB_ROOT_PASSWORD, MINIO_ACCESS_KEY. -- Backups на books VDS — `/backups/books-{db,mongo}-pre-split-20260526-*.tar.gz`. -- **Reference:** [[bookva-tenant-cutover-prep]] § Closure note + Post-closure follow-up — full session trace. +- pass `books-vds/full-env` — VDS root SSH key path (`~/.ssh/id_ed25519_books_ops`), Portainer URL+API key, books-db root password. +- pass `gitea/victor-books-ci-bookva-overlay` — PAT (read:repository) backup для BOOKVA_OVERLAY_TOKEN secret. +- Backups на books VDS — `/backups/books-{db,mongo}-pre-split-20260526-*.tar.gz` (auto-cleanup post 7 дней). +- Shared wiki concept ingested this session: [tenant-overlay-config-volume-mount-path-pitfall](~/projects/.wiki/concepts/tenant-overlay-config-volume-mount-path-pitfall.md). +- **Reference task:** [[books-ops-mcp-host-promote]] (closed) — все 5 design Qs locked + 10-step playbook executed. diff --git a/.tasks/STATUS.md b/.tasks/STATUS.md index 504ac08..9313a03 100644 --- a/.tasks/STATUS.md +++ b/.tasks/STATUS.md @@ -1,4 +1,6 @@ # Admin Task Board +_Updated: 2026-05-27 (ночь) — bookva-tenant deep-debug session. 6 багов одной природы (incomplete cutover-prep): (1) bookva-web без config volume → slovo DB leak; (2) bookva-{api,scheduler,task-runner} mount path /app→/usr/src/app; (3) NITRO/NUXT env не reference'или в compose; (4) bookva-scheduler без RECONCILER_ENABLED → dry-run handlers; (5) bookva-scheduler без healthcheck → CI timeout; (6) deploy.yml svc=job-scheduler vs secret=SCHEDULER mismatch. + composite agendaJobId display fix + books-web config mount (slovo also missed) + jobs.post.js agenda.db.collection→MongoClient. Ingested wiki concept tenant-overlay-config-volume-mount-path-pitfall (shared). ops-mcp multi-tenant код в image готов но stack 26 ещё не пере-PUT'нут (next session)._ + _Updated: 2026-05-26 (поздний вечер) — `books-ops-mcp-host-promote` 🟢 **closed** в ту же сессию следом за заведением. Stack 26 (books-ops-mcp) in-place PUT в host-level compose без container churn. Stack 43 (bookva-ops-mcp) deleted. 2 Gitea secrets revoked. `victor/books ae3ab14` + `victor/bookva-overlay 50f5bbb` + `.admin` host-stacks/books-vds/ops-mcp.compose.yml. ops-mcp теперь management-plane (manual Portainer)._ _Updated: 2026-05-26 (поздний вечер) — заведена `books-ops-mcp-host-promote` ⚪ (deploy run#437/439 валился на bookva-ops-mcp restart-loop → hot-fix `bookva-overlay 437d024` idle-stub interim → user ideology «books VDS = host, slovo/bookva = клиентские стэки» → promote `books-docker-proxy-ro`+`books-ops-mcp` в host-level stack, drop bookva-ops-mcp). Deploy run#440 зелёный._ diff --git a/host-stacks/books-vds/ops-mcp.compose.yml b/host-stacks/books-vds/ops-mcp.compose.yml index e0cda97..01e0ee3 100644 --- a/host-stacks/books-vds/ops-mcp.compose.yml +++ b/host-stacks/books-vds/ops-mcp.compose.yml @@ -3,8 +3,9 @@ # Ideology: books VDS = наш host, slovo/bookva = клиентские tenant-стэки. # Один host-wide ops-mcp видит все контейнеры через `tcp://books-docker-proxy-ro:2375`. # -# Tenant DB scope (current): slovo's books-db + slovo's Mongo (per ops-mcp/config/default.json). -# Bookva-tenant DB observability через MCP — design call отложен (см. .tasks/books-ops-mcp-host-promote.md). +# Tenant DB scope: per-tenant pools (slovo + bookva). Tools принимают optional +# `tenant: 'slovo'|'bookva'` (default = config.defaultTenant). См. config/default.json +# в `packages/ops-mcp/` (mirrored на host'е через mount). # # Lifecycle: # - Managed manually через Portainer UI (как traefik + portainer — management plane exception @@ -50,7 +51,10 @@ services: NODE_ENV: production TZ: Europe/Moscow DOCKER_HOST: tcp://books-docker-proxy-ro:2375 + # MARIADB_PASSWORD = slovo's books-db ops_ro; BOOKVA_MARIADB_PASSWORD = bookva-db's + # (bookva-db = cp -a of books-db, ops_ro password идентичен; rotate отдельно). MARIADB_PASSWORD: ${MARIADB_PASSWORD} + BOOKVA_MARIADB_PASSWORD: ${BOOKVA_MARIADB_PASSWORD} volumes: # ВНИМАНИЕ: на VPS файл /opt/books/books-ops-mcp/config/default.json # должен существовать ДО первого старта стека, иначе Docker создаст