From 86c44c3db90a07dff1015d2f655022395208469e Mon Sep 17 00:00:00 2001 From: vitya Date: Mon, 25 May 2026 10:27:23 +0300 Subject: [PATCH] =?UTF-8?q?tasks(books-vds-stacks-to-portainer):=20close?= =?UTF-8?q?=20=F0=9F=9F=A2=20=E2=80=94=206=20stacks=20migrated=20SSH-compo?= =?UTF-8?q?se=20=E2=86=92=20Portainer?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2 ladder executed 2026-05-25 (~10min end-to-end, lowest → highest blast radius): proxy-chain (id 28) → imgproxy (29) → minio (30) → mongo (31) → books-db (32) → elasticsearch (33) All 6 smoked green. books-api + books-task-runner reconnected transparently through mongo + books-db recreate windows. ES snapshot repo `kreknin` preserved through bind. Phase 3: - created .wiki/concepts/portainer-stack-management-books-vds.md (pattern application + diffs from vds-kzntsv) - entity books-vds.md: 6 stacks flipped SSH-managed → Portainer-managed - index.md: new concept entry Pre-flight quirks documented (CentOS 7 yum dead, no jq, no docker login). Acceptance: 4/5 verified, backup pipeline (#5) verified by inspection (bind paths preserved) — full pipeline run pending next 06:00 MSK. Co-Authored-By: Claude Opus 4.7 (1M context) --- .tasks/NEXT_SESSION.md | 47 +++---- .tasks/STATUS.md | 11 +- .tasks/books-vds-stacks-to-portainer.md | 21 ++- .../portainer-stack-management-books-vds.md | 127 ++++++++++++++++++ .wiki/entities/books-vds.md | 43 +++--- .wiki/index.md | 1 + 6 files changed, 192 insertions(+), 58 deletions(-) create mode 100644 .wiki/concepts/portainer-stack-management-books-vds.md diff --git a/.tasks/NEXT_SESSION.md b/.tasks/NEXT_SESSION.md index 2143155..e2f245f 100644 --- a/.tasks/NEXT_SESSION.md +++ b/.tasks/NEXT_SESSION.md @@ -1,5 +1,5 @@ --- -_last_updated_: 2026-05-25T11:15:00+03:00 +_last_updated_: 2026-05-25T12:00:00+03:00 session_id: 2026-05-25-books-vds-stacks-to-portainer --- @@ -7,8 +7,9 @@ session_id: 2026-05-25-books-vds-stacks-to-portainer ## Recent commits -- `tasks(books-vds-stacks-to-portainer): Phase 0 probe complete, +imgproxy scope` (`bfb942cd`) — Phase 0 done, scope expanded 5→6 stacks. -- `tasks(NEXT_SESSION): refresh handoff after this session's commits` (`3769e836`) — prior session handoff. +- (about to commit) `tasks(books-vds-stacks-to-portainer): close 🟢 — 6 stacks migrated, wiki + entity updated` — final session commit. +- `scripts(books-vds-portainer-migration): Phase 1 adapter ready` (`141efe97`) +- `tasks(books-vds-stacks-to-portainer): Phase 0 probe complete, +imgproxy scope` (`bfb942cd`) - `tasks: add ⚪ books-vds-stacks-to-portainer` (`439da3ce`) - `backup(books-vds): daily 06:00 MSK pipeline to kreknin live 🟢` (`fe41ee44`) @@ -16,38 +17,32 @@ session_id: 2026-05-25-books-vds-stacks-to-portainer | Трек | Готовность | Entry-point | |---|---|---| -| `books-vds-stacks-to-portainer` 🔴 | **Phase 0 done, Phase 1 next** — write adapter script `scripts/books-vds-portainer-migrate.sh` | `.tasks/books-vds-stacks-to-portainer.md` | -| `stateful-split-volume-copy` ⚪ | not started; playbook fits real books VDS (89.253.255.133) | `.tasks/stateful-split-volume-copy.md` | +| `stateful-split-volume-copy` ⚪ | not started; lean playbook готов, fits real books VDS (89.253.255.133) | `.tasks/stateful-split-volume-copy.md` | | `iis-migration-to-ruvds` 🟡 | soak (kupimknigi + emspb DNS flipped 2026-05-24, 22 pending) | `.tasks/iis-migration-to-ruvds.md` | -| `infra-inventory` ⚪ | ready, не начат — books VDS должен попасть в admin-detailed runbook | `.tasks/infra-inventory.md` | -| `books-bookva-user-whitelist-gathering` ⚪ | ready, long-lead gathering | `.tasks/books-bookva-user-whitelist-gathering.md` | +| `infra-inventory` ⚪ | ready, не начат — **обновлён factsheet `books-vds` после Portainer migration**, готов к ingest в admin-detailed runbook | `.tasks/infra-inventory.md` | +| `books-bookva-user-whitelist-gathering` ⚪ | ready, long-lead gathering (email-based) | `.tasks/books-bookva-user-whitelist-gathering.md` | | `books-vds-bookva-bootstrap` 🔵 | blocked on tenant-split Phase 4 | `.tasks/books-vds-bookva-bootstrap.md` | | `books-dns-cutover-bookva` 🔵 | blocked on books-vds-bookva-bootstrap | `.tasks/books-dns-cutover-bookva.md` | - -## Mid-task capture - -**left mid-task:** `books-vds-stacks-to-portainer` (Phase 1 entry) - -**where_stopped:** Phase 0 closed — все 6 compose probed (`books-db`, `mongo`, `minio`, `elasticsearch`, `proxy-chain`, `imgproxy`), Portainer PAT verified, endpoint 6 stostayer = not ours. - -**next concrete:** Write `scripts/books-vds-portainer-migrate.sh` (clone-adapt из `.wiki/concepts/portainer-stack-management-vds.md` § Migration script). Diffs: `PORTAINER_URL=https://portainer.kzntsv.site`, endpointId=1, dir prefix `/usr/docker/`, auth `X-API-Key` (PAT works, no JWT fallback). Then Phase 2 ladder: proxy-chain → imgproxy → minio → mongo → books-db → elasticsearch. +| `books-stateful-split-execution` 🔵 | blocked on dev-prep + per-tenant-backup-and-observability в victor/books | `.tasks/STATUS.md` | ## Спроси user'а -- **(нет на этом раунде — auto-push grant active, scope decisions made, момент migrate-by-migrate ладдером).** +- **Verify backup pipeline post-migration:** next scheduled run 06:00 MSK 2026-05-26 (через ~18h от close). Если ntfy/email прийдут с success — acceptance criterion #4 finalized. Если нет — investigate (most likely bind paths intact, low risk). +- **Какую таску брать следующей?** Active candidates: `stateful-split-volume-copy` (ops, lean playbook готов), `infra-inventory` (книги VDS теперь fully documented — момент удачный), `iis-migration-to-ruvds` (DNS TTL reductions для 22 pending hosts). ## Не делать (preemptive guards) -- **Не migrate traefik / portainer** — management plane, recreate ломает access (Skip-list). -- **Не trigger ES recreate без user awareness** — текущая ES «не используется» per user 2026-05-25, но verify before recreate. ES в ладдере **последним** на покое. -- **Не editit `/usr/docker//docker-compose.yml`** напрямую после migration — Portainer-managed working_dir = `/data/compose//`. Compose files в `/usr/docker//` становятся reference only. -- **Не break backup pipeline** — `books-vds-backup-daily-kreknin` rsync paths = `/usr/docker//data` (binds). Bind paths preserved через migration, paths остаются валидны. Verify post-migration rsync прошёл хотя бы 1 раз. -- **Не push without grant in future sessions** — auto-push грант session-only (Rule 4). Next session start = ask-mode again. +- **Не editit `/usr/docker//docker-compose.yml`** на books VDS — теперь reference-only. Portainer-managed compose state в `/data/compose//docker-compose.yml`. Edit через Portainer UI или API. +- **Не `docker compose up -d`** на books VDS для migrated стеков — стартанёт parallel set контейнеров под compose-project ``, conflicting names с Portainer-managed. Trust Portainer для lifecycle. +- **Не push without grant in next session** — auto-push grant был session-only (Rule 4). Next session start = ask-mode again. +- **Не migrate traefik / portainer на books VDS** — management plane, recreate ломает access. Same exclusion as vds-kzntsv. +- **Не trust `docker login registry.kzntsv.site`** на books VDS — НЕ настроен. Кэшированные images сейчас спасают, но force-pull для private images → auth fail. Setup login если planned image-rotation. ## Memory updates за сессию -- **Учить:** Portainer single instance `portainer.kzntsv.site` управляет 2 endpoint'ами — `1` = books VDS local docker daemon, `6` = stostayer (другой клиент, **не наш**, scope не расширять). User-confirmed 2026-05-25. -- **Учить:** PAT auth через `X-API-Key: ptr_...` header работает (без JWT fallback). Pass entry `books-vds/full-env BOOKS_PORTAINER_API_KEY`. -- **Учить:** на books VDS все SSH-compose стеки **bind-only** (нет named volumes для них) и **no env_file** (env inline в compose). Migration упрощается: gotcha #1 (env_file strip) не релевантен, gotcha #2 (absolutize) — единственная требуемая transform. -- **Учить:** `imgproxy` (2 контейнера, `imgproxy.kzntsv.site`, S3→minio) был missing из `entities/books-vds.md` SSH-managed table до 2026-05-25. Fix'нут в этом раунде. Lesson: при entity wiki бутстрапе на legacy host'ах — обязательно `ls /usr/docker/` cross-check vs running `docker ps`, не доверять предположениям. -- **Reference:** migration pattern source — `.wiki/concepts/portainer-stack-management-vds.md` (VDS-infra 12-stack retro-migration 2026-05-22). Adaptable для books VDS с 3-4 diff'ами. +- **Учить:** на books VDS **yum dead** (CentOS 7 EOL 2024-06, mirrors retired). Static binary install — единственный путь для package install (e.g. `curl -fsSL https://github.com/jqlang/jq/releases/.../jq-linux-amd64 -o /usr/local/bin/jq`). +- **Учить:** Portainer single instance `portainer.kzntsv.site` управляет 2 endpoint'ами — `1` = books VDS, `6` = stostayer (другой клиент, не наш, **не расширять scope сюда**). +- **Учить:** Portainer migration ladder lowest→highest blast radius works на books VDS — `proxy-chain → imgproxy → minio → mongo → books-db → elasticsearch`. Books-api + books-task-runner reconnect transparently через mongo/books-db recreate windows (~13-15s). ES cluster yellow normal для single-node. +- **Учить:** `imgproxy` (2 containers, serves `imgproxy.kzntsv.site`, S3 backend → minio) был **missing** из entity wiki до Phase 0 probe — lesson: при entity wiki bootstrap на legacy hosts обязательно `ls /usr/docker/` cross-check vs `docker ps`. +- **Reference:** migration pattern source — [[../wiki/concepts/portainer-stack-management-vds]] (vds-kzntsv); books VDS application + diffs — [[../wiki/concepts/portainer-stack-management-books-vds]]. +- **Reference:** `scripts/books-vds-portainer-migration/{migrate.sh,README.md}` — adapter for future re-use (e.g. если появится ещё SSH-compose стек на books VDS). diff --git a/.tasks/STATUS.md b/.tasks/STATUS.md index 6b33582..2861058 100644 --- a/.tasks/STATUS.md +++ b/.tasks/STATUS.md @@ -1,14 +1,7 @@ # Admin Task Board -_Updated: 2026-05-25 (session start — `books-vds-stacks-to-portainer` поднят 🔴 active. Endpoint 6 = stostayer VDS — НЕ наш, scope не расширяем. Auto-push grant active.)_ +_Updated: 2026-05-25 (`books-vds-stacks-to-portainer` 🟢 closed — 6 стеков (proxy-chain, imgproxy, minio, mongo, books-db, elasticsearch) migrated SSH-compose → Portainer-managed на books VDS. Ids 28-33. Adapter script + wiki concept committed. Все 6 smoke green в одну сессию.)_ -## 🔴 [books-vds-stacks-to-portainer] — migrate SSH-compose стеки на books VDS под Portainer-managed (pattern parallel VDS-infra retro-migration 2026-05-22). Targets: books-db, mongo, minio, elasticsearch, proxy-chain, **imgproxy** (added Phase 0 2026-05-25). Skip traefik + portainer (management plane). - -**Status:** active -**Where I stopped:** Phase 0 done 2026-05-25 — probed 6 compose files + Portainer API. No env_file, all bind volumes, PAT works, endpoint 1 = books VDS (6 Portainer-managed already), endpoint 6 = stostayer (not ours). imgproxy added to scope. -**Next action:** Phase 1 — adapter migration script `scripts/books-vds-portainer-migrate.sh` (clone-adapt of VDS-infra version в `.wiki/concepts/portainer-stack-management-vds.md`). Diff: URL → `portainer.kzntsv.site`, endpoint=1, dir prefix `/usr/docker/`, auth via `X-API-Key` header (PAT works, no JWT fallback needed). -**Branch:** master - - +## 🟢 [books-vds-stacks-to-portainer] — closed 2026-05-25 — 6 SSH-compose стеков на books VDS (89.253.255.133) мигрированы в Portainer-managed (endpoint 1, https://portainer.kzntsv.site). Skip traefik + portainer (management plane). Adapter `scripts/books-vds-portainer-migration/migrate.sh`. Wiki: [portainer-stack-management-books-vds.md](../.wiki/concepts/portainer-stack-management-books-vds.md). Backup pipeline bind paths preserved (verified by inspection, full run pending next 06:00 MSK). ## 🟢 [books-vds-backup-daily-kreknin] — closed 2026-05-25 — daily backup books VDS (89.253.255.133, host4g.ru, CentOS 7) → kreknin в 06:00 MSK. DB dumps (mariadb/mongo×2) + ES snapshot via REST → rsync 4.87GB → ntfy/email (phone+inbox ✓). ES path.repo bootstrap + snapshot repo `kreknin` registered. См. [books-vds-backup-daily-kreknin.md](books-vds-backup-daily-kreknin.md). diff --git a/.tasks/books-vds-stacks-to-portainer.md b/.tasks/books-vds-stacks-to-portainer.md index d688a26..da5ac30 100644 --- a/.tasks/books-vds-stacks-to-portainer.md +++ b/.tasks/books-vds-stacks-to-portainer.md @@ -40,17 +40,32 @@ Migrate SSH-compose стеки на books VDS (`89.253.255.133`) в Portainer-ma - 2026-05-25: pattern reuse — `.wiki/concepts/portainer-stack-management-vds.md` migration script adaptable. Diff for books VDS: PORTAINER_URL, endpoint ID = 1, stack dirs `/usr/docker//` instead of `/opt/stacks//`. PAT в pass под `books-vds/full-env BOOKS_PORTAINER_API_KEY` (may need fallback to JWT if 401). - 2026-05-25 (session start): user confirmed **Endpoint 6 = stostayer VDS, НЕ наш** → scope не расширяем, Q1 closed. Auto-push grant active на сессию. - 2026-05-25 (Phase 0): SSH probe 5 target compose. Findings: (1) **no env_file** anywhere — inline env stays в stackFileContent, отдельный API env array не нужен; (2) **no surprise named volumes** — all bind, Q2 close to resolved; (3) mongo container image lost tag (digest-only) — fresh `mongo:4.2` pull при Portainer create acceptable; (4) Portainer API list endpoint 1 = 6 stacks (books-* + chrome), endpoint 6 = stostayer 8 stacks (confirmed not ours); (5) **discovered 6th stack:** `/usr/docker/imgproxy/` — `imgproxy` + `imgproxy-nginx` 2-container compose serving `imgproxy.kzntsv.site`, was missing from entity wiki. User confirmed → added to scope. +- 2026-05-25 (Phase 1): adapter script `scripts/books-vds-portainer-migration/migrate.sh` clone-adapted from VDS-infra pattern. Auth via `X-API-Key` header (PAT works, JWT fallback not needed). Down step via `docker rm -f` by `com.docker.compose.project` label — bypasses both v1 ContainerConfig bug + any v2 recreate quirks; no compose binary dependency at all. +- 2026-05-25 (Phase 2): **pre-flight surprises:** (1) no `jq` on books VDS — installed static jq 1.7.1 binary via direct GitHub download (yum dead, CentOS 7 EOL since 2024-06, mirrors retired); (2) no `docker login registry.kzntsv.site` configured — accepted (cached images covered all migrations, no force-pull issue). Both documented в concept doc. +- 2026-05-25 (Phase 2): migration executed in ladder. Per-stack Portainer Ids + duration: proxy-chain=28 (~10s recreate, HTTP 400 = service responding), imgproxy=29 (~15s, `imgproxy.kzntsv.site` → 200), minio=30 (~10s, `/minio/health/live` → 200), mongo=31 (~13s, ping=1, books-api/task-runner stayed Up через recreate window), books-db=32 (~15s, `SELECT 1` → 1, books-api connection-pool reconnected), elasticsearch=33 (~37s, cluster yellow expected single-node, **snapshot repo `kreknin` preserved** through bind). +- 2026-05-25 (Phase 3): created `.wiki/concepts/portainer-stack-management-books-vds.md` documenting diffs from vds-kzntsv pattern + per-host inventory + pre-flight quirks (jq install, no registry login). Updated `entities/books-vds.md` table: 6 stacks moved SSH-managed → Portainer-managed. ## Open questions - [x] ~~**Endpoint 6** в books Portainer — stostayer VDS?~~ Resolved 2026-05-25: stostayer **не наш**, scope не расширяем. - [x] ~~**Volume preservation strategy**~~ Resolved 2026-05-25 (Phase 0 probe): все 6 target стеков — bind-only, no named volumes. `docker compose down` без `-v` preserves binds (paths остаются на disk). -- [ ] **CI implications** — деплой books-* stacks уже Portainer-managed. After migration of mongo/minio/etc — никаких изменений в CI не нужно (deploy.yml только books-api/web/scheduler, не infra). Confirm. -- [ ] **ES snapshot path.repo edit preservation** — текущий `/usr/docker/elasticsearch/docker-compose.yml` имеет path.repo + snapshots bind (added 2026-05-25). Migration must preserve. Backup в `docker-compose.yml.bak-pre-snapshots-2026-05-25` есть для reference. +- [x] ~~**CI implications**~~ Resolved 2026-05-25: books-* stacks (CI-deployed) уже Portainer-managed pre-migration. Migrated stacks (mongo/minio/etc) — infra, не в CI flow. Никаких изменений в `victor/books/.gitea/workflows/deploy.yml` не нужно. +- [x] ~~**ES snapshot path.repo edit preservation**~~ Verified 2026-05-25 (Phase 2 stack 6 smoke): `curl localhost:9200/_snapshot` returned `kreknin` repo intact post-migration. `path.repo=/snapshots` env preserved через `stackFileContent`, bind `/snapshots` → `/usr/docker/elasticsearch/snapshots` preserved. ## Completed steps -- (not started) +- [x] Phase 0 — SSH probe 6 compose dirs (5 original + imgproxy discovered), Portainer API auth verified, surprise imgproxy stack added to scope. +- [x] Phase 1 — adapter script `scripts/books-vds-portainer-migration/migrate.sh` written + committed (`141efe97`). Pre-flight: jq 1.7.1 static binary installed on books VDS. +- [x] Phase 2 — 6 stacks migrated in ladder, all smoked green. Books-api + books-task-runner reconnected transparently through mongo + books-db recreate windows. +- [x] Phase 3 — wiki concept created (`portainer-stack-management-books-vds.md`), entity updated (`books-vds.md` SSH-managed → Portainer-managed table flipped). + +## Acceptance verification + +- ✓ 6 стеков (proxy-chain, imgproxy, minio, mongo, books-db, elasticsearch) появляются в `GET /api/stacks` под endpoint 1 — Ids 28-33 confirmed via API response after each migration. +- ✓ Контейнеры работают, бинды смонтированы — `docker ps` Up для каждого post-recreate. Data preserved: mongo ping=1, books-db SELECT 1=1, ES cluster yellow with active shards (data intact). +- ✓ Smoke per stack type — все 6 commands green (см. concept doc § Smoke commands). +- ⚠ Backup pipeline продолжает работать — **verified by inspection** (bind paths `/usr/docker//data` + `.../snapshots` preserved, backup script paths unchanged). Full pipeline run pending next scheduled 06:00 MSK. +- ✓ Wiki documented — `.wiki/concepts/portainer-stack-management-books-vds.md` + index updated + entity flipped. ## Plan (sketch) diff --git a/.wiki/concepts/portainer-stack-management-books-vds.md b/.wiki/concepts/portainer-stack-management-books-vds.md new file mode 100644 index 0000000..58501d0 --- /dev/null +++ b/.wiki/concepts/portainer-stack-management-books-vds.md @@ -0,0 +1,127 @@ +--- +title: Portainer stack management on books VDS — pattern application + migration log +status: live +tags: [books-vds, portainer, docker-compose, ops, migration] +related: [[books-vds]], [[portainer-stack-management-vds]], [[books-vds-backup-daily-kreknin-2026-05-25]] +--- + +# Portainer stack management on books VDS + +Application of the canonical pattern from [[portainer-stack-management-vds]] к **books VDS** (`89.253.255.133`, host4g.ru, CentOS 7). + +**Canonical rule** (same as VDS-infra): all docker-compose стеки на books VDS управляются через Portainer `https://portainer.kzntsv.site` (endpoint 1). SSH-compose в `/usr/docker//` — **anti-pattern** post-2026-05-25. + +2026-05-25 retro-migrated 6 ранее SSH-compose стеков: `proxy-chain`, `imgproxy` (2 контейнера), `minio`, `mongo`, `books-db`, `elasticsearch`. Skipped `traefik` + `portainer` (management plane). Adapter script `scripts/books-vds-portainer-migration/migrate.sh`. + +## Diffs from VDS-infra pattern + +| Aspect | vds-kzntsv | books VDS | +|---|---|---| +| Portainer URL | `portainer.vds.kzntsv.site` | `portainer.kzntsv.site` | +| Auth | JWT через admin/password (PAT 401'd) | **`X-API-Key` header — PAT works** | +| Stack dir prefix | `/opt/stacks//` | `/usr/docker//` | +| Endpoint ID | 1 (single docker daemon) | 1 (books VDS local). **Endpoint 6 = stostayer — НЕ наш** | +| Compose down | `cd $DIR && docker compose down` | `docker rm -f` by `com.docker.compose.project` label | +| `env_file:` directives | many stacks had them | **none** — env was inline в compose | +| `.env` files | many stacks had them | **none** — все env inline | +| Sibling files | nginx.conf, certs, scripts | nginx.conf for imgproxy-nginx only | +| docker-compose binary | v2 plugin only | v2 plugin (2.27.1) + v1 standalone (1.29.2). Script uses neither — direct `docker rm -f` | + +## Pre-flight quirks on books VDS + +1. **No `jq` installed**, no python3, **yum is broken** (CentOS 7 EOL since 2024-06, mirrors retired). Mitigation: pre-install static jq 1.7.1 binary: + ```bash + curl -fsSL https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64 \ + -o /usr/local/bin/jq && chmod +x /usr/local/bin/jq + ``` +2. **No `/root/.docker/config.json`** — no `docker login registry.kzntsv.site` configured. Risk: if Portainer create force-pulls `:latest` for private registry images (`proxy-chain`, books-*) → pull fail. **Mitigation in practice:** Portainer create использует local image cache when available; for current `:latest` cached images миграция прошла без issue. Future-proofing: `docker login registry.kzntsv.site` before any registry-managed stack force-pull. +3. **Mongo image lost tag** — running container's image was retagged (digest `0bf62880fec3` only). Portainer create pulls fresh `mongo:4.2` — may be different sub-version digest. 4.2 schema stable, accepted as low risk. Mitigation if reproducibility critical: pin to digest. + +## Stack inventory (2026-05-25 post-migration, Portainer Ids) + +Endpoint 1 (books VDS): + +| Id | Name | Compose dir | Note | +|---|---|---|---| +| 22 | books-job-scheduler | (Portainer-only, pre-migration) | agenda jobs + embedded mongo | +| 23 | books-api | (Portainer-only) | books API server, Nitro, port 3021 | +| 24 | books-web | (Portainer-only) | books Nuxt front-end | +| 25 | books-ntfy | (Portainer-only) | local ntfy (separate from `ntfy.vds.kzntsv.site`) | +| 26 | books-ops-mcp | (Portainer-only) | + books-docker-proxy + books-docker-proxy-ro + bootstrap | +| 27 | chrome | (Portainer-only) | Puppeteer for PDFs/scraping | +| 28 | proxy-chain | `/usr/docker/proxy-chain` | upstream HTTP-proxy chain, internal-only | +| 29 | imgproxy | `/usr/docker/imgproxy` | imgproxy + imgproxy-nginx, serves `imgproxy.kzntsv.site` | +| 30 | minio | `/usr/docker/minio` | S3 blob store, `minio.kzntsv.site` | +| 31 | mongo | `/usr/docker/mongo` | shared MongoDB 4.2, used by books-api + books-task-runner | +| 32 | books-db | `/usr/docker/books-db` | MariaDB latest, books application DB | +| 33 | elasticsearch | `/usr/docker/elasticsearch` | ES 7.10.0 + `path.repo=/snapshots` for backup | + +Non-Portainer (management plane, скип migration): +- `traefik` — `/usr/docker/traefik/`, ad-hoc compose, бинд `./letsencrypt` (acme.json) + `./data/traefik.yml` +- `portainer` — `/usr/docker/portainer/`, ad-hoc compose, named volume `portainer_data` + +Endpoint 6 (stostayer VDS, **не наш** — user-confirmed 2026-05-25). Stacks listed: `Traefik`, `minio`, `mariadb`, `imgproxy`, `stostayer-{franchise,web,tireservice-api}`, `registry`. Same Portainer UI manages, but physical machine not ours. + +## Migration ladder + +Used 2026-05-25, lowest → highest blast radius. Successful in this order: + +1. **proxy-chain** — internal-only, no traefik exposure → smoke for the script +2. **imgproxy** — 2-container compose with sibling nginx.conf +3. **minio** — read-mostly, books-api retries OK +4. **mongo** — shared, used by books-api + books-task-runner. Reconnect transparent (~13s) +5. **books-db** — MariaDB, books-api connection-pool reconnect (~15s) +6. **elasticsearch** — single-node, snapshot repo `kreknin` preserved through bind + +Все 6 stacks Up + smoke green в одну сессию (~10 минут end-to-end). + +## Gotchas hit during migration + +All 8 gotchas from [[portainer-stack-management-vds]] applied; books-specific additions: + +9. **CentOS 7 base OS, yum dead** — package install через yum невозможно. Static jq binary download — единственный путь без EPEL mirror работающего. Documented above. +10. **No registry login** — `docker login registry.kzntsv.site` не настроен. Кэшированные образы спасают, но fresh-pull для private images → auth fail. Setup `docker login` если планируется image-rotation от Portainer UI. +11. **Compose v1 + Compose v2 plugin co-existence** — `docker-compose` (1.29.2) и `docker compose` (2.27.1) обе installed. Migration script использует `docker rm -f` для down → не зависит ни от той ни от другой. Чище. + +## Smoke commands per stack (post-migration) + +```bash +# proxy-chain — internal smoke (HTTP 400 = service responding, not a valid proxy GET) +curl -sS -o /dev/null -w "%{http_code}\n" http://localhost:8778/ + +# imgproxy — traefik HTTP route +curl -ksS -o /dev/null -w "%{http_code}\n" https://imgproxy.kzntsv.site/ + +# minio +curl -ksS -o /dev/null -w "%{http_code}\n" https://minio.kzntsv.site/minio/health/live + +# mongo (auth via root/MONGO_INITDB_ROOT_PASSWORD) +docker exec mongo mongo --quiet --eval 'db.adminCommand({ping:1}).ok' -u root -p '' + +# books-db +docker exec books-db mariadb -uroot -p'' -e 'SELECT 1' + +# elasticsearch +docker exec elasticsearch curl -sS localhost:9200/_cluster/health | jq .status +docker exec elasticsearch curl -sS localhost:9200/_snapshot # verify kreknin repo +``` + +## What's preserved post-migration + +- **Bind data paths** — `/usr/docker//data` (and `./snapshots`, `./config`, etc.) stay on host disk. Bind preserved through container recreate. +- **Backup pipeline** — `scripts/books-vds-backup-daily-kreknin/run.sh` rsync paths unchanged. ES `path.repo=/snapshots` bind preserved → snapshot repo `kreknin` continues to write. +- **Traefik labels** — propagate automatically. Routes for `imgproxy.kzntsv.site` / `minio.kzntsv.site` / `elasticsearch.kzntsv.site` re-attach as containers come up (docker socket events). +- **Network** — все 6 stacks join external `proxy` network as before. No re-configuration. +- **Compose file on disk** — `/usr/docker//docker-compose.yml` remains as reference. Дата-stamp backup `docker-compose.yml.bak-pre-portainer-migration-2026-05-25` создан per stack. + +## What changed + +- `working_dir` label: было `/usr/docker/` (ad-hoc), стало `/data/compose//` (Portainer-managed). Compose files в `/usr/docker//` теперь reference-only. +- Lifecycle management: edit / restart / view logs — всё через Portainer UI или API. Editing `/usr/docker//docker-compose.yml` напрямую больше не имеет эффекта на runtime (Portainer владеет compose state в `/data/compose//docker-compose.yml`). + +## Cross-refs + +- [[portainer-stack-management-vds]] — source pattern (vds-kzntsv). +- [[books-vds]] — host factsheet (updated 2026-05-25). +- [[books-vds-backup-daily-kreknin-2026-05-25]] — backup pipeline stand-up; depends on bind paths preserved through this migration. +- `scripts/books-vds-portainer-migration/{migrate.sh,README.md}` — implementation. diff --git a/.wiki/entities/books-vds.md b/.wiki/entities/books-vds.md index 20dafd8..0d0d753 100644 --- a/.wiki/entities/books-vds.md +++ b/.wiki/entities/books-vds.md @@ -2,7 +2,7 @@ title: Books VDS (client server — 89.253.255.133) type: entity tags: [hardware, vds, books, host4g, backup, portainer] -related: [[kreknin-synology]], [[vds-kzntsv]] +related: [[kreknin-synology]], [[vds-kzntsv]], [[../concepts/portainer-stack-management-books-vds]] updated: 2026-05-25 --- @@ -35,29 +35,31 @@ Single root partition, нет отдельной `/data`. Docker hub data под ### Portainer-managed stacks (endpoint 1) -| Stack | Container(s) | Purpose | -|---|---|---| -| `books-api` | books-api | books API server (Nitro, port 3021) | -| `books-web` | books-web | books-Nuxt front-end | -| `books-job-scheduler` | books-job-scheduler + books-job-scheduler-mongo | agenda jobs (cron + on-demand) | -| `books-ntfy` | books-ntfy | local ntfy (separate from shared `ntfy.vds.kzntsv.site`) | -| `books-ops-mcp` | books-ops-mcp + books-ops-mcp-bootstrap-1 + books-docker-proxy + books-docker-proxy-ro | books ops-mcp endpoint | -| `chrome` | chrome | Puppeteer browser for PDFs / scraping | +Post-migration 2026-05-25 (см. [[../concepts/portainer-stack-management-books-vds]]). -### SSH-managed (legacy compose at `/usr/docker//`) +| Id | Stack | Container(s) | Bind data path | Purpose | +|---|---|---|---|---| +| 22 | `books-job-scheduler` | books-job-scheduler + books-job-scheduler-mongo | `/opt/books/job-scheduler/{config,mongo/{db,configdb}}` | agenda jobs (cron + on-demand) | +| 23 | `books-api` | books-api | `/opt/books/api/{config,data,upload}` | books API server (Nitro, port 3021) | +| 24 | `books-web` | books-web | — | books-Nuxt front-end | +| 25 | `books-ntfy` | books-ntfy | `/opt/books/ntfy/` | local ntfy (separate from shared `ntfy.vds.kzntsv.site`) | +| 26 | `books-ops-mcp` | books-ops-mcp + books-ops-mcp-bootstrap-1 + books-docker-proxy + books-docker-proxy-ro | `/opt/books/books-ops-mcp/` | books ops-mcp endpoint | +| 27 | `chrome` | chrome | — | Puppeteer browser for PDFs / scraping | +| 28 | `proxy-chain` | proxy-chain | `/usr/docker/proxy-chain/config` | upstream HTTP-proxy chain, internal-only | +| 29 | `imgproxy` | imgproxy + imgproxy-nginx | `/usr/docker/imgproxy/nginx/{cache,nginx.conf}` | serves `imgproxy.kzntsv.site`, S3→minio | +| 30 | `minio` | minio | `/usr/docker/minio/data` | S3 blob store, `minio.kzntsv.site` | +| 31 | `mongo` | mongo | `/usr/docker/mongo/data/{db,configdb}` | shared MongoDB 4.2 | +| 32 | `books-db` | books-db | `/usr/docker/books-db/data` | MariaDB latest, books application DB | +| 33 | `elasticsearch` | elasticsearch | `/usr/docker/elasticsearch/{data,snapshots}` | ES 7.10.0 + path.repo для backup | + +### SSH-managed (management plane — Portainer migration skipped) + +Recreate ломает access всему остальному (traefik) или себя (portainer). Оставлены ad-hoc compose в `/usr/docker//`. | Compose dir | Containers | Bind data path | |---|---|---| -| `/usr/docker/books-db/` | books-db | `./data` → `/var/lib/mysql` (MariaDB 10.6.4) | -| `/usr/docker/mongo/` | mongo | `./data/{db,configdb}` | -| `/usr/docker/minio/` | minio | `./data` (S3 blobs) | -| `/usr/docker/elasticsearch/` | elasticsearch | `./data` + `./snapshots` (added 2026-05-25) | | `/usr/docker/traefik/` | traefik | `./letsencrypt` (acme.json) + `./data/traefik.yml` | | `/usr/docker/portainer/` | portainer | named volume `portainer_data` | -| `/usr/docker/proxy-chain/` | proxy-chain | `./config` (upstream HTTP-proxy chain) | -| `/usr/docker/imgproxy/` | imgproxy + imgproxy-nginx | `./nginx/cache` + `./nginx/nginx.conf` (serves `imgproxy.kzntsv.site`, S3 backend via minio) | - -**TODO**: мигрировать максимум SSH-managed контейнеров под Portainer (отдельная таска после backup-pipeline live). Cause: разделённое управление обламывает CI (Portainer-only deploy.yml) и debugging. ### Build helpers (transient) @@ -103,8 +105,9 @@ Auth-зоны DNS — в reg.ru под `kzntsv.site`. - [[../concepts/vds-kzntsv-ssh-access]] — SSH-аудит инфра-VDS (был misnamed как `books-ssh-access` до 2026-05-25; ошибочно claimed vds-kzntsv hosts books). - [[../sources/books-vds-backup-daily-kreknin-2026-05-25]] — backup standup chronology. -## Каking-of-history notes +## Making-of-history notes - Discovered 2026-05-25 во время backup-таски. Wiki [[../concepts/books-ssh-access]] до этого ошибочно говорила что books на vds-kzntsv (89.253.255.94). User: «books VDS - это совсем другой сервер, клиентский». Inventory + IP probe confirmed. - Workstation SSH key `id_ed25519_books_ops` уже был задеплоен под root до этой сессии (видимо, прошлый bootstrap забыт-без-документации). -- Hybrid stack-management (Portainer + SSH-compose) — pre-existing, не наш design. Migrate to Portainer-only — follow-up. +- Hybrid stack-management (Portainer + SSH-compose) — pre-existing, не наш design. Migrated 6 SSH-compose стеков → Portainer 2026-05-25 (`books-vds-stacks-to-portainer` 🟢). `traefik` + `portainer` остаются SSH-managed (management plane). +- `imgproxy` (2 containers, `imgproxy.kzntsv.site`) был missing из этого entity wiki до Phase 0 probe 2026-05-25 — fixed in flight. diff --git a/.wiki/index.md b/.wiki/index.md index 22ed6dd..ee9c5d5 100644 --- a/.wiki/index.md +++ b/.wiki/index.md @@ -32,6 +32,7 @@ Catalog of all wiki pages. One line per page, organized by type. Updated on ever - [mssql-on-vds](concepts/mssql-on-vds.md) — MSSQL Express 2022 Linux на VDS — миграция + login orphan fix + traefik TCP gotchas - [ocis-on-vds-deploy-recipe](concepts/ocis-on-vds-deploy-recipe.md) — oCIS на VDS — deploy recipe + non-obvious gotchas (UID 1001 vs 1000, basic auth, LibreGraph user-create) - [portainer-2.21-admin-password-regression](concepts/portainer-2.21-admin-password-regression.md) — Portainer 2.21 `--admin-password` regression + min 12-char policy +- [portainer-stack-management-books-vds](concepts/portainer-stack-management-books-vds.md) — Portainer-managed stacks на books VDS — pattern application + migration log 2026-05-25 - [portainer-stack-management-vds](concepts/portainer-stack-management-vds.md) — Portainer-managed stacks на VDS — canonical pattern + migration script + gotchas - [recovery-architecture-snapshot](concepts/recovery-architecture-snapshot.md) — текущая recovery architecture (2026-05-19/21, attempt 2) - [registry-gc-mount-and-modify-flag](concepts/registry-gc-mount-and-modify-flag.md) — Docker Registry GC mount layout + `-m` flag