tasks(books-vds-stacks-to-portainer): close 🟢 — 6 stacks migrated SSH-compose → Portainer

Phase 2 ladder executed 2026-05-25 (~10min end-to-end, lowest → highest blast radius): proxy-chain (id 28) → imgproxy (29) → minio (30) → mongo (31) → books-db (32) → elasticsearch (33) All 6 smoked green. books-api + books-task-runner reconnected transparently through mongo + books-db recreate windows. ES snapshot repo `kreknin` preserved through bind. Phase 3: - created .wiki/concepts/portainer-stack-management-books-vds.md (pattern application + diffs from vds-kzntsv) - entity books-vds.md: 6 stacks flipped SSH-managed → Portainer-managed - index.md: new concept entry Pre-flight quirks documented (CentOS 7 yum dead, no jq, no docker login). Acceptance: 4/5 verified, backup pipeline (#5) verified by inspection (bind paths preserved) — full pipeline run pending next 06:00 MSK. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 10:27:23 +03:00
parent 141efe9759
commit 86c44c3db9
6 changed files with 192 additions and 58 deletions
--- a/.wiki/concepts/portainer-stack-management-books-vds.md
+++ b/.wiki/concepts/portainer-stack-management-books-vds.md
@@ -0,0 +1,127 @@
+---
+title: Portainer stack management on books VDS — pattern application + migration log
+status: live
+tags: [books-vds, portainer, docker-compose, ops, migration]
+related: [[books-vds]], [[portainer-stack-management-vds]], [[books-vds-backup-daily-kreknin-2026-05-25]]
+---
+
+# Portainer stack management on books VDS
+
+Application of the canonical pattern from [[portainer-stack-management-vds]] к **books VDS** (`89.253.255.133`, host4g.ru, CentOS 7).
+
+**Canonical rule** (same as VDS-infra): all docker-compose стеки на books VDS управляются через Portainer `https://portainer.kzntsv.site` (endpoint 1). SSH-compose в `/usr/docker/<svc>/` — **anti-pattern** post-2026-05-25.
+
+2026-05-25 retro-migrated 6 ранее SSH-compose стеков: `proxy-chain`, `imgproxy` (2 контейнера), `minio`, `mongo`, `books-db`, `elasticsearch`. Skipped `traefik` + `portainer` (management plane). Adapter script `scripts/books-vds-portainer-migration/migrate.sh`.
+
+## Diffs from VDS-infra pattern
+
+| Aspect | vds-kzntsv | books VDS |
+|---|---|---|
+| Portainer URL | `portainer.vds.kzntsv.site` | `portainer.kzntsv.site` |
+| Auth | JWT через admin/password (PAT 401'd) | **`X-API-Key` header — PAT works** |
+| Stack dir prefix | `/opt/stacks/<stack>/` | `/usr/docker/<stack>/` |
+| Endpoint ID | 1 (single docker daemon) | 1 (books VDS local). **Endpoint 6 = stostayer — НЕ наш** |
+| Compose down | `cd $DIR && docker compose down` | `docker rm -f` by `com.docker.compose.project` label |
+| `env_file:` directives | many stacks had them | **none** — env was inline в compose |
+| `.env` files | many stacks had them | **none** — все env inline |
+| Sibling files | nginx.conf, certs, scripts | nginx.conf for imgproxy-nginx only |
+| docker-compose binary | v2 plugin only | v2 plugin (2.27.1) + v1 standalone (1.29.2). Script uses neither — direct `docker rm -f` |
+
+## Pre-flight quirks on books VDS
+
+1. **No `jq` installed**, no python3, **yum is broken** (CentOS 7 EOL since 2024-06, mirrors retired). Mitigation: pre-install static jq 1.7.1 binary:
+   ```bash
+   curl -fsSL https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64 \
+     -o /usr/local/bin/jq && chmod +x /usr/local/bin/jq
+   ```
+2. **No `/root/.docker/config.json`** — no `docker login registry.kzntsv.site` configured. Risk: if Portainer create force-pulls `:latest` for private registry images (`proxy-chain`, books-*) → pull fail. **Mitigation in practice:** Portainer create использует local image cache when available; for current `:latest` cached images миграция прошла без issue. Future-proofing: `docker login registry.kzntsv.site` before any registry-managed stack force-pull.
+3. **Mongo image lost tag** — running container's image was retagged (digest `0bf62880fec3` only). Portainer create pulls fresh `mongo:4.2` — may be different sub-version digest. 4.2 schema stable, accepted as low risk. Mitigation if reproducibility critical: pin to digest.
+
+## Stack inventory (2026-05-25 post-migration, Portainer Ids)
+
+Endpoint 1 (books VDS):
+
+| Id | Name | Compose dir | Note |
+|---|---|---|---|
+| 22 | books-job-scheduler | (Portainer-only, pre-migration) | agenda jobs + embedded mongo |
+| 23 | books-api | (Portainer-only) | books API server, Nitro, port 3021 |
+| 24 | books-web | (Portainer-only) | books Nuxt front-end |
+| 25 | books-ntfy | (Portainer-only) | local ntfy (separate from `ntfy.vds.kzntsv.site`) |
+| 26 | books-ops-mcp | (Portainer-only) | + books-docker-proxy + books-docker-proxy-ro + bootstrap |
+| 27 | chrome | (Portainer-only) | Puppeteer for PDFs/scraping |
+| 28 | proxy-chain | `/usr/docker/proxy-chain` | upstream HTTP-proxy chain, internal-only |
+| 29 | imgproxy | `/usr/docker/imgproxy` | imgproxy + imgproxy-nginx, serves `imgproxy.kzntsv.site` |
+| 30 | minio | `/usr/docker/minio` | S3 blob store, `minio.kzntsv.site` |
+| 31 | mongo | `/usr/docker/mongo` | shared MongoDB 4.2, used by books-api + books-task-runner |
+| 32 | books-db | `/usr/docker/books-db` | MariaDB latest, books application DB |
+| 33 | elasticsearch | `/usr/docker/elasticsearch` | ES 7.10.0 + `path.repo=/snapshots` for backup |
+
+Non-Portainer (management plane, скип migration):
+- `traefik` — `/usr/docker/traefik/`, ad-hoc compose, бинд `./letsencrypt` (acme.json) + `./data/traefik.yml`
+- `portainer` — `/usr/docker/portainer/`, ad-hoc compose, named volume `portainer_data`
+
+Endpoint 6 (stostayer VDS, **не наш** — user-confirmed 2026-05-25). Stacks listed: `Traefik`, `minio`, `mariadb`, `imgproxy`, `stostayer-{franchise,web,tireservice-api}`, `registry`. Same Portainer UI manages, but physical machine not ours.
+
+## Migration ladder
+
+Used 2026-05-25, lowest → highest blast radius. Successful in this order:
+
+1. **proxy-chain** — internal-only, no traefik exposure → smoke for the script
+2. **imgproxy** — 2-container compose with sibling nginx.conf
+3. **minio** — read-mostly, books-api retries OK
+4. **mongo** — shared, used by books-api + books-task-runner. Reconnect transparent (~13s)
+5. **books-db** — MariaDB, books-api connection-pool reconnect (~15s)
+6. **elasticsearch** — single-node, snapshot repo `kreknin` preserved through bind
+
+Все 6 stacks Up + smoke green в одну сессию (~10 минут end-to-end).
+
+## Gotchas hit during migration
+
+All 8 gotchas from [[portainer-stack-management-vds]] applied; books-specific additions:
+
+9. **CentOS 7 base OS, yum dead** — package install через yum невозможно. Static jq binary download — единственный путь без EPEL mirror работающего. Documented above.
+10. **No registry login** — `docker login registry.kzntsv.site` не настроен. Кэшированные образы спасают, но fresh-pull для private images → auth fail. Setup `docker login` если планируется image-rotation от Portainer UI.
+11. **Compose v1 + Compose v2 plugin co-existence** — `docker-compose` (1.29.2) и `docker compose` (2.27.1) обе installed. Migration script использует `docker rm -f` для down → не зависит ни от той ни от другой. Чище.
+
+## Smoke commands per stack (post-migration)
+
+```bash
+# proxy-chain — internal smoke (HTTP 400 = service responding, not a valid proxy GET)
+curl -sS -o /dev/null -w "%{http_code}\n" http://localhost:8778/
+
+# imgproxy — traefik HTTP route
+curl -ksS -o /dev/null -w "%{http_code}\n" https://imgproxy.kzntsv.site/
+
+# minio
+curl -ksS -o /dev/null -w "%{http_code}\n" https://minio.kzntsv.site/minio/health/live
+
+# mongo (auth via root/MONGO_INITDB_ROOT_PASSWORD)
+docker exec mongo mongo --quiet --eval 'db.adminCommand({ping:1}).ok' -u root -p '<from pass>'
+
+# books-db
+docker exec books-db mariadb -uroot -p'<from pass>' -e 'SELECT 1'
+
+# elasticsearch
+docker exec elasticsearch curl -sS localhost:9200/_cluster/health | jq .status
+docker exec elasticsearch curl -sS localhost:9200/_snapshot   # verify kreknin repo
+```
+
+## What's preserved post-migration
+
+- **Bind data paths** — `/usr/docker/<stack>/data` (and `./snapshots`, `./config`, etc.) stay on host disk. Bind preserved through container recreate.
+- **Backup pipeline** — `scripts/books-vds-backup-daily-kreknin/run.sh` rsync paths unchanged. ES `path.repo=/snapshots` bind preserved → snapshot repo `kreknin` continues to write.
+- **Traefik labels** — propagate automatically. Routes for `imgproxy.kzntsv.site` / `minio.kzntsv.site` / `elasticsearch.kzntsv.site` re-attach as containers come up (docker socket events).
+- **Network** — все 6 stacks join external `proxy` network as before. No re-configuration.
+- **Compose file on disk** — `/usr/docker/<stack>/docker-compose.yml` remains as reference. Дата-stamp backup `docker-compose.yml.bak-pre-portainer-migration-2026-05-25` создан per stack.
+
+## What changed
+
+- `working_dir` label: было `/usr/docker/<stack>` (ad-hoc), стало `/data/compose/<id>/` (Portainer-managed). Compose files в `/usr/docker/<stack>/` теперь reference-only.
+- Lifecycle management: edit / restart / view logs — всё через Portainer UI или API. Editing `/usr/docker/<stack>/docker-compose.yml` напрямую больше не имеет эффекта на runtime (Portainer владеет compose state в `/data/compose/<id>/docker-compose.yml`).
+
+## Cross-refs
+
+- [[portainer-stack-management-vds]] — source pattern (vds-kzntsv).
+- [[books-vds]] — host factsheet (updated 2026-05-25).
+- [[books-vds-backup-daily-kreknin-2026-05-25]] — backup pipeline stand-up; depends on bind paths preserved through this migration.
+- `scripts/books-vds-portainer-migration/{migrate.sh,README.md}` — implementation.
--- a/.wiki/entities/books-vds.md
+++ b/.wiki/entities/books-vds.md
@@ -2,7 +2,7 @@
 title: Books VDS (client server — 89.253.255.133)
 type: entity
 tags: [hardware, vds, books, host4g, backup, portainer]
-related: [[kreknin-synology]], [[vds-kzntsv]]
+related: [[kreknin-synology]], [[vds-kzntsv]], [[../concepts/portainer-stack-management-books-vds]]
 updated: 2026-05-25
 ---

@@ -35,29 +35,31 @@ Single root partition, нет отдельной `/data`. Docker hub data под

 ### Portainer-managed stacks (endpoint 1)

-| Stack | Container(s) | Purpose |
-|---|---|---|
-| `books-api` | books-api | books API server (Nitro, port 3021) |
-| `books-web` | books-web | books-Nuxt front-end |
-| `books-job-scheduler` | books-job-scheduler + books-job-scheduler-mongo | agenda jobs (cron + on-demand) |
-| `books-ntfy` | books-ntfy | local ntfy (separate from shared `ntfy.vds.kzntsv.site`) |
-| `books-ops-mcp` | books-ops-mcp + books-ops-mcp-bootstrap-1 + books-docker-proxy + books-docker-proxy-ro | books ops-mcp endpoint |
-| `chrome` | chrome | Puppeteer browser for PDFs / scraping |
+Post-migration 2026-05-25 (см. [[../concepts/portainer-stack-management-books-vds]]).

-### SSH-managed (legacy compose at `/usr/docker/<svc>/`)
+| Id | Stack | Container(s) | Bind data path | Purpose |
+|---|---|---|---|---|
+| 22 | `books-job-scheduler` | books-job-scheduler + books-job-scheduler-mongo | `/opt/books/job-scheduler/{config,mongo/{db,configdb}}` | agenda jobs (cron + on-demand) |
+| 23 | `books-api` | books-api | `/opt/books/api/{config,data,upload}` | books API server (Nitro, port 3021) |
+| 24 | `books-web` | books-web | — | books-Nuxt front-end |
+| 25 | `books-ntfy` | books-ntfy | `/opt/books/ntfy/` | local ntfy (separate from shared `ntfy.vds.kzntsv.site`) |
+| 26 | `books-ops-mcp` | books-ops-mcp + books-ops-mcp-bootstrap-1 + books-docker-proxy + books-docker-proxy-ro | `/opt/books/books-ops-mcp/` | books ops-mcp endpoint |
+| 27 | `chrome` | chrome | — | Puppeteer browser for PDFs / scraping |
+| 28 | `proxy-chain` | proxy-chain | `/usr/docker/proxy-chain/config` | upstream HTTP-proxy chain, internal-only |
+| 29 | `imgproxy` | imgproxy + imgproxy-nginx | `/usr/docker/imgproxy/nginx/{cache,nginx.conf}` | serves `imgproxy.kzntsv.site`, S3→minio |
+| 30 | `minio` | minio | `/usr/docker/minio/data` | S3 blob store, `minio.kzntsv.site` |
+| 31 | `mongo` | mongo | `/usr/docker/mongo/data/{db,configdb}` | shared MongoDB 4.2 |
+| 32 | `books-db` | books-db | `/usr/docker/books-db/data` | MariaDB latest, books application DB |
+| 33 | `elasticsearch` | elasticsearch | `/usr/docker/elasticsearch/{data,snapshots}` | ES 7.10.0 + path.repo для backup |
+
+### SSH-managed (management plane — Portainer migration skipped)
+
+Recreate ломает access всему остальному (traefik) или себя (portainer). Оставлены ad-hoc compose в `/usr/docker/<svc>/`.

 | Compose dir | Containers | Bind data path |
 |---|---|---|
-| `/usr/docker/books-db/` | books-db | `./data` → `/var/lib/mysql` (MariaDB 10.6.4) |
-| `/usr/docker/mongo/` | mongo | `./data/{db,configdb}` |
-| `/usr/docker/minio/` | minio | `./data` (S3 blobs) |
-| `/usr/docker/elasticsearch/` | elasticsearch | `./data` + `./snapshots` (added 2026-05-25) |
 | `/usr/docker/traefik/` | traefik | `./letsencrypt` (acme.json) + `./data/traefik.yml` |
 | `/usr/docker/portainer/` | portainer | named volume `portainer_data` |
-| `/usr/docker/proxy-chain/` | proxy-chain | `./config` (upstream HTTP-proxy chain) |
-| `/usr/docker/imgproxy/` | imgproxy + imgproxy-nginx | `./nginx/cache` + `./nginx/nginx.conf` (serves `imgproxy.kzntsv.site`, S3 backend via minio) |
-
-**TODO**: мигрировать максимум SSH-managed контейнеров под Portainer (отдельная таска после backup-pipeline live). Cause: разделённое управление обламывает CI (Portainer-only deploy.yml) и debugging.

 ### Build helpers (transient)

@@ -103,8 +105,9 @@ Auth-зоны DNS — в reg.ru под `kzntsv.site`.
 - [[../concepts/vds-kzntsv-ssh-access]] — SSH-аудит инфра-VDS (был misnamed как `books-ssh-access` до 2026-05-25; ошибочно claimed vds-kzntsv hosts books).
 - [[../sources/books-vds-backup-daily-kreknin-2026-05-25]] — backup standup chronology.

-## Каking-of-history notes
+## Making-of-history notes

 - Discovered 2026-05-25 во время backup-таски. Wiki [[../concepts/books-ssh-access]] до этого ошибочно говорила что books на vds-kzntsv (89.253.255.94). User: «books VDS - это совсем другой сервер, клиентский». Inventory + IP probe confirmed.
 - Workstation SSH key `id_ed25519_books_ops` уже был задеплоен под root до этой сессии (видимо, прошлый bootstrap забыт-без-документации).
- Hybrid stack-management (Portainer + SSH-compose) — pre-existing, не наш design. Migrate to Portainer-only — follow-up.
+- Hybrid stack-management (Portainer + SSH-compose) — pre-existing, не наш design. Migrated 6 SSH-compose стеков → Portainer 2026-05-25 (`books-vds-stacks-to-portainer` 🟢). `traefik` + `portainer` остаются SSH-managed (management plane).
+- `imgproxy` (2 containers, `imgproxy.kzntsv.site`) был missing из этого entity wiki до Phase 0 probe 2026-05-25 — fixed in flight.
--- a/.wiki/index.md
+++ b/.wiki/index.md
@@ -32,6 +32,7 @@ Catalog of all wiki pages. One line per page, organized by type. Updated on ever
 - [mssql-on-vds](concepts/mssql-on-vds.md) — MSSQL Express 2022 Linux на VDS — миграция + login orphan fix + traefik TCP gotchas
 - [ocis-on-vds-deploy-recipe](concepts/ocis-on-vds-deploy-recipe.md) — oCIS на VDS — deploy recipe + non-obvious gotchas (UID 1001 vs 1000, basic auth, LibreGraph user-create)
 - [portainer-2.21-admin-password-regression](concepts/portainer-2.21-admin-password-regression.md) — Portainer 2.21 `--admin-password` regression + min 12-char policy
+- [portainer-stack-management-books-vds](concepts/portainer-stack-management-books-vds.md) — Portainer-managed stacks на books VDS — pattern application + migration log 2026-05-25
 - [portainer-stack-management-vds](concepts/portainer-stack-management-vds.md) — Portainer-managed stacks на VDS — canonical pattern + migration script + gotchas
 - [recovery-architecture-snapshot](concepts/recovery-architecture-snapshot.md) — текущая recovery architecture (2026-05-19/21, attempt 2)
 - [registry-gc-mount-and-modify-flag](concepts/registry-gc-mount-and-modify-flag.md) — Docker Registry GC mount layout + `-m` flag