admin/.tasks/books-vds-stacks-to-portainer.md

books-vds-stacks-to-portainer

Migrate SSH-compose стеки на books VDS (89.253.255.133) в Portainer-managed через API. Pattern parallel [[.wiki/concepts/portainer-stack-management-vds]] (VDS-infra retro-migration 2026-05-22), но другой endpoint/Portainer URL/stack dirs.

Goal

Унифицировать ops surface на books VDS — все docker-compose стеки управляются через https://portainer.kzntsv.site (endpoint 1). Конец hybrid pain'у (Portainer для books-{api,web,scheduler,...} + SSH-compose для elasticsearch/mongo/minio/books-db/traefik/proxy-chain). CI deploy workflow (.gitea/workflows/deploy.yml в victor/books) только через Portainer API; SSH-compose vert не подключается; debug требует знать «is it Portainer or SSH compose» — больно.

Scope

Stack	Current dir	Adapt notes
books-db (MariaDB latest)	/usr/docker/books-db/	bind ./data → absolutize. Env inline (no env_file).
mongo (shared, 4.2)	/usr/docker/mongo/	bind ./data/{db,configdb} → absolutize. Env inline.
minio (2020-07-13)	/usr/docker/minio/	bind ./data → absolutize. Env inline.
elasticsearch (7.10.0)	/usr/docker/elasticsearch/	bind ./data + ./snapshots (added 2026-05-25 для backup) → absolutize. path.repo env preserved.
proxy-chain	/usr/docker/proxy-chain/	bind ./config → absolutize. Internal-only (no traefik).
imgproxy (2 containers: imgproxy + imgproxy-nginx)	/usr/docker/imgproxy/	added 2026-05-25 (Phase 0 discovery). Binds ./nginx/cache + ./nginx/nginx.conf → absolutize. Secrets inline.

Skip (management plane):

• traefik — /usr/docker/traefik/ — recreate ломает access всему остальному.
• portainer — /usr/docker/portainer/ — self-managed; can't recreate себя.

Acceptance

• 6 стеков (books-db, mongo, minio, elasticsearch, proxy-chain, imgproxy) появляются в GET /api/stacks под endpoint 1 (наряду с книгами books-{api,web,scheduler,ntfy,ops-mcp} + chrome).
• Контейнеры работают, бинды смонтированы, data preserved (named volumes + binds через migration).
• Smoke: каждый стек проходит health check соответствующий своему типу:
  • books-db: docker exec books-db mariadb -uroot -p... -e 'SELECT 1' → 1
  • mongo: docker exec mongo mongosh --eval 'db.adminCommand({ping:1})' → ok:1
  • minio: HTTP https://elasticsearch.kzntsv.site (или внутренний minio endpoint) — 200 OK
  • elasticsearch: docker exec elasticsearch curl localhost:9200/_cluster/health → green|yellow
  • proxy-chain: smoke через тестового агента (TBD)
• Backup pipeline (books-vds-backup-daily-kreknin) продолжает работать post-migration (compose dirs не используются runtime после migration, but /usr/docker/<svc>/data binds preserved — rsync paths остаются валидны).
• Document в .wiki/concepts/portainer-stack-management-books-vds.md (или extend existing portainer-stack-management-vds.md с books VDS sections).

Decisions log

• 2026-05-25: создана по запросу user'а после успешного books-vds-backup-daily-kreknin. Promised after backup pipeline live.
• 2026-05-25: pattern reuse — .wiki/concepts/portainer-stack-management-vds.md migration script adaptable. Diff for books VDS: PORTAINER_URL, endpoint ID = 1, stack dirs /usr/docker/<svc>/ instead of /opt/stacks/<svc>/. PAT в pass под books-vds/full-env BOOKS_PORTAINER_API_KEY (may need fallback to JWT if 401).
• 2026-05-25 (session start): user confirmed Endpoint 6 = stostayer VDS, НЕ наш → scope не расширяем, Q1 closed. Auto-push grant active на сессию.
• 2026-05-25 (Phase 0): SSH probe 5 target compose. Findings: (1) no env_file anywhere — inline env stays в stackFileContent, отдельный API env array не нужен; (2) no surprise named volumes — all bind, Q2 close to resolved; (3) mongo container image lost tag (digest-only) — fresh mongo:4.2 pull при Portainer create acceptable; (4) Portainer API list endpoint 1 = 6 stacks (books-* + chrome), endpoint 6 = stostayer 8 stacks (confirmed not ours); (5) discovered 6th stack: /usr/docker/imgproxy/ — imgproxy + imgproxy-nginx 2-container compose serving imgproxy.kzntsv.site, was missing from entity wiki. User confirmed → added to scope.
• 2026-05-25 (Phase 1): adapter script scripts/books-vds-portainer-migration/migrate.sh clone-adapted from VDS-infra pattern. Auth via X-API-Key header (PAT works, JWT fallback not needed). Down step via docker rm -f by com.docker.compose.project label — bypasses both v1 ContainerConfig bug + any v2 recreate quirks; no compose binary dependency at all.
• 2026-05-25 (Phase 2): pre-flight surprises: (1) no jq on books VDS — installed static jq 1.7.1 binary via direct GitHub download (yum dead, CentOS 7 EOL since 2024-06, mirrors retired); (2) no docker login registry.kzntsv.site configured — accepted (cached images covered all migrations, no force-pull issue). Both documented в concept doc.
• 2026-05-25 (Phase 2): migration executed in ladder. Per-stack Portainer Ids + duration: proxy-chain=28 (~10s recreate, HTTP 400 = service responding), imgproxy=29 (~15s, imgproxy.kzntsv.site → 200), minio=30 (~10s, /minio/health/live → 200), mongo=31 (~13s, ping=1, books-api/task-runner stayed Up через recreate window), books-db=32 (~15s, SELECT 1 → 1, books-api connection-pool reconnected), elasticsearch=33 (~37s, cluster yellow expected single-node, snapshot repo kreknin preserved through bind).
• 2026-05-25 (Phase 3): created .wiki/concepts/portainer-stack-management-books-vds.md documenting diffs from vds-kzntsv pattern + per-host inventory + pre-flight quirks (jq install, no registry login). Updated entities/books-vds.md table: 6 stacks moved SSH-managed → Portainer-managed.

Open questions

• Endpoint 6 в books Portainer — stostayer VDS? Resolved 2026-05-25: stostayer не наш, scope не расширяем.
• Volume preservation strategy Resolved 2026-05-25 (Phase 0 probe): все 6 target стеков — bind-only, no named volumes. docker compose down без -v preserves binds (paths остаются на disk).
• CI implications Resolved 2026-05-25: books-* stacks (CI-deployed) уже Portainer-managed pre-migration. Migrated stacks (mongo/minio/etc) — infra, не в CI flow. Никаких изменений в victor/books/.gitea/workflows/deploy.yml не нужно.
• ES snapshot path.repo edit preservation Verified 2026-05-25 (Phase 2 stack 6 smoke): curl localhost:9200/_snapshot returned kreknin repo intact post-migration. path.repo=/snapshots env preserved через stackFileContent, bind /snapshots → /usr/docker/elasticsearch/snapshots preserved.

Completed steps

• Phase 0 — SSH probe 6 compose dirs (5 original + imgproxy discovered), Portainer API auth verified, surprise imgproxy stack added to scope.
• Phase 1 — adapter script scripts/books-vds-portainer-migration/migrate.sh written + committed (141efe97). Pre-flight: jq 1.7.1 static binary installed on books VDS.
• Phase 2 — 6 stacks migrated in ladder, all smoked green. Books-api + books-task-runner reconnected transparently through mongo + books-db recreate windows.
• Phase 3 — wiki concept created (portainer-stack-management-books-vds.md), entity updated (books-vds.md SSH-managed → Portainer-managed table flipped).

Acceptance verification

• ✓ 6 стеков (proxy-chain, imgproxy, minio, mongo, books-db, elasticsearch) появляются в GET /api/stacks под endpoint 1 — Ids 28-33 confirmed via API response after each migration.
• ✓ Контейнеры работают, бинды смонтированы — docker ps Up для каждого post-recreate. Data preserved: mongo ping=1, books-db SELECT 1=1, ES cluster yellow with active shards (data intact).
• ✓ Smoke per stack type — все 6 commands green (см. concept doc § Smoke commands).
• ⚠ Backup pipeline продолжает работать — verified by inspection (bind paths /usr/docker/<stack>/data + .../snapshots preserved, backup script paths unchanged). Full pipeline run pending next scheduled 06:00 MSK.
• ✓ Wiki documented — .wiki/concepts/portainer-stack-management-books-vds.md + index updated + entity flipped.

Plan (sketch)

Phase 0 — pre-flight:

1. Read each stack's docker-compose.yml + .env (if exists).
2. Document existing env vars + binds per stack.
3. Backup current compose files (docker-compose.yml.bak-pre-portainer-migration-2026-MM-DD).

Phase 1 — Portainer API setup:

1. Verify PAT works (curl ... /api/stacks). If 401 → fallback to JWT via admin password (from pass show books-vds/full-env BOOKS_VDS_ROOT_PASS adapted to Portainer admin — may differ).
2. Build adapter script scripts/books-vds-portainer-migrate.sh (clone-and-adapt of VDS-infra version, see .wiki/concepts/portainer-stack-management-vds § Migration script).

Phase 2 — per-stack migrate (one at a time, verify between):

1. proxy-chain (lowest blast radius — no traefik exposure)
2. imgproxy (2-container compose, serves imgproxy.kzntsv.site — drive-by added 2026-05-25)
3. minio (read-mostly, books-api retries OK)
4. mongo (shared) — books-api + books-task-runner retry-tolerant? VERIFY first
5. books-db (MariaDB) — books-api uses connection-pool, brief downtime survivable
6. elasticsearch — currently unused (per user), zero impact

Phase 3 — wiki + close:

1. Document в .wiki/concepts/portainer-stack-management-books-vds.md (or extend existing).
2. Update .wiki/entities/books-vds.md — flip каждый стек в SSH-managed → Portainer-managed table.
3. Update STATUS.md → 🟢.

Notes

• books-job-scheduler-mongo уже Portainer-managed (часть books-job-scheduler stack). Не в scope этой таски.
• books-docker-proxy + books-docker-proxy-ro — часть books-ops-mcp stack (Portainer). Not in scope.
• Buildx builders (buildx_buildkit_builder-*) — transient containers без compose, игнорируем.
• Backup pipeline depend — после migration paths /usr/docker/<svc>/data остаются как bind sources, поэтому books-vds-backup-daily-kreknin/run.sh не требует изменений. Compose files (/usr/docker/<svc>/docker-compose.yml) станут reference only после migration.

Cross-refs

• .wiki/concepts/portainer-stack-management-vds — VDS-infra retro-migration pattern (source of script).
• .wiki/entities/books-vds — host factsheet (will update post-migration).
• books-vds-backup-daily-kreknin — backup taskdone 2026-05-25; depend on stable bind paths.
• victor/books deploy/README.md — books-* CI deploy через Portainer API (already migrated).