admin/.tasks/books-vds-stacks-to-portainer.md

# books-vds-stacks-to-portainer

Migrate SSH-compose стеки на books VDS (`89.253.255.133`) в Portainer-managed через API. Pattern parallel `[[.wiki/concepts/portainer-stack-management-vds]]` (VDS-infra retro-migration 2026-05-22), но другой endpoint/Portainer URL/stack dirs.

## Goal

Унифицировать ops surface на books VDS — все docker-compose стеки управляются через `https://portainer.kzntsv.site` (endpoint 1). Конец hybrid pain'у (Portainer для `books-{api,web,scheduler,...}` + SSH-compose для `elasticsearch/mongo/minio/books-db/traefik/proxy-chain`). CI deploy workflow (`.gitea/workflows/deploy.yml` в victor/books) только через Portainer API; SSH-compose vert не подключается; debug требует знать «is it Portainer or SSH compose» — больно.

## Scope

| Stack | Current dir | Adapt notes |
|---|---|---|
| `books-db` (MariaDB latest) | `/usr/docker/books-db/` | bind `./data` → absolutize. Env inline (no env_file). |
| `mongo` (shared, 4.2) | `/usr/docker/mongo/` | bind `./data/{db,configdb}` → absolutize. Env inline. |
| `minio` (2020-07-13) | `/usr/docker/minio/` | bind `./data` → absolutize. Env inline. |
| `elasticsearch` (7.10.0) | `/usr/docker/elasticsearch/` | bind `./data` + `./snapshots` (added 2026-05-25 для backup) → absolutize. path.repo env preserved. |
| `proxy-chain` | `/usr/docker/proxy-chain/` | bind `./config` → absolutize. Internal-only (no traefik). |
| `imgproxy` (2 containers: imgproxy + imgproxy-nginx) | `/usr/docker/imgproxy/` | added 2026-05-25 (Phase 0 discovery). Binds `./nginx/cache` + `./nginx/nginx.conf` → absolutize. Secrets inline. |

**Skip (management plane):**
- `traefik` — `/usr/docker/traefik/` — recreate ломает access всему остальному.
- `portainer` — `/usr/docker/portainer/` — self-managed; can't recreate себя.

## Acceptance

- 6 стеков (books-db, mongo, minio, elasticsearch, proxy-chain, imgproxy) появляются в `GET /api/stacks` под endpoint 1 (наряду с книгами `books-{api,web,scheduler,ntfy,ops-mcp}` + `chrome`).
- Контейнеры работают, бинды смонтированы, data preserved (named volumes + binds через migration).
- Smoke: каждый стек проходит health check соответствующий своему типу:
  - books-db: `docker exec books-db mariadb -uroot -p... -e 'SELECT 1'` → `1`
  - mongo: `docker exec mongo mongosh --eval 'db.adminCommand({ping:1})'` → `ok:1`
  - minio: HTTP `https://elasticsearch.kzntsv.site` (или внутренний minio endpoint) — `200 OK`
  - elasticsearch: `docker exec elasticsearch curl localhost:9200/_cluster/health` → `green|yellow`
  - proxy-chain: smoke через тестового агента (TBD)
- Backup pipeline (`books-vds-backup-daily-kreknin`) продолжает работать post-migration (compose dirs не используются runtime после migration, but `/usr/docker/<svc>/data` binds preserved — rsync paths остаются валидны).
- Document в `.wiki/concepts/portainer-stack-management-books-vds.md` (или extend existing `portainer-stack-management-vds.md` с books VDS sections).

## Decisions log

- 2026-05-25: создана по запросу user'а после успешного `books-vds-backup-daily-kreknin`. Promised after backup pipeline live.
- 2026-05-25: pattern reuse — `.wiki/concepts/portainer-stack-management-vds.md` migration script adaptable. Diff for books VDS: PORTAINER_URL, endpoint ID = 1, stack dirs `/usr/docker/<svc>/` instead of `/opt/stacks/<svc>/`. PAT в pass под `books-vds/full-env BOOKS_PORTAINER_API_KEY` (may need fallback to JWT if 401).
- 2026-05-25 (session start): user confirmed **Endpoint 6 = stostayer VDS, НЕ наш** → scope не расширяем, Q1 closed. Auto-push grant active на сессию.
- 2026-05-25 (Phase 0): SSH probe 5 target compose. Findings: (1) **no env_file** anywhere — inline env stays в stackFileContent, отдельный API env array не нужен; (2) **no surprise named volumes** — all bind, Q2 close to resolved; (3) mongo container image lost tag (digest-only) — fresh `mongo:4.2` pull при Portainer create acceptable; (4) Portainer API list endpoint 1 = 6 stacks (books-* + chrome), endpoint 6 = stostayer 8 stacks (confirmed not ours); (5) **discovered 6th stack:** `/usr/docker/imgproxy/` — `imgproxy` + `imgproxy-nginx` 2-container compose serving `imgproxy.kzntsv.site`, was missing from entity wiki. User confirmed → added to scope.
- 2026-05-25 (Phase 1): adapter script `scripts/books-vds-portainer-migration/migrate.sh` clone-adapted from VDS-infra pattern. Auth via `X-API-Key` header (PAT works, JWT fallback not needed). Down step via `docker rm -f` by `com.docker.compose.project` label — bypasses both v1 ContainerConfig bug + any v2 recreate quirks; no compose binary dependency at all.
- 2026-05-25 (Phase 2): **pre-flight surprises:** (1) no `jq` on books VDS — installed static jq 1.7.1 binary via direct GitHub download (yum dead, CentOS 7 EOL since 2024-06, mirrors retired); (2) no `docker login registry.kzntsv.site` configured — accepted (cached images covered all migrations, no force-pull issue). Both documented в concept doc.
- 2026-05-25 (Phase 2): migration executed in ladder. Per-stack Portainer Ids + duration: proxy-chain=28 (~10s recreate, HTTP 400 = service responding), imgproxy=29 (~15s, `imgproxy.kzntsv.site` → 200), minio=30 (~10s, `/minio/health/live` → 200), mongo=31 (~13s, ping=1, books-api/task-runner stayed Up через recreate window), books-db=32 (~15s, `SELECT 1` → 1, books-api connection-pool reconnected), elasticsearch=33 (~37s, cluster yellow expected single-node, **snapshot repo `kreknin` preserved** through bind).
- 2026-05-25 (Phase 3): created `.wiki/concepts/portainer-stack-management-books-vds.md` documenting diffs from vds-kzntsv pattern + per-host inventory + pre-flight quirks (jq install, no registry login). Updated `entities/books-vds.md` table: 6 stacks moved SSH-managed → Portainer-managed.

## Open questions

- [x] ~~**Endpoint 6** в books Portainer — stostayer VDS?~~ Resolved 2026-05-25: stostayer **не наш**, scope не расширяем.
- [x] ~~**Volume preservation strategy**~~ Resolved 2026-05-25 (Phase 0 probe): все 6 target стеков — bind-only, no named volumes. `docker compose down` без `-v` preserves binds (paths остаются на disk).
- [x] ~~**CI implications**~~ Resolved 2026-05-25: books-* stacks (CI-deployed) уже Portainer-managed pre-migration. Migrated stacks (mongo/minio/etc) — infra, не в CI flow. Никаких изменений в `victor/books/.gitea/workflows/deploy.yml` не нужно.
- [x] ~~**ES snapshot path.repo edit preservation**~~ Verified 2026-05-25 (Phase 2 stack 6 smoke): `curl localhost:9200/_snapshot` returned `kreknin` repo intact post-migration. `path.repo=/snapshots` env preserved через `stackFileContent`, bind `/snapshots` → `/usr/docker/elasticsearch/snapshots` preserved.

## Completed steps

- [x] Phase 0 — SSH probe 6 compose dirs (5 original + imgproxy discovered), Portainer API auth verified, surprise imgproxy stack added to scope.
- [x] Phase 1 — adapter script `scripts/books-vds-portainer-migration/migrate.sh` written + committed (`141efe97`). Pre-flight: jq 1.7.1 static binary installed on books VDS.
- [x] Phase 2 — 6 stacks migrated in ladder, all smoked green. Books-api + books-task-runner reconnected transparently through mongo + books-db recreate windows.
- [x] Phase 3 — wiki concept created (`portainer-stack-management-books-vds.md`), entity updated (`books-vds.md` SSH-managed → Portainer-managed table flipped).

## Acceptance verification

- ✓ 6 стеков (proxy-chain, imgproxy, minio, mongo, books-db, elasticsearch) появляются в `GET /api/stacks` под endpoint 1 — Ids 28-33 confirmed via API response after each migration.
- ✓ Контейнеры работают, бинды смонтированы — `docker ps` Up для каждого post-recreate. Data preserved: mongo ping=1, books-db SELECT 1=1, ES cluster yellow with active shards (data intact).
- ✓ Smoke per stack type — все 6 commands green (см. concept doc § Smoke commands).
- ⚠ Backup pipeline продолжает работать — **verified by inspection** (bind paths `/usr/docker/<stack>/data` + `.../snapshots` preserved, backup script paths unchanged). Full pipeline run pending next scheduled 06:00 MSK.
- ✓ Wiki documented — `.wiki/concepts/portainer-stack-management-books-vds.md` + index updated + entity flipped.

## Plan (sketch)

**Phase 0 — pre-flight:**
1. Read each stack's `docker-compose.yml` + `.env` (if exists).
2. Document existing env vars + binds per stack.
3. Backup current compose files (`docker-compose.yml.bak-pre-portainer-migration-2026-MM-DD`).

**Phase 1 — Portainer API setup:**
1. Verify PAT works (`curl ... /api/stacks`). If 401 → fallback to JWT via admin password (from `pass show books-vds/full-env BOOKS_VDS_ROOT_PASS` adapted to Portainer admin — may differ).
2. Build adapter script `scripts/books-vds-portainer-migrate.sh` (clone-and-adapt of VDS-infra version, see [[.wiki/concepts/portainer-stack-management-vds]] § Migration script).

**Phase 2 — per-stack migrate (one at a time, verify between):**
1. **proxy-chain** (lowest blast radius — no traefik exposure)
2. **imgproxy** (2-container compose, serves `imgproxy.kzntsv.site` — drive-by added 2026-05-25)
3. **minio** (read-mostly, books-api retries OK)
4. **mongo** (shared) — books-api + books-task-runner retry-tolerant? **VERIFY first**
5. **books-db** (MariaDB) — books-api uses connection-pool, brief downtime survivable
6. **elasticsearch** — currently unused (per user), zero impact

**Phase 3 — wiki + close:**
1. Document в `.wiki/concepts/portainer-stack-management-books-vds.md` (or extend existing).
2. Update `.wiki/entities/books-vds.md` — flip каждый стек в SSH-managed → Portainer-managed table.
3. Update STATUS.md → 🟢.

## Notes

- **`books-job-scheduler-mongo`** уже Portainer-managed (часть `books-job-scheduler` stack). Не в scope этой таски.
- **`books-docker-proxy` + `books-docker-proxy-ro`** — часть `books-ops-mcp` stack (Portainer). Not in scope.
- **Buildx builders** (`buildx_buildkit_builder-*`) — transient containers без compose, игнорируем.
- **Backup pipeline depend** — после migration paths `/usr/docker/<svc>/data` остаются как bind sources, поэтому `books-vds-backup-daily-kreknin/run.sh` не требует изменений. Compose files (`/usr/docker/<svc>/docker-compose.yml`) станут reference only после migration.

## Cross-refs

- [[.wiki/concepts/portainer-stack-management-vds]] — VDS-infra retro-migration pattern (source of script).
- [[.wiki/entities/books-vds]] — host factsheet (will update post-migration).
- [[books-vds-backup-daily-kreknin]] — backup taskdone 2026-05-25; depend on stable bind paths.
- victor/books `deploy/README.md` — books-* CI deploy через Portainer API (already migrated).

<!-- created-by: claude / 2026-05-25 / trigger: user-requested follow-up after books-vds-backup-daily-kreknin 🟢 -->