scripts(books-vds-portainer-migration): Phase 1 adapter ready
Clone-adapted from .wiki/concepts/portainer-stack-management-vds.md § Migration script. Diffs from VDS-infra version: - PORTAINER_URL = portainer.kzntsv.site - Auth via X-API-Key (PAT works, no JWT fallback needed) - DIR prefix /usr/docker/<stack> (not /opt/stacks/<stack>) - Down step via `docker rm -f` by compose-project label (bypasses docker-compose v1 ContainerConfig bug entirely; no compose binary on path required) - Refuses traefik/portainer migration (management plane) Idempotent: delete-then-create against existing stack by name on endpoint 1. Re-runs are safe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
84
scripts/books-vds-portainer-migration/README.md
Normal file
84
scripts/books-vds-portainer-migration/README.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# books-vds-portainer-migration
|
||||
|
||||
Migrates SSH-compose stacks at `/usr/docker/<stack>/` on books VDS (`89.253.255.133`) to Portainer-managed via API. Adapted from canonical pattern: [`.wiki/concepts/portainer-stack-management-vds.md`](../../.wiki/concepts/portainer-stack-management-vds.md).
|
||||
|
||||
Related task: [`books-vds-stacks-to-portainer`](../../.tasks/books-vds-stacks-to-portainer.md).
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# 1. Get PAT from pass
|
||||
export BOOKS_PORTAINER_API_KEY=$(pass show books-vds/full-env BOOKS_PORTAINER_API_KEY)
|
||||
|
||||
# 2. Copy script to VDS
|
||||
scp -i ~/.ssh/id_ed25519_books_ops migrate.sh root@89.253.255.133:/tmp/
|
||||
|
||||
# 3. Migrate one stack at a time, verify between
|
||||
ssh -i ~/.ssh/id_ed25519_books_ops root@89.253.255.133 \
|
||||
"BOOKS_PORTAINER_API_KEY='$BOOKS_PORTAINER_API_KEY' bash /tmp/migrate.sh proxy-chain"
|
||||
```
|
||||
|
||||
## Ladder (lowest → highest blast radius)
|
||||
|
||||
1. **proxy-chain** — internal-only, no traefik exposure → smoke for the script
|
||||
2. **imgproxy** — 2-container compose, serves `imgproxy.kzntsv.site`
|
||||
3. **minio** — read-mostly, books-api retries OK
|
||||
4. **mongo** — shared, used by books-api + books-task-runner. **VERIFY retry-tolerance first**
|
||||
5. **books-db** — MariaDB, books-api connection-pool reconnects
|
||||
6. **elasticsearch** — currently "not used" per user. Verify before recreate.
|
||||
|
||||
## Smoke per stack (after each migration)
|
||||
|
||||
```bash
|
||||
# proxy-chain — internal smoke (no public endpoint)
|
||||
docker exec proxy-chain wget -qO- localhost:8000 || echo "$?"
|
||||
|
||||
# imgproxy — serves imgproxy.kzntsv.site
|
||||
curl -sS -o /dev/null -w "%{http_code}\n" https://imgproxy.kzntsv.site/
|
||||
|
||||
# minio
|
||||
curl -sS -o /dev/null -w "%{http_code}\n" https://minio.kzntsv.site/minio/health/live
|
||||
|
||||
# mongo
|
||||
docker exec mongo mongo --quiet --eval 'db.adminCommand({ping:1}).ok' \
|
||||
-u root -p "$(pass show books-vds/full-env | grep MONGO_PWD)"
|
||||
|
||||
# books-db
|
||||
docker exec books-db mariadb -uroot -p"$(pass show books-vds/full-env | grep BOOKS_DB_PWD)" -e 'SELECT 1'
|
||||
|
||||
# elasticsearch
|
||||
docker exec elasticsearch curl -sS localhost:9200/_cluster/health | jq .status
|
||||
```
|
||||
|
||||
## What the script does
|
||||
|
||||
1. **Backup** `docker-compose.yml.bak-pre-portainer-migration-<date>` (idempotent — keeps first backup).
|
||||
2. **Transform** compose: strip `env_file:` (defensive — books VDS has none), absolutize `./` paths to `/usr/docker/<stack>/`.
|
||||
3. **Parse** `.env` into API env array (no-op for books VDS — no `.env` files).
|
||||
4. **Delete** existing Portainer stack with same name on endpoint 1 (idempotency for re-runs).
|
||||
5. **Down** ad-hoc containers via `docker rm -f` by `com.docker.compose.project` label (bypasses docker-compose v1 `ContainerConfig` bug).
|
||||
6. **POST** `/api/stacks/create/standalone/string?endpointId=1` — Portainer pulls images, creates network attachments, applies traefik labels through docker socket events.
|
||||
|
||||
## Diffs from VDS-infra script (`portainer-stack-management-vds.md`)
|
||||
|
||||
| Aspect | VDS-infra | books VDS |
|
||||
|---|---|---|
|
||||
| Portainer URL | `portainer.vds.kzntsv.site` | `portainer.kzntsv.site` |
|
||||
| Endpoint ID | 1 (same) | 1 |
|
||||
| Stack dir prefix | `/opt/stacks/<stack>/` | `/usr/docker/<stack>/` |
|
||||
| Auth | JWT (PAT 401'd) | **X-API-Key** (PAT works) |
|
||||
| Compose down | `cd $DIR && docker compose down` | **`docker rm -f` by label** (skips compose binary entirely) |
|
||||
| `env_file:` strip | required (most stacks had .env) | defensive (books VDS has none) |
|
||||
|
||||
## Safety
|
||||
|
||||
- **Re-runnable.** Delete-then-create idempotency means re-running for the same stack just rebuilds it (data binds preserved, names preserved, network re-attached).
|
||||
- **Compose file preserved on disk** (in `/usr/docker/<stack>/` + dated backup). Portainer manages lifecycle but doesn't move the file.
|
||||
- **Bind paths preserved** — `/usr/docker/<stack>/data` etc. stay at the same disk path. Backup pipeline (`scripts/books-vds-backup-daily-kreknin/`) doesn't need adjustment.
|
||||
- **Skips traefik / portainer** by name — refuses to migrate management plane.
|
||||
|
||||
## Post-migration
|
||||
|
||||
- Update [`.wiki/entities/books-vds.md`](../../.wiki/entities/books-vds.md) — flip each stack from "SSH-managed" to "Portainer-managed" table.
|
||||
- Extend or fork [`.wiki/concepts/portainer-stack-management-vds.md`](../../.wiki/concepts/portainer-stack-management-vds.md) with books VDS section + gotchas if new ones encountered.
|
||||
- Close task as 🟢.
|
||||
109
scripts/books-vds-portainer-migration/migrate.sh
Normal file
109
scripts/books-vds-portainer-migration/migrate.sh
Normal file
@@ -0,0 +1,109 @@
|
||||
#!/bin/bash
|
||||
# books-vds-portainer-migration/migrate.sh
|
||||
#
|
||||
# Migrates one SSH-compose stack at /usr/docker/<STACK>/ to Portainer-managed via API.
|
||||
# Pattern source: .wiki/concepts/portainer-stack-management-vds.md § Migration script.
|
||||
# Diffs from VDS-infra version:
|
||||
# PORTAINER_URL = https://portainer.kzntsv.site
|
||||
# ENDPOINT_ID = 1 (books VDS local docker daemon)
|
||||
# DIR prefix = /usr/docker/<stack> (not /opt/stacks/<stack>)
|
||||
# Auth = X-API-Key header (PAT works, no JWT fallback)
|
||||
# Down step = direct `docker rm -f` by compose-project label
|
||||
# (bypasses both v1 standalone quirks and Portainer recreate edge cases)
|
||||
#
|
||||
# Usage (run on books VDS):
|
||||
# scp migrate.sh root@89.253.255.133:/tmp/
|
||||
# ssh root@89.253.255.133
|
||||
# export BOOKS_PORTAINER_API_KEY="ptr_..." # from `pass show books-vds/full-env`
|
||||
# bash /tmp/migrate.sh <stack-name>
|
||||
#
|
||||
# Targets (recommended ladder, lowest → highest blast radius):
|
||||
# proxy-chain → imgproxy → minio → mongo → books-db → elasticsearch
|
||||
#
|
||||
# Skip (management plane — would lose access):
|
||||
# traefik, portainer
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
STACK="${1:?stack name required — e.g. proxy-chain | imgproxy | minio | mongo | books-db | elasticsearch}"
|
||||
DIR="/usr/docker/$STACK"
|
||||
PORTAINER_URL="https://portainer.kzntsv.site"
|
||||
ENDPOINT_ID=1
|
||||
API_KEY="${BOOKS_PORTAINER_API_KEY:?env var required — pass show books-vds/full-env BOOKS_PORTAINER_API_KEY}"
|
||||
|
||||
# Refuse to migrate management-plane stacks
|
||||
case "$STACK" in
|
||||
traefik|portainer)
|
||||
echo "✗ refusing to migrate '$STACK' — management plane (recreate breaks access)"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
[ -d "$DIR" ] || { echo "✗ missing $DIR"; exit 1; }
|
||||
[ -f "$DIR/docker-compose.yml" ] || { echo "✗ missing $DIR/docker-compose.yml"; exit 1; }
|
||||
|
||||
echo "=== migrate '$STACK' from $DIR → Portainer endpoint $ENDPOINT_ID ==="
|
||||
|
||||
# 1. Backup compose file (idempotent — keeps first backup if re-run)
|
||||
BACKUP="$DIR/docker-compose.yml.bak-pre-portainer-migration-$(date +%Y-%m-%d)"
|
||||
if [ ! -f "$BACKUP" ]; then
|
||||
cp "$DIR/docker-compose.yml" "$BACKUP"
|
||||
echo "→ backup: $BACKUP"
|
||||
else
|
||||
echo "→ backup exists: $BACKUP (keeping original)"
|
||||
fi
|
||||
|
||||
# 2. Transform compose:
|
||||
# - strip env_file directives (defensive — Portainer string-mode can't materialize .env)
|
||||
# - absolutize ./ paths (Portainer working_dir = /data/compose/<id>/, not stack dir)
|
||||
COMPOSE=$(sed \
|
||||
-e '/^\s*env_file:/d' \
|
||||
-e '/^\s*-\s*\.env\s*$/d' \
|
||||
-e "s|\./|$DIR/|g" \
|
||||
"$DIR/docker-compose.yml")
|
||||
|
||||
# 3. Parse .env into Portainer API env array (no-op if absent — books VDS has no .env files)
|
||||
ENV_ARRAY="[]"
|
||||
if [ -f "$DIR/.env" ]; then
|
||||
ENV_ARRAY=$(grep -vE '^\s*(#|$)' "$DIR/.env" | jq -Rs '
|
||||
split("\n") | map(select(length>0) | split("=") | {name: .[0], value: (.[1:] | join("="))})')
|
||||
fi
|
||||
|
||||
PAYLOAD=$(jq -n --arg name "$STACK" --arg compose "$COMPOSE" --argjson env "$ENV_ARRAY" \
|
||||
'{name:$name, stackFileContent:$compose, env:$env, fromAppTemplate:false}')
|
||||
|
||||
# 4. Idempotency: delete existing Portainer stack with same name on same endpoint
|
||||
EXISTING_ID=$(curl -ksS -H "X-API-Key: $API_KEY" "$PORTAINER_URL/api/stacks" | \
|
||||
jq -r ".[] | select(.Name==\"$STACK\" and .EndpointId==$ENDPOINT_ID) | .Id" | head -1)
|
||||
if [ -n "$EXISTING_ID" ]; then
|
||||
echo "→ existing Portainer stack id=$EXISTING_ID — deleting"
|
||||
curl -ksS -X DELETE "$PORTAINER_URL/api/stacks/$EXISTING_ID?endpointId=$ENDPOINT_ID" \
|
||||
-H "X-API-Key: $API_KEY" >/dev/null
|
||||
fi
|
||||
|
||||
# 5. Down ad-hoc containers (direct `docker rm -f` by compose-project label —
|
||||
# bypasses docker-compose v1 ContainerConfig bug + any v2 recreate quirks)
|
||||
CONTAINERS=$(docker ps -aq --filter "label=com.docker.compose.project=$STACK" || true)
|
||||
if [ -n "$CONTAINERS" ]; then
|
||||
COUNT=$(echo "$CONTAINERS" | wc -l)
|
||||
echo "→ removing $COUNT ad-hoc container(s) (by compose-project label)"
|
||||
docker rm -f $CONTAINERS
|
||||
fi
|
||||
|
||||
# 6. Create via Portainer API
|
||||
echo "→ POST /api/stacks/create/standalone/string"
|
||||
RESP=$(curl -ksS -X POST "$PORTAINER_URL/api/stacks/create/standalone/string?endpointId=$ENDPOINT_ID" \
|
||||
-H "X-API-Key: $API_KEY" -H "Content-Type: application/json" --data "$PAYLOAD")
|
||||
NEW_ID=$(echo "$RESP" | jq -r '.Id // empty')
|
||||
|
||||
if [ -n "$NEW_ID" ]; then
|
||||
echo "✓ stack '$STACK' created, Portainer Id=$NEW_ID"
|
||||
echo
|
||||
echo "→ containers now running:"
|
||||
docker ps --filter "label=com.docker.compose.project=$STACK" \
|
||||
--format 'table {{.Names}}\t{{.Image}}\t{{.Status}}'
|
||||
else
|
||||
echo "✗ FAILED:"
|
||||
echo "$RESP" | jq .
|
||||
exit 1
|
||||
fi
|
||||
Reference in New Issue
Block a user