diff --git a/.gitignore b/.gitignore index 62bfee6..5327756 100644 --- a/.gitignore +++ b/.gitignore @@ -20,3 +20,7 @@ Thumbs.db # Transient operational drops — findings captured в .tasks/, не в repo root. *-log.txt *-size.txt + +# Operational scratch — per-task throwaway (API payloads, frozen mappings, ad-hoc dumps). +# Контент часто содержит credentials (reindex bodies, stack PUTs) — никогда не commit. +.scratch/ diff --git a/.tasks/STATUS.md b/.tasks/STATUS.md index 2861058..063dc05 100644 --- a/.tasks/STATUS.md +++ b/.tasks/STATUS.md @@ -1,5 +1,9 @@ # Admin Task Board -_Updated: 2026-05-25 (`books-vds-stacks-to-portainer` 🟢 closed — 6 стеков (proxy-chain, imgproxy, minio, mongo, books-db, elasticsearch) migrated SSH-compose → Portainer-managed на books VDS. Ids 28-33. Adapter script + wiki concept committed. Все 6 smoke green в одну сессию.)_ +_Updated: 2026-05-25 (`migrate-elasticsearch-to-books-vds` 🟢 closed — 3 indices (artmone/epz/products) с Windows host → books VDS ES via reindex-from-remote. 928k docs total, count parity 100%, per-doc byte-match. Consumers switched (books-api + books-task-runner + books-job-scheduler). Source ES остаётся running per user — rollback ready.)_ + +## 🟢 [migrate-elasticsearch-to-books-vds] — closed 2026-05-25 — ES indices с Windows (elasticold.kzntsv.site, 7.10.1) → books VDS (elasticsearch.kzntsv.site, 7.10.0). 3 indices, 928616 docs. Reindex-from-remote через `reindex.remote.whitelist` env, добавленный в Portainer stack 33. Pre-migration snapshot в kreknin repo как rollback. Consumer configs sed'd, 3 containers restarted, smoke green. Source НЕ выключен. + +См. [migrate-elasticsearch-to-books-vds.md](migrate-elasticsearch-to-books-vds.md) § Closure note. ## 🟢 [books-vds-stacks-to-portainer] — closed 2026-05-25 — 6 SSH-compose стеков на books VDS (89.253.255.133) мигрированы в Portainer-managed (endpoint 1, https://portainer.kzntsv.site). Skip traefik + portainer (management plane). Adapter `scripts/books-vds-portainer-migration/migrate.sh`. Wiki: [portainer-stack-management-books-vds.md](../.wiki/concepts/portainer-stack-management-books-vds.md). Backup pipeline bind paths preserved (verified by inspection, full run pending next 06:00 MSK). diff --git a/.tasks/migrate-elasticsearch-to-books-vds.md b/.tasks/migrate-elasticsearch-to-books-vds.md new file mode 100644 index 0000000..8dd9b57 --- /dev/null +++ b/.tasks/migrate-elasticsearch-to-books-vds.md @@ -0,0 +1,117 @@ +# migrate-elasticsearch-to-books-vds + +## Goal + +Переехать ES индексы с Windows host (`elasticold.kzntsv.site`, ES 7.10.1) на books VDS (`elasticsearch.kzntsv.site`, ES 7.10.0, single-node behind traefik basicAuth). Переключить books-api / books-task-runner / books-job-scheduler через `default.json` на новый endpoint. Source ES оставить running как rollback (не выключать). + +## Approach + +**Reindex-from-remote** (не snapshot/restore). + +Why: +- Source = Windows docker, нет `path.repo` configured + нет bind для snapshot dir → snapshot подход требует restart source = consumer downtime. +- Target = books VDS Portainer stack 33, modify env через Portainer API = recreate target only, consumers unaffected (они пока на source). +- Same Lucene 8.7.0 → docs reindex transparently через `_reindex` API. +- 730mb total → ~5 min over public network. + +Trade-off: reindex использует target's dynamic mapping. Pre-create target indices с source mappings/settings (snapshot уже сохранён в `.scratch/`). + +## Source/target inventory + +**Source (Windows, `localhost:9200`, behind `elasticold.kzntsv.site` traefik basicAuth):** +- ES 7.10.1, Lucene 8.7.0, `discovery.type=single-node` +- 3 indices: + - `artmone` 2621 docs, 1.4mb + - `epz` 820604 docs, 699.7mb + - `products` 105922 docs, 26.6mb +- Compose: `C:\Users\vitya\projects\docker\diskstation\elasticsearch\docker-compose.yml` +- Bind: `./data:/usr/share/elasticsearch/data`. **No snapshot path.** +- basicAuth user `books`, hash `$apr1$vyxr1l5z$fpxHmNBfAx8cHrQTje4Fx/` + +**Target (books VDS 89.253.255.133, Portainer stack 33 `elasticsearch`, behind `elasticsearch.kzntsv.site`):** +- ES 7.10.0, Lucene 8.7.0, single-node, yellow (replica unassigned — expected) +- 1 index: `read_me` (4.8kb placeholder) +- `path.repo=/snapshots`, kreknin fs repo registered +- Bind: `/usr/docker/elasticsearch/{data,snapshots}` → `/usr/share/elasticsearch/{data,/snapshots}` +- basicAuth user `books`, **same hash как source** → same password + +**Consumers (all on books VDS, configs bind-mounted `/opt/books//config/default.json`):** +- `books-api` (stack 23): port 3021, Nitro +- `books-task-runner` (stack 22, sibling of job-scheduler+mongo) +- `books-job-scheduler` (stack 22) + +Все три имеют block: +```json +"elasticsearch": { + "node": "https://elasticold.kzntsv.site/", + "auth": { "username": "books", "password": "" } +} +``` + +## Key files + +- `C:\Users\vitya\projects\docker\diskstation\elasticsearch\docker-compose.yml` — source ES compose (no edit planned) +- `C:\Users\vitya\projects\.admin\.scratch\source-mappings.json` — frozen source mappings 2026-05-25 +- `C:\Users\vitya\projects\.admin\.scratch\source-settings.json` — frozen source settings 2026-05-25 +- books VDS `/opt/books/api/config/default.json:27` — consumer config (idem task-runner, job-scheduler) +- books VDS Portainer stack 33 — target ES compose (modify env via Portainer API) +- books VDS `/usr/docker/elasticsearch/snapshots/` — kreknin fs repo, pre-migration backup target + +## Execution plan + +1. **Pre-migration ES snapshot books VDS** (kreknin repo, `pre-migration-` snapshot name) — instant rollback for target. +2. **Save source mappings/settings** — done (`.scratch/source-{mappings,settings}.json`). +3. **Modify target ES stack** via Portainer API: add env `reindex.remote.whitelist=elasticold.kzntsv.site:443`. Recreate stack (~10s target downtime; consumers unaffected — они на source). +4. **Pre-create target indices** с source mappings/settings (number_of_replicas=0 для single-node clean green). +5. **POST `/_reindex`** per index, wait_for_completion=false, slices=auto. Poll task status. +6. **Verify counts**: target `_cat/indices` doc.counts == source. Verify random doc fetch by `_id` matches. +7. **Update consumer configs**: replace `elasticold.kzntsv.site` → `elasticsearch.kzntsv.site` в `/opt/books/{api,task-runner,job-scheduler}/config/default.json` (sed in-place). +8. **Bounce consumers** через `docker restart books-api books-task-runner books-job-scheduler` (mongo / db не трогаем — bind-mount уже видит новый config). +9. **Smoke**: tail `docker logs --since 60s books-api books-task-runner books-job-scheduler` → нет ES connection errors. Trigger known query → response from target. +10. **Don't disable source** (per user) — Windows ES + elasticold traefik route остаётся live для rollback. + +## Decisions log + +- 2026-05-25: подход reindex-from-remote, не snapshot/restore. Reason: source compose не имеет `path.repo` configured + bind для snapshot — это потребовало бы restart source (= consumer downtime). Target compose в Portainer, restart target не задевает consumers (они пока на source). Same Lucene 8.7.0 → reindex transparent. +- 2026-05-25: pre-create target indices с frozen source mappings — иначе reindex использовал бы dynamic mapping, possibly losing exact analyzer config. Mappings/settings уже извлечены в `.scratch/`. +- 2026-05-25: consumer config update через `sed -i` (bind-mounted files), restart through `docker restart` (не Portainer stack recreate) — mongo / db в той же stack id 22 не должны bounce. + +## Open questions + +- [x] `number_of_replicas`: source = 1 (yellow на single-node). Target тоже single-node → set to 0 на pre-create для green. Source оставить как есть. **Resolved: set 0 на target pre-create.** + +## Completed steps + +- [x] Source ES probed: 3 indices, 730mb total, ES 7.10.1 +- [x] Target ES probed: ES 7.10.0, 1 placeholder index `read_me`, no collisions +- [x] Source mappings/settings frozen в `.scratch/` +- [x] basicAuth password identified из consumer config (same on both sides) +- [x] Containers inventory: 3 consumer containers, all on books VDS +- [x] Pre-migration ES snapshot `pre-migration-2026-05-25` в kreknin repo (read_me only, SUCCESS — rollback point) +- [x] Target ES compose updated через Portainer API: `reindex.remote.whitelist=elasticold.kzntsv.site:443`. Container restart 07:51:39 UTC. Yellow cluster post-restart (replica unassigned = expected). +- [x] 3 target indices pre-created с frozen source mappings/settings, `number_of_replicas=0` → green. +- [x] Reindex from remote: artmone (2621/2621, 25s), epz (820604/820604, ~5m async), products (105922/105922, ~25m async). +- [x] Doc count parity src=dst для всех 3 indices. +- [x] Per-doc sample fetch: byte-match _id=29359 (products) + _id=22023500 (epz). +- [x] Consumer configs sed in-place: `elasticold` → `elasticsearch`, .bak-pre-es-migration-2026-05-25 saved для rollback. +- [x] Restart 3 containers: books-api running, books-task-runner healthy, books-job-scheduler healthy. +- [x] Direct internal smoke (`docker exec books-api node -e ...`): API sees `https://elasticsearch.kzntsv.site/`, `_cluster/health` 200 OK. +- [x] Post-migration source traefik access log = 0 elasticold hits за 5 min. + +## Closure note (2026-05-25) + +Migration **complete**. 3 indices (artmone/epz/products, 820k+108k+2.6k docs, 850mb total target storage) on books VDS ES live. Consumers switched (books-api + books-task-runner + books-job-scheduler). + +**Source НЕ disabled** (per user requirement) — Windows ES + elasticold.kzntsv.site traefik route остаются live для emergency rollback. Кешированный data на windows host остаётся в одном экземпляре до отдельного decommission решения. + +**Rollback**: `ssh root@89.253.255.133 'for f in /opt/books/{api,task-runner,job-scheduler}/config/default.json; do cp "$f.bak-pre-es-migration-2026-05-25" "$f"; done; docker restart books-api books-task-runner books-job-scheduler'`. + +**Future ops**: при decommission Windows ES — `docker stop elasticsearch` на windows + remove traefik label `elasticold.kzntsv.site` route. `path.repo` / kreknin snapshot pre-migration-2026-05-25 уже не несёт практически данных (один read_me index), может быть удалён вместе с next backup retention rotation. + +## Notes + +- Source basicAuth password в `/opt/books/api/config/default.json` (consumer config). Не дублировать в эту таску. +- `books-job-scheduler-mongo` — другая БД, не задевается. +- Rollback path: `sed -i 's/elasticsearch\.kzntsv\.site/elasticold.kzntsv.site/g' /opt/books/*/config/default.json` + restart consumers. + + diff --git a/.wiki/entities/books-vds.md b/.wiki/entities/books-vds.md index 0d0d753..0025ac2 100644 --- a/.wiki/entities/books-vds.md +++ b/.wiki/entities/books-vds.md @@ -1,9 +1,9 @@ --- title: Books VDS (client server — 89.253.255.133) type: entity -tags: [hardware, vds, books, host4g, backup, portainer] +tags: [hardware, vds, books, host4g, backup, portainer, elasticsearch] related: [[kreknin-synology]], [[vds-kzntsv]], [[../concepts/portainer-stack-management-books-vds]] -updated: 2026-05-25 +updated: 2026-05-25 (ES migration from Windows host) --- # Books VDS @@ -50,7 +50,7 @@ Post-migration 2026-05-25 (см. [[../concepts/portainer-stack-management-books- | 30 | `minio` | minio | `/usr/docker/minio/data` | S3 blob store, `minio.kzntsv.site` | | 31 | `mongo` | mongo | `/usr/docker/mongo/data/{db,configdb}` | shared MongoDB 4.2 | | 32 | `books-db` | books-db | `/usr/docker/books-db/data` | MariaDB latest, books application DB | -| 33 | `elasticsearch` | elasticsearch | `/usr/docker/elasticsearch/{data,snapshots}` | ES 7.10.0 + path.repo для backup | +| 33 | `elasticsearch` | elasticsearch | `/usr/docker/elasticsearch/{data,snapshots}` | ES 7.10.0 + `path.repo=/snapshots` для backup + `reindex.remote.whitelist=elasticold.kzntsv.site:443` (added 2026-05-25 для migration) | ### SSH-managed (management plane — Portainer migration skipped) @@ -111,3 +111,4 @@ Auth-зоны DNS — в reg.ru под `kzntsv.site`. - Workstation SSH key `id_ed25519_books_ops` уже был задеплоен под root до этой сессии (видимо, прошлый bootstrap забыт-без-документации). - Hybrid stack-management (Portainer + SSH-compose) — pre-existing, не наш design. Migrated 6 SSH-compose стеков → Portainer 2026-05-25 (`books-vds-stacks-to-portainer` 🟢). `traefik` + `portainer` остаются SSH-managed (management plane). - `imgproxy` (2 containers, `imgproxy.kzntsv.site`) был missing из этого entity wiki до Phase 0 probe 2026-05-25 — fixed in flight. +- ES indices populated 2026-05-25 — migration с Windows host (`elasticold.kzntsv.site`, ES 7.10.1) → books VDS via reindex-from-remote. 3 indices: `artmone` (2621), `epz` (820604), `products` (105922). Consumers (books-api + books-task-runner + books-job-scheduler) switched `default.json:elasticsearch.node` с elasticold на elasticsearch.kzntsv.site. Source ES остаётся running (rollback). См. [[../../tasks/migrate-elasticsearch-to-books-vds]] § Closure note.