tasks(migrate-elasticsearch-to-books-vds): 🟢 closed — 3 indices Windows→books VDS

928616 docs (artmone 2621 / epz 820604 / products 105922) migrated via
reindex-from-remote. Pre-migration snapshot к kreknin repo, target stack 33
получил `reindex.remote.whitelist=elasticold.kzntsv.site:443` через Portainer
API. Consumer configs (books-api + books-task-runner + books-job-scheduler)
sed'd elasticold→elasticsearch.kzntsv.site, 3 containers restarted.
Per-doc byte-match verified, internal smoke от books-api → новый ES 200 OK.
Source ES оставлен running per user (rollback ready).

.gitignore

@@ -20,3 +20,7 @@ Thumbs.db
 # Transient operational drops — findings captured в .tasks/, не в repo root.
 *-log.txt
 *-size.txt
+
+# Operational scratch — per-task throwaway (API payloads, frozen mappings, ad-hoc dumps).
+# Контент часто содержит credentials (reindex bodies, stack PUTs) — никогда не commit.
+.scratch/

.tasks/STATUS.md

@@ -1,5 +1,9 @@
 # Admin Task Board
-_Updated: 2026-05-25 (`books-vds-stacks-to-portainer` 🟢 closed — 6 стеков (proxy-chain, imgproxy, minio, mongo, books-db, elasticsearch) migrated SSH-compose → Portainer-managed на books VDS. Ids 28-33. Adapter script + wiki concept committed. Все 6 smoke green в одну сессию.)_
+_Updated: 2026-05-25 (`migrate-elasticsearch-to-books-vds` 🟢 closed — 3 indices (artmone/epz/products) с Windows host → books VDS ES via reindex-from-remote. 928k docs total, count parity 100%, per-doc byte-match. Consumers switched (books-api + books-task-runner + books-job-scheduler). Source ES остаётся running per user — rollback ready.)_
+
+## 🟢 [migrate-elasticsearch-to-books-vds] — closed 2026-05-25 — ES indices с Windows (elasticold.kzntsv.site, 7.10.1) → books VDS (elasticsearch.kzntsv.site, 7.10.0). 3 indices, 928616 docs. Reindex-from-remote через `reindex.remote.whitelist` env, добавленный в Portainer stack 33. Pre-migration snapshot в kreknin repo как rollback. Consumer configs sed'd, 3 containers restarted, smoke green. Source НЕ выключен.
+
+См. [migrate-elasticsearch-to-books-vds.md](migrate-elasticsearch-to-books-vds.md) § Closure note.
 
 ## 🟢 [books-vds-stacks-to-portainer] — closed 2026-05-25 — 6 SSH-compose стеков на books VDS (89.253.255.133) мигрированы в Portainer-managed (endpoint 1, https://portainer.kzntsv.site). Skip traefik + portainer (management plane). Adapter `scripts/books-vds-portainer-migration/migrate.sh`. Wiki: [portainer-stack-management-books-vds.md](../.wiki/concepts/portainer-stack-management-books-vds.md). Backup pipeline bind paths preserved (verified by inspection, full run pending next 06:00 MSK).
 

.tasks/migrate-elasticsearch-to-books-vds.md

@@ -0,0 +1,117 @@
+# migrate-elasticsearch-to-books-vds
+
+## Goal
+
+Переехать ES индексы с Windows host (`elasticold.kzntsv.site`, ES 7.10.1) на books VDS (`elasticsearch.kzntsv.site`, ES 7.10.0, single-node behind traefik basicAuth). Переключить books-api / books-task-runner / books-job-scheduler через `default.json` на новый endpoint. Source ES оставить running как rollback (не выключать).
+
+## Approach
+
+**Reindex-from-remote** (не snapshot/restore).
+
+Why:
+- Source = Windows docker, нет `path.repo` configured + нет bind для snapshot dir → snapshot подход требует restart source = consumer downtime.
+- Target = books VDS Portainer stack 33, modify env через Portainer API = recreate target only, consumers unaffected (они пока на source).
+- Same Lucene 8.7.0 → docs reindex transparently через `_reindex` API.
+- 730mb total → ~5 min over public network.
+
+Trade-off: reindex использует target's dynamic mapping. Pre-create target indices с source mappings/settings (snapshot уже сохранён в `.scratch/`).
+
+## Source/target inventory
+
+**Source (Windows, `localhost:9200`, behind `elasticold.kzntsv.site` traefik basicAuth):**
+- ES 7.10.1, Lucene 8.7.0, `discovery.type=single-node`
+- 3 indices:
+  - `artmone` 2621 docs, 1.4mb
+  - `epz` 820604 docs, 699.7mb
+  - `products` 105922 docs, 26.6mb
+- Compose: `C:\Users\vitya\projects\docker\diskstation\elasticsearch\docker-compose.yml`
+- Bind: `./data:/usr/share/elasticsearch/data`. **No snapshot path.**
+- basicAuth user `books`, hash `$apr1$vyxr1l5z$fpxHmNBfAx8cHrQTje4Fx/`
+
+**Target (books VDS 89.253.255.133, Portainer stack 33 `elasticsearch`, behind `elasticsearch.kzntsv.site`):**
+- ES 7.10.0, Lucene 8.7.0, single-node, yellow (replica unassigned — expected)
+- 1 index: `read_me` (4.8kb placeholder)
+- `path.repo=/snapshots`, kreknin fs repo registered
+- Bind: `/usr/docker/elasticsearch/{data,snapshots}` → `/usr/share/elasticsearch/{data,/snapshots}`
+- basicAuth user `books`, **same hash как source** → same password
+
+**Consumers (all on books VDS, configs bind-mounted `/opt/books/<svc>/config/default.json`):**
+- `books-api` (stack 23): port 3021, Nitro
+- `books-task-runner` (stack 22, sibling of job-scheduler+mongo)
+- `books-job-scheduler` (stack 22)
+
+Все три имеют block:
+```json
+"elasticsearch": {
+    "node": "https://elasticold.kzntsv.site/",
+    "auth": { "username": "books", "password": "<same>" }
+}
+```
+
+## Key files
+
+- `C:\Users\vitya\projects\docker\diskstation\elasticsearch\docker-compose.yml` — source ES compose (no edit planned)
+- `C:\Users\vitya\projects\.admin\.scratch\source-mappings.json` — frozen source mappings 2026-05-25
+- `C:\Users\vitya\projects\.admin\.scratch\source-settings.json` — frozen source settings 2026-05-25
+- books VDS `/opt/books/api/config/default.json:27` — consumer config (idem task-runner, job-scheduler)
+- books VDS Portainer stack 33 — target ES compose (modify env via Portainer API)
+- books VDS `/usr/docker/elasticsearch/snapshots/` — kreknin fs repo, pre-migration backup target
+
+## Execution plan
+
+1. **Pre-migration ES snapshot books VDS** (kreknin repo, `pre-migration-<date>` snapshot name) — instant rollback for target.
+2. **Save source mappings/settings** — done (`.scratch/source-{mappings,settings}.json`).
+3. **Modify target ES stack** via Portainer API: add env `reindex.remote.whitelist=elasticold.kzntsv.site:443`. Recreate stack (~10s target downtime; consumers unaffected — они на source).
+4. **Pre-create target indices** с source mappings/settings (number_of_replicas=0 для single-node clean green).
+5. **POST `/_reindex`** per index, wait_for_completion=false, slices=auto. Poll task status.
+6. **Verify counts**: target `_cat/indices` doc.counts == source. Verify random doc fetch by `_id` matches.
+7. **Update consumer configs**: replace `elasticold.kzntsv.site` → `elasticsearch.kzntsv.site` в `/opt/books/{api,task-runner,job-scheduler}/config/default.json` (sed in-place).
+8. **Bounce consumers** через `docker restart books-api books-task-runner books-job-scheduler` (mongo / db не трогаем — bind-mount уже видит новый config).
+9. **Smoke**: tail `docker logs --since 60s books-api books-task-runner books-job-scheduler` → нет ES connection errors. Trigger known query → response from target.
+10. **Don't disable source** (per user) — Windows ES + elasticold traefik route остаётся live для rollback.
+
+## Decisions log
+
+- 2026-05-25: подход reindex-from-remote, не snapshot/restore. Reason: source compose не имеет `path.repo` configured + bind для snapshot — это потребовало бы restart source (= consumer downtime). Target compose в Portainer, restart target не задевает consumers (они пока на source). Same Lucene 8.7.0 → reindex transparent.
+- 2026-05-25: pre-create target indices с frozen source mappings — иначе reindex использовал бы dynamic mapping, possibly losing exact analyzer config. Mappings/settings уже извлечены в `.scratch/`.
+- 2026-05-25: consumer config update через `sed -i` (bind-mounted files), restart through `docker restart` (не Portainer stack recreate) — mongo / db в той же stack id 22 не должны bounce.
+
+## Open questions
+
+- [x] `number_of_replicas`: source = 1 (yellow на single-node). Target тоже single-node → set to 0 на pre-create для green. Source оставить как есть. **Resolved: set 0 на target pre-create.**
+
+## Completed steps
+
+- [x] Source ES probed: 3 indices, 730mb total, ES 7.10.1
+- [x] Target ES probed: ES 7.10.0, 1 placeholder index `read_me`, no collisions
+- [x] Source mappings/settings frozen в `.scratch/`
+- [x] basicAuth password identified из consumer config (same on both sides)
+- [x] Containers inventory: 3 consumer containers, all on books VDS
+- [x] Pre-migration ES snapshot `pre-migration-2026-05-25` в kreknin repo (read_me only, SUCCESS — rollback point)
+- [x] Target ES compose updated через Portainer API: `reindex.remote.whitelist=elasticold.kzntsv.site:443`. Container restart 07:51:39 UTC. Yellow cluster post-restart (replica unassigned = expected).
+- [x] 3 target indices pre-created с frozen source mappings/settings, `number_of_replicas=0` → green.
+- [x] Reindex from remote: artmone (2621/2621, 25s), epz (820604/820604, ~5m async), products (105922/105922, ~25m async).
+- [x] Doc count parity src=dst для всех 3 indices.
+- [x] Per-doc sample fetch: byte-match _id=29359 (products) + _id=22023500 (epz).
+- [x] Consumer configs sed in-place: `elasticold` → `elasticsearch`, .bak-pre-es-migration-2026-05-25 saved для rollback.
+- [x] Restart 3 containers: books-api running, books-task-runner healthy, books-job-scheduler healthy.
+- [x] Direct internal smoke (`docker exec books-api node -e ...`): API sees `https://elasticsearch.kzntsv.site/`, `_cluster/health` 200 OK.
+- [x] Post-migration source traefik access log = 0 elasticold hits за 5 min.
+
+## Closure note (2026-05-25)
+
+Migration **complete**. 3 indices (artmone/epz/products, 820k+108k+2.6k docs, 850mb total target storage) on books VDS ES live. Consumers switched (books-api + books-task-runner + books-job-scheduler).
+
+**Source НЕ disabled** (per user requirement) — Windows ES + elasticold.kzntsv.site traefik route остаются live для emergency rollback. Кешированный data на windows host остаётся в одном экземпляре до отдельного decommission решения.
+
+**Rollback**: `ssh root@89.253.255.133 'for f in /opt/books/{api,task-runner,job-scheduler}/config/default.json; do cp "$f.bak-pre-es-migration-2026-05-25" "$f"; done; docker restart books-api books-task-runner books-job-scheduler'`.
+
+**Future ops**: при decommission Windows ES — `docker stop elasticsearch` на windows + remove traefik label `elasticold.kzntsv.site` route. `path.repo` / kreknin snapshot pre-migration-2026-05-25 уже не несёт практически данных (один read_me index), может быть удалён вместе с next backup retention rotation.
+
+## Notes
+
+- Source basicAuth password в `/opt/books/api/config/default.json` (consumer config). Не дублировать в эту таску.
+- `books-job-scheduler-mongo` — другая БД, не задевается.
+- Rollback path: `sed -i 's/elasticsearch\.kzntsv\.site/elasticold.kzntsv.site/g' /opt/books/*/config/default.json` + restart consumers.
+
+<!-- created-by: vitya / 2026-05-25 / trigger: ES consolidation на books VDS, source-host decommission preparation -->

.wiki/entities/books-vds.md

@@ -1,9 +1,9 @@
 ---
 title: Books VDS (client server — 89.253.255.133)
 type: entity
-tags: [hardware, vds, books, host4g, backup, portainer]
+tags: [hardware, vds, books, host4g, backup, portainer, elasticsearch]
 related: [[kreknin-synology]], [[vds-kzntsv]], [[../concepts/portainer-stack-management-books-vds]]
-updated: 2026-05-25
+updated: 2026-05-25 (ES migration from Windows host)
 ---
 
 # Books VDS
@@ -50,7 +50,7 @@ Post-migration 2026-05-25 (см. [[../concepts/portainer-stack-management-books-
 | 30 | `minio` | minio | `/usr/docker/minio/data` | S3 blob store, `minio.kzntsv.site` |
 | 31 | `mongo` | mongo | `/usr/docker/mongo/data/{db,configdb}` | shared MongoDB 4.2 |
 | 32 | `books-db` | books-db | `/usr/docker/books-db/data` | MariaDB latest, books application DB |
-| 33 | `elasticsearch` | elasticsearch | `/usr/docker/elasticsearch/{data,snapshots}` | ES 7.10.0 + path.repo для backup |
+| 33 | `elasticsearch` | elasticsearch | `/usr/docker/elasticsearch/{data,snapshots}` | ES 7.10.0 + `path.repo=/snapshots` для backup + `reindex.remote.whitelist=elasticold.kzntsv.site:443` (added 2026-05-25 для migration) |
 
 ### SSH-managed (management plane — Portainer migration skipped)
 
@@ -111,3 +111,4 @@ Auth-зоны DNS — в reg.ru под `kzntsv.site`.
 - Workstation SSH key `id_ed25519_books_ops` уже был задеплоен под root до этой сессии (видимо, прошлый bootstrap забыт-без-документации).
 - Hybrid stack-management (Portainer + SSH-compose) — pre-existing, не наш design. Migrated 6 SSH-compose стеков → Portainer 2026-05-25 (`books-vds-stacks-to-portainer` 🟢). `traefik` + `portainer` остаются SSH-managed (management plane).
 - `imgproxy` (2 containers, `imgproxy.kzntsv.site`) был missing из этого entity wiki до Phase 0 probe 2026-05-25 — fixed in flight.
+- ES indices populated 2026-05-25 — migration с Windows host (`elasticold.kzntsv.site`, ES 7.10.1) → books VDS via reindex-from-remote. 3 indices: `artmone` (2621), `epz` (820604), `products` (105922). Consumers (books-api + books-task-runner + books-job-scheduler) switched `default.json:elasticsearch.node` с elasticold на elasticsearch.kzntsv.site. Source ES остаётся running (rollback). См. [[../../tasks/migrate-elasticsearch-to-books-vds]] § Closure note.