tasks(NEXT_SESSION): refresh handoff after ES migration commit

928616 docs (artmone 2621 / epz 820604 / products 105922) migrated via
reindex-from-remote. Pre-migration snapshot к kreknin repo, target stack 33
получил `reindex.remote.whitelist=elasticold.kzntsv.site:443` через Portainer
API. Consumer configs (books-api + books-task-runner + books-job-scheduler)
sed'd elasticold→elasticsearch.kzntsv.site, 3 containers restarted.
Per-doc byte-match verified, internal smoke от books-api → новый ES 200 OK.
Source ES оставлен running per user (rollback ready).

.gitignore

@@ -20,3 +20,7 @@ Thumbs.db
 # Transient operational drops — findings captured в .tasks/, не в repo root.
 *-log.txt
 *-size.txt
+
+# Operational scratch — per-task throwaway (API payloads, frozen mappings, ad-hoc dumps).
+# Контент часто содержит credentials (reindex bodies, stack PUTs) — никогда не commit.
+.scratch/

.tasks/NEXT_SESSION.md

@@ -1,25 +1,25 @@
 ---
-_last_updated_: 2026-05-25T12:00:00+03:00
-session_id: 2026-05-25-books-vds-stacks-to-portainer
+_last_updated_: 2026-05-25T11:30:00+03:00
+session_id: 2026-05-25-es-windows-to-books-vds
 ---
 
 # Next session handoff
 
 ## Recent commits
 
-- `tasks(books-vds-stacks-to-portainer): close 🟢 — 6 stacks migrated SSH-compose → Portainer` (`86c44c3d`) — session close.
+- `tasks(migrate-elasticsearch-to-books-vds): 🟢 closed — 3 indices Windows→books VDS` (`3246dabb`) — session close.
+- `tasks(NEXT_SESSION): finalize handoff at session close` (`97757b89`) — prev session close.
+- `tasks(books-vds-stacks-to-portainer): close 🟢 — 6 stacks migrated SSH-compose → Portainer` (`86c44c3d`)
 - `scripts(books-vds-portainer-migration): Phase 1 adapter ready` (`141efe97`)
 - `tasks(books-vds-stacks-to-portainer): Phase 0 probe complete, +imgproxy scope` (`bfb942cd`)
-- `tasks: add ⚪ books-vds-stacks-to-portainer` (`439da3ce`)
-- `backup(books-vds): daily 06:00 MSK pipeline to kreknin live 🟢` (`fe41ee44`)
 
 ## Open треки
 
 | Трек | Готовность | Entry-point |
 |---|---|---|
 | `stateful-split-volume-copy` ⚪ | not started; lean playbook готов, fits real books VDS (89.253.255.133) | `.tasks/stateful-split-volume-copy.md` |
-| `iis-migration-to-ruvds` 🟡 | soak (kupimknigi + emspb DNS flipped 2026-05-24, 22 pending) | `.tasks/iis-migration-to-ruvds.md` |
-| `infra-inventory` ⚪ | ready, не начат — **обновлён factsheet `books-vds` после Portainer migration**, готов к ingest в admin-detailed runbook | `.tasks/infra-inventory.md` |
+| `iis-migration-to-ruvds` 🟡 | soak (kupimknigi + emspb DNS flipped 2026-05-24, 22 pending DNS swaps) | `.tasks/iis-migration-to-ruvds.md` |
+| `infra-inventory` ⚪ | ready, не начат — **books VDS factsheet ещё свежее после ES migration**, готов к ingest в admin-detailed runbook | `.tasks/infra-inventory.md` |
 | `books-bookva-user-whitelist-gathering` ⚪ | ready, long-lead gathering (email-based) | `.tasks/books-bookva-user-whitelist-gathering.md` |
 | `books-vds-bookva-bootstrap` 🔵 | blocked on tenant-split Phase 4 | `.tasks/books-vds-bookva-bootstrap.md` |
 | `books-dns-cutover-bookva` 🔵 | blocked on books-vds-bookva-bootstrap | `.tasks/books-dns-cutover-bookva.md` |
@@ -27,22 +27,28 @@ session_id: 2026-05-25-books-vds-stacks-to-portainer
 
 ## Спроси user'а
 
-- **Verify backup pipeline post-migration:** next scheduled run 06:00 MSK 2026-05-26 (через ~18h от close). Если ntfy/email прийдут с success — acceptance criterion #4 finalized. Если нет — investigate (most likely bind paths intact, low risk).
-- **Какую таску брать следующей?** Active candidates: `stateful-split-volume-copy` (ops, lean playbook готов), `infra-inventory` (книги VDS теперь fully documented — момент удачный), `iis-migration-to-ruvds` (DNS TTL reductions для 22 pending hosts).
+- **Decommission Windows ES** — когда выключаем `elasticsearch` container на windows host + удаляем traefik label `elasticold.kzntsv.site`? Source оставлен running per request, но это transient: live load теперь = 0 hits (verified 5 min post-cutover), кэш на windows host практический balast. Trigger момент — на user'е.
+- **Rollback retention** — `.bak-pre-es-migration-2026-05-25` файлы в `/opt/books/{api,task-runner,job-scheduler}/config/` стоит держать до decommission момента или дольше?
+- **kreknin snapshot `pre-migration-2026-05-25`** (только read_me, 4.8kb) — оставить до retention rotation или явно удалить как «полезность нулевая»?
+- **Какую таску брать следующей?** Хорошие кандидаты: `infra-inventory` (books VDS теперь fully documented + ES populated — момент удачный), `iis-migration-to-ruvds` (DNS TTL для 22 hostnames), `stateful-split-volume-copy` (lean playbook готов).
 
 ## Не делать (preemptive guards)
 
-- **Не editit `/usr/docker/<svc>/docker-compose.yml`** на books VDS — теперь reference-only. Portainer-managed compose state в `/data/compose/<id>/docker-compose.yml`. Edit через Portainer UI или API.
-- **Не `docker compose up -d`** на books VDS для migrated стеков — стартанёт parallel set контейнеров под compose-project `<dirname>`, conflicting names с Portainer-managed. Trust Portainer для lifecycle.
-- **Не push without grant in next session** — auto-push grant был session-only (Rule 4). Next session start = ask-mode again.
-- **Не migrate traefik / portainer на books VDS** — management plane, recreate ломает access. Same exclusion as vds-kzntsv.
-- **Не trust `docker login registry.kzntsv.site`** на books VDS — НЕ настроен. Кэшированные images сейчас спасают, но force-pull для private images → auth fail. Setup login если planned image-rotation.
+- **Не выключать Windows ES** (`docker stop elasticsearch` на windows host) без user'ского явного решения — это decommission step, отдельная задача.
+- **Не редактировать `/opt/books/<svc>/config/default.json`** напрямую без `cp <file>.bak-<date>` перед — bind-mounted, change live для приложения; восстановление только через .bak копии.
+- **Не удалять `reindex.remote.whitelist`** из target ES env в Portainer stack 33 — harmless idle, может пригодиться для будущих re-imports.
+- **Не push без grant** — auto-push grant был session-only (Rule 4). Next session start = ask-mode again.
+- **Не migrate traefik / portainer** на books VDS — management plane, recreate ломает access.
+- **Не trust `https://elasticsearch.kzntsv.site/`** без basicAuth — endpoint behind traefik middleware `elasticsearch-auth`, user `books`. Pwd живёт в `/opt/books/api/config/default.json` (не в pass entry yet).
 
 ## Memory updates за сессию
 
-- **Учить:** на books VDS **yum dead** (CentOS 7 EOL 2024-06, mirrors retired). Static binary install — единственный путь для package install (e.g. `curl -fsSL https://github.com/jqlang/jq/releases/.../jq-linux-amd64 -o /usr/local/bin/jq`).
-- **Учить:** Portainer single instance `portainer.kzntsv.site` управляет 2 endpoint'ами — `1` = books VDS, `6` = stostayer (другой клиент, не наш, **не расширять scope сюда**).
-- **Учить:** Portainer migration ladder lowest→highest blast radius works на books VDS — `proxy-chain → imgproxy → minio → mongo → books-db → elasticsearch`. Books-api + books-task-runner reconnect transparently через mongo/books-db recreate windows (~13-15s). ES cluster yellow normal для single-node.
-- **Учить:** `imgproxy` (2 containers, serves `imgproxy.kzntsv.site`, S3 backend → minio) был **missing** из entity wiki до Phase 0 probe — lesson: при entity wiki bootstrap на legacy hosts обязательно `ls /usr/docker/` cross-check vs `docker ps`.
-- **Reference:** migration pattern source — [[../wiki/concepts/portainer-stack-management-vds]] (vds-kzntsv); books VDS application + diffs — [[../wiki/concepts/portainer-stack-management-books-vds]].
-- **Reference:** `scripts/books-vds-portainer-migration/{migrate.sh,README.md}` — adapter for future re-use (e.g. если появится ещё SSH-compose стек на books VDS).
+- **Учить:** `pass show books-vds/full-env` НЕ содержит ES basicAuth pwd. Pwd живёт в самих consumer configs `/opt/books/<svc>/config/default.json` (key `elasticsearch.auth.password`). Source basicAuth hash = target basicAuth hash → один password покрывает оба endpoint'а.
+- **Учить:** `path.repo` + `reindex.remote.whitelist` — это **static** ES settings, требуют env vars + container restart. НЕ показываются в `_cluster/settings?include_defaults=true` (только dynamic settings). Verify через `docker inspect <container> --format '{{range .Config.Env}}{{println .}}{{end}}'`.
+- **Учить:** ES `_reindex` from remote НЕ поддерживает `slices` (manual или auto). Single-threaded scroll → bulk. Для 700mb / 820k docs over public network — ~5 мин. Для 105k docs products (мелкие 549 bytes/doc) — ~25 мин (per-doc HTTP overhead). artmone 2621 docs — 25s (sync).
+- **Учить:** ES `_search` `total` capped по умолчанию `track_total_hits=10000` в 7.x. Для accurate count — `?track_total_hits=true` или просто `/<index>/_count`.
+- **Учить:** Portainer-managed stack env update через `PUT /api/stacks/<id>?endpointId=1` с body `{stackFileContent, env, prune, pullImage}`. Recreate происходит автоматически. Verify по `docker inspect ... StartedAt` change.
+- **Учить:** `python3` НЕ работает в git-bash на windows (запускает Windows Store launcher → exit 49, печатает "Python"). `python` — works (PATH'е есть Python 3.13). При heredoc + `2>&1` всё иногда падает silently — лучше прямой `python -c "..."` или `python <<'PY'`.
+- **Учить:** Windows PS `Invoke-WebRequest` НЕ имеет `-SkipCertificateCheck` в 5.1. PS 7 имеет. На этой машине — PS 5.1, поэтому для self-signed/auth checks использовать curl через bash.
+- **Reference:** task chronology — [[.tasks/migrate-elasticsearch-to-books-vds]] § Closure note (rollback recipe + decommission notes).
+- **Reference:** books VDS entity wiki [[.wiki/entities/books-vds]] обновлён 2026-05-25 — stack 33 теперь reflects env additions + ES indices populated note в Making-of-history.

.tasks/STATUS.md

@@ -1,5 +1,9 @@
 # Admin Task Board
-_Updated: 2026-05-25 (`books-vds-stacks-to-portainer` 🟢 closed — 6 стеков (proxy-chain, imgproxy, minio, mongo, books-db, elasticsearch) migrated SSH-compose → Portainer-managed на books VDS. Ids 28-33. Adapter script + wiki concept committed. Все 6 smoke green в одну сессию.)_
+_Updated: 2026-05-25 (`migrate-elasticsearch-to-books-vds` 🟢 closed — 3 indices (artmone/epz/products) с Windows host → books VDS ES via reindex-from-remote. 928k docs total, count parity 100%, per-doc byte-match. Consumers switched (books-api + books-task-runner + books-job-scheduler). Source ES остаётся running per user — rollback ready.)_
+
+## 🟢 [migrate-elasticsearch-to-books-vds] — closed 2026-05-25 — ES indices с Windows (elasticold.kzntsv.site, 7.10.1) → books VDS (elasticsearch.kzntsv.site, 7.10.0). 3 indices, 928616 docs. Reindex-from-remote через `reindex.remote.whitelist` env, добавленный в Portainer stack 33. Pre-migration snapshot в kreknin repo как rollback. Consumer configs sed'd, 3 containers restarted, smoke green. Source НЕ выключен.
+
+См. [migrate-elasticsearch-to-books-vds.md](migrate-elasticsearch-to-books-vds.md) § Closure note.
 
 ## 🟢 [books-vds-stacks-to-portainer] — closed 2026-05-25 — 6 SSH-compose стеков на books VDS (89.253.255.133) мигрированы в Portainer-managed (endpoint 1, https://portainer.kzntsv.site). Skip traefik + portainer (management plane). Adapter `scripts/books-vds-portainer-migration/migrate.sh`. Wiki: [portainer-stack-management-books-vds.md](../.wiki/concepts/portainer-stack-management-books-vds.md). Backup pipeline bind paths preserved (verified by inspection, full run pending next 06:00 MSK).
 

.tasks/migrate-elasticsearch-to-books-vds.md

@@ -0,0 +1,117 @@
+# migrate-elasticsearch-to-books-vds
+
+## Goal
+
+Переехать ES индексы с Windows host (`elasticold.kzntsv.site`, ES 7.10.1) на books VDS (`elasticsearch.kzntsv.site`, ES 7.10.0, single-node behind traefik basicAuth). Переключить books-api / books-task-runner / books-job-scheduler через `default.json` на новый endpoint. Source ES оставить running как rollback (не выключать).
+
+## Approach
+
+**Reindex-from-remote** (не snapshot/restore).
+
+Why:
+- Source = Windows docker, нет `path.repo` configured + нет bind для snapshot dir → snapshot подход требует restart source = consumer downtime.
+- Target = books VDS Portainer stack 33, modify env через Portainer API = recreate target only, consumers unaffected (они пока на source).
+- Same Lucene 8.7.0 → docs reindex transparently через `_reindex` API.
+- 730mb total → ~5 min over public network.
+
+Trade-off: reindex использует target's dynamic mapping. Pre-create target indices с source mappings/settings (snapshot уже сохранён в `.scratch/`).
+
+## Source/target inventory
+
+**Source (Windows, `localhost:9200`, behind `elasticold.kzntsv.site` traefik basicAuth):**
+- ES 7.10.1, Lucene 8.7.0, `discovery.type=single-node`
+- 3 indices:
+  - `artmone` 2621 docs, 1.4mb
+  - `epz` 820604 docs, 699.7mb
+  - `products` 105922 docs, 26.6mb
+- Compose: `C:\Users\vitya\projects\docker\diskstation\elasticsearch\docker-compose.yml`
+- Bind: `./data:/usr/share/elasticsearch/data`. **No snapshot path.**
+- basicAuth user `books`, hash `$apr1$vyxr1l5z$fpxHmNBfAx8cHrQTje4Fx/`
+
+**Target (books VDS 89.253.255.133, Portainer stack 33 `elasticsearch`, behind `elasticsearch.kzntsv.site`):**
+- ES 7.10.0, Lucene 8.7.0, single-node, yellow (replica unassigned — expected)
+- 1 index: `read_me` (4.8kb placeholder)
+- `path.repo=/snapshots`, kreknin fs repo registered
+- Bind: `/usr/docker/elasticsearch/{data,snapshots}` → `/usr/share/elasticsearch/{data,/snapshots}`
+- basicAuth user `books`, **same hash как source** → same password
+
+**Consumers (all on books VDS, configs bind-mounted `/opt/books/<svc>/config/default.json`):**
+- `books-api` (stack 23): port 3021, Nitro
+- `books-task-runner` (stack 22, sibling of job-scheduler+mongo)
+- `books-job-scheduler` (stack 22)
+
+Все три имеют block:
+```json
+"elasticsearch": {
+    "node": "https://elasticold.kzntsv.site/",
+    "auth": { "username": "books", "password": "<same>" }
+}
+```
+
+## Key files
+
+- `C:\Users\vitya\projects\docker\diskstation\elasticsearch\docker-compose.yml` — source ES compose (no edit planned)
+- `C:\Users\vitya\projects\.admin\.scratch\source-mappings.json` — frozen source mappings 2026-05-25
+- `C:\Users\vitya\projects\.admin\.scratch\source-settings.json` — frozen source settings 2026-05-25
+- books VDS `/opt/books/api/config/default.json:27` — consumer config (idem task-runner, job-scheduler)
+- books VDS Portainer stack 33 — target ES compose (modify env via Portainer API)
+- books VDS `/usr/docker/elasticsearch/snapshots/` — kreknin fs repo, pre-migration backup target
+
+## Execution plan
+
+1. **Pre-migration ES snapshot books VDS** (kreknin repo, `pre-migration-<date>` snapshot name) — instant rollback for target.
+2. **Save source mappings/settings** — done (`.scratch/source-{mappings,settings}.json`).
+3. **Modify target ES stack** via Portainer API: add env `reindex.remote.whitelist=elasticold.kzntsv.site:443`. Recreate stack (~10s target downtime; consumers unaffected — они на source).
+4. **Pre-create target indices** с source mappings/settings (number_of_replicas=0 для single-node clean green).
+5. **POST `/_reindex`** per index, wait_for_completion=false, slices=auto. Poll task status.
+6. **Verify counts**: target `_cat/indices` doc.counts == source. Verify random doc fetch by `_id` matches.
+7. **Update consumer configs**: replace `elasticold.kzntsv.site` → `elasticsearch.kzntsv.site` в `/opt/books/{api,task-runner,job-scheduler}/config/default.json` (sed in-place).
+8. **Bounce consumers** через `docker restart books-api books-task-runner books-job-scheduler` (mongo / db не трогаем — bind-mount уже видит новый config).
+9. **Smoke**: tail `docker logs --since 60s books-api books-task-runner books-job-scheduler` → нет ES connection errors. Trigger known query → response from target.
+10. **Don't disable source** (per user) — Windows ES + elasticold traefik route остаётся live для rollback.
+
+## Decisions log
+
+- 2026-05-25: подход reindex-from-remote, не snapshot/restore. Reason: source compose не имеет `path.repo` configured + bind для snapshot — это потребовало бы restart source (= consumer downtime). Target compose в Portainer, restart target не задевает consumers (они пока на source). Same Lucene 8.7.0 → reindex transparent.
+- 2026-05-25: pre-create target indices с frozen source mappings — иначе reindex использовал бы dynamic mapping, possibly losing exact analyzer config. Mappings/settings уже извлечены в `.scratch/`.
+- 2026-05-25: consumer config update через `sed -i` (bind-mounted files), restart through `docker restart` (не Portainer stack recreate) — mongo / db в той же stack id 22 не должны bounce.
+
+## Open questions
+
+- [x] `number_of_replicas`: source = 1 (yellow на single-node). Target тоже single-node → set to 0 на pre-create для green. Source оставить как есть. **Resolved: set 0 на target pre-create.**
+
+## Completed steps
+
+- [x] Source ES probed: 3 indices, 730mb total, ES 7.10.1
+- [x] Target ES probed: ES 7.10.0, 1 placeholder index `read_me`, no collisions
+- [x] Source mappings/settings frozen в `.scratch/`
+- [x] basicAuth password identified из consumer config (same on both sides)
+- [x] Containers inventory: 3 consumer containers, all on books VDS
+- [x] Pre-migration ES snapshot `pre-migration-2026-05-25` в kreknin repo (read_me only, SUCCESS — rollback point)
+- [x] Target ES compose updated через Portainer API: `reindex.remote.whitelist=elasticold.kzntsv.site:443`. Container restart 07:51:39 UTC. Yellow cluster post-restart (replica unassigned = expected).
+- [x] 3 target indices pre-created с frozen source mappings/settings, `number_of_replicas=0` → green.
+- [x] Reindex from remote: artmone (2621/2621, 25s), epz (820604/820604, ~5m async), products (105922/105922, ~25m async).
+- [x] Doc count parity src=dst для всех 3 indices.
+- [x] Per-doc sample fetch: byte-match _id=29359 (products) + _id=22023500 (epz).
+- [x] Consumer configs sed in-place: `elasticold` → `elasticsearch`, .bak-pre-es-migration-2026-05-25 saved для rollback.
+- [x] Restart 3 containers: books-api running, books-task-runner healthy, books-job-scheduler healthy.
+- [x] Direct internal smoke (`docker exec books-api node -e ...`): API sees `https://elasticsearch.kzntsv.site/`, `_cluster/health` 200 OK.
+- [x] Post-migration source traefik access log = 0 elasticold hits за 5 min.
+
+## Closure note (2026-05-25)
+
+Migration **complete**. 3 indices (artmone/epz/products, 820k+108k+2.6k docs, 850mb total target storage) on books VDS ES live. Consumers switched (books-api + books-task-runner + books-job-scheduler).
+
+**Source НЕ disabled** (per user requirement) — Windows ES + elasticold.kzntsv.site traefik route остаются live для emergency rollback. Кешированный data на windows host остаётся в одном экземпляре до отдельного decommission решения.
+
+**Rollback**: `ssh root@89.253.255.133 'for f in /opt/books/{api,task-runner,job-scheduler}/config/default.json; do cp "$f.bak-pre-es-migration-2026-05-25" "$f"; done; docker restart books-api books-task-runner books-job-scheduler'`.
+
+**Future ops**: при decommission Windows ES — `docker stop elasticsearch` на windows + remove traefik label `elasticold.kzntsv.site` route. `path.repo` / kreknin snapshot pre-migration-2026-05-25 уже не несёт практически данных (один read_me index), может быть удалён вместе с next backup retention rotation.
+
+## Notes
+
+- Source basicAuth password в `/opt/books/api/config/default.json` (consumer config). Не дублировать в эту таску.
+- `books-job-scheduler-mongo` — другая БД, не задевается.
+- Rollback path: `sed -i 's/elasticsearch\.kzntsv\.site/elasticold.kzntsv.site/g' /opt/books/*/config/default.json` + restart consumers.
+
+<!-- created-by: vitya / 2026-05-25 / trigger: ES consolidation на books VDS, source-host decommission preparation -->

.wiki/entities/books-vds.md

@@ -1,9 +1,9 @@
 ---
 title: Books VDS (client server — 89.253.255.133)
 type: entity
-tags: [hardware, vds, books, host4g, backup, portainer]
+tags: [hardware, vds, books, host4g, backup, portainer, elasticsearch]
 related: [[kreknin-synology]], [[vds-kzntsv]], [[../concepts/portainer-stack-management-books-vds]]
-updated: 2026-05-25
+updated: 2026-05-25 (ES migration from Windows host)
 ---
 
 # Books VDS
@@ -50,7 +50,7 @@ Post-migration 2026-05-25 (см. [[../concepts/portainer-stack-management-books-
 | 30 | `minio` | minio | `/usr/docker/minio/data` | S3 blob store, `minio.kzntsv.site` |
 | 31 | `mongo` | mongo | `/usr/docker/mongo/data/{db,configdb}` | shared MongoDB 4.2 |
 | 32 | `books-db` | books-db | `/usr/docker/books-db/data` | MariaDB latest, books application DB |
-| 33 | `elasticsearch` | elasticsearch | `/usr/docker/elasticsearch/{data,snapshots}` | ES 7.10.0 + path.repo для backup |
+| 33 | `elasticsearch` | elasticsearch | `/usr/docker/elasticsearch/{data,snapshots}` | ES 7.10.0 + `path.repo=/snapshots` для backup + `reindex.remote.whitelist=elasticold.kzntsv.site:443` (added 2026-05-25 для migration) |
 
 ### SSH-managed (management plane — Portainer migration skipped)
 
@@ -111,3 +111,4 @@ Auth-зоны DNS — в reg.ru под `kzntsv.site`.
 - Workstation SSH key `id_ed25519_books_ops` уже был задеплоен под root до этой сессии (видимо, прошлый bootstrap забыт-без-документации).
 - Hybrid stack-management (Portainer + SSH-compose) — pre-existing, не наш design. Migrated 6 SSH-compose стеков → Portainer 2026-05-25 (`books-vds-stacks-to-portainer` 🟢). `traefik` + `portainer` остаются SSH-managed (management plane).
 - `imgproxy` (2 containers, `imgproxy.kzntsv.site`) был missing из этого entity wiki до Phase 0 probe 2026-05-25 — fixed in flight.
+- ES indices populated 2026-05-25 — migration с Windows host (`elasticold.kzntsv.site`, ES 7.10.1) → books VDS via reindex-from-remote. 3 indices: `artmone` (2621), `epz` (820604), `products` (105922). Consumers (books-api + books-task-runner + books-job-scheduler) switched `default.json:elasticsearch.node` с elasticold на elasticsearch.kzntsv.site. Source ES остаётся running (rollback). См. [[../../tasks/migrate-elasticsearch-to-books-vds]] § Closure note.