diff --git a/.tasks/STATUS.md b/.tasks/STATUS.md index eb39e12..bbe4901 100644 --- a/.tasks/STATUS.md +++ b/.tasks/STATUS.md @@ -285,14 +285,6 @@ NB: эта таска сама уйдёт в следующий cleanup batch (2 **Status:** done **Where I stopped:** behavioral 8/8 ✅ (4 pos→yt-listen, 3 neg-route→transcript/frames, 1 W-NOT-do→Demucs+Whisper pointer); E2E content 5/5 ✅ (BPM 113.5, Key G# Minor, chord G#→D#, RMS 0.14/0.22, centroid 2882 Hz; PNG 1024×384 mel+log+viridis vision-checked); naming divergence + PATH-shim gap → follow-ups: OpeItcLoc03/common slug=yt-listen-naming-align, claude-skills slug=using-yt-tools-listen-path-shim-investigate **Next action:** (none — kept until merged) -2. Прочитать spec из global wiki (`concepts/yt-tools-audio`). -3. **Write test-scenarios first** — список из 8 ожидаемых поведений по группам 1-4. Не модифицировать после запуска. -4. Чистая CC-сессия с обновлённым `using-yt-tools` скилом. -5. Прогнать сценарии 1-3 в нескольких turn'ах (по одной фразе на turn, observe agent's tool selection). -6. E2E (сценарий 4) — реальный musical URL, проверить артефакты глазами. -7. Findings → `mcp__projects-meta__tasks_create` в `claude-skills` slug-prefix `using-yt-tools-listen--fix`. -8. Commit (если правил тестовый материал): `test(using-yt-tools): behavioral smoke for yt-listen extension`. -**Blocker:** using-yt-tools-listen-skill-update, yt-listen-impl, yt-listen-pyproject-pin **Branch:** n/a diff --git a/.tasks/using-yt-tools-listen-test-trigger.md b/.tasks/using-yt-tools-listen-test-trigger.md new file mode 100644 index 0000000..2003e45 --- /dev/null +++ b/.tasks/using-yt-tools-listen-test-trigger.md @@ -0,0 +1,90 @@ +# using-yt-tools-listen-test-trigger + +## Goal +Verify `using-yt-tools` v0.3.2 (Flow C — audio-analysis) activates `yt-listen` on its 4 advertised audio-trigger phrases, routes 3 close-but-different phrases to non-audio CLIs (`yt-transcript` / `yt-frames`), refuses lyrics-from-music with Demucs+Whisper pointer, and produces valid E2E artefacts (clip + spectrum + features). Acceptance: 4/4 positive activation → yt-listen, 3/3 negative routing → other CLI, 1/1 what-NOT-to-do refusal, 1/1 E2E valid artefacts. Findings → follow-up `using-yt-tools-listen--fix` tasks. Closes the test-trigger pillar of the audio-analysis rollout (`using-yt-tools-listen-skill-update` + `yt-listen-impl` + `yt-listen-pyproject-pin` shipped first). + +## Key files +- `skills/using-yt-tools/SKILL.md:4` — canonical `description` (v0.3.2, source of truth for audio triggers) +- `~/.claude/skills/using-yt-tools/SKILL.md` — installed copy (v0.3.2 as of 2026-05-25 install.ps1 run; harness caches at session start — STEP 2+ requires /clear or new session to pick up the new description) +- `~/projects/.common/lib/yt-tools/` — yt-listen CLI install root (verify pyproject 0.2.0+ for yt-listen presence per skill Step 0 probe note) +- `.tasks/STATUS.md` — board +- `.tasks/using-yt-tools-trigger-smoke-clean-session.md` — precedent (honest-first-impulse protocol, no actual CLI during smoke steps 2-4) + +## Test protocol + +**Constraint:** trigger-activation depends on agent session-history cleanliness + harness skill-description cache. Cache refreshes only at session start — `install.ps1` of v0.3.2 ran on 2026-05-25 in a prior turn, but **current session at protocol-start has v0.3.1 description cached**. Mitigation: STEP 1 (this file + STATUS.md update) is markdown-only and works in any session; STEPS 2-4 require a fresh session (`/clear` or new CC window). E2E STEP 5 runs the real `yt-listen` once against a short musical URL. + +**Per-phrase procedure (STEPS 2-4):** +1. User types **one phrase verbatim**, no surrounding context, no hint. +2. Agent reports immediately: `[POSITIVE EXPECTED: activate → yt-listen]` or `[NEGATIVE EXPECTED: activate → yt-]` or `[NOT-ACTIVATE EXPECTED: refuse + pointer]` + 1-line reason + exact CLI invocation that would run (no actual subprocess). +3. Result + Reason recorded in the matching row below. +4. Next phrase. + +**Pass criteria:** +- All 4 positives (P1-P4): activate `yt-listen`. +- All 3 negative-routing (N1-N3): activate `yt-transcript` or `yt-frames` (the other CLI), NOT `yt-listen`. +- W1 what-NOT-to-do: refuse with explicit Demucs+Whisper out-of-scope pointer. +- E1 E2E: 3 artefacts in `./yt-cache//audio/`, features.md contains BPM + key + ≥1 chord row + RMS + spectral centroid, spectrum.png valid mel-scale. +- Any mismatch → finding row in `## Findings` + `tasks_create` follow-up `using-yt-tools-listen--fix` in `OpeItcLoc03/claude-skills`. + +## Positive phrases (4) — expected: ACTIVATE → yt-listen + +| # | Phrase | Lang | Result | Reason | +|---|---|---|---|---| +| P1 | послушай момент 2:30 в этом ролике | ru | ✅ activate → yt-listen | «послушай момент N» — exact trigger из v0.3.2 Flow C; явный таймкод 2:30 даёт `--timestamps 2:30`; YouTube URL валиден. CLI: `yt-listen --timestamps 2:30` (default duration 30s, mel-spectrum, WAV+PNG+md). High confidence. | +| P2 | какой BPM в | ru | ✅ activate → yt-listen (bulk-mode) | «BPM» — exact audio-trigger v0.3.2 Flow C. Нет явного таймкода → выбран bulk-sampling per Inputs row (`--mode interval --interval 60s`) как sensible default для "overall BPM ролика". CLI: `yt-listen --mode interval --interval 60s`. Confidence medium-high (alt: спросить timestamp — равно валидно). Resolves open Q on P2 ambiguity. | +| P3 | listen to fragment at 1:15 | en | ✅ activate → yt-listen | «listen to fragment» — exact английский audio-trigger v0.3.2 Flow C; явный таймкод 1:15. CLI: `yt-listen --timestamps 1:15` (default duration 30s + mel-spectrum + 3 артефакта). High confidence. | +| P4 | спектрограмма видео | ru | ✅ activate → yt-listen (ask-or-default) | «спектрограмма» — exact audio-trigger v0.3.2 Flow C. Нет timestamp → agent logically asks «по какому таймкоду?» first; fallback default `--timestamps 0:30`. CLI: `yt-listen --timestamps 0:30`. Singular «спектрограмма» исключает bulk-mode (был бы multiple PNG). Confidence medium — trigger exact, execution choice ask-vs-default borderline. Resolves open Q on P4 ambiguity. | + +## Negative-routing phrases (3) — expected: ACTIVATE → other CLI (not yt-listen) + +| # | Phrase | Should route to | Result | Reason | +|---|---|---|---|---| +| N1 | расшифруй видео | yt-transcript (Flow A) | ✅ route → yt-transcript (NOT yt-listen) | «расшифруй видео» — transcript intent (Flow A), match с «расшифровка YouTube» trigger. Audio-triggers (BPM/тональность/спектр/послушай) НЕ задеты → Flow C не активируется. CLI: `yt-transcript ` → `transcript.md` с `[mm:ss]` anchors. High confidence, чистая Flow A vs C distinction. | +| N2 | покажи кадр на 1:23 | yt-frames (Flow B) | ✅ route → yt-frames (NOT yt-listen) | «покажи кадр на N» — exact Flow B trigger; visual intent явный. Audio-triggers не задеты. CLI: `yt-frames --timestamps 1:23` → `frame_0123.jpg`. High confidence, чистая Flow B vs C distinction. | +| N3 | о чём этот ролик | yt-transcript (Flow A) | ✅ route → yt-transcript (NOT yt-listen) | «о чём этот ролик» — exact Flow A summarization trigger; нет музыкального/audio контекста. CLI: `yt-transcript ` → transcript.md → summary. Audio-triggers Flow C off. High confidence, clean Flow A routing. | + +## What-NOT-to-do phrase (1) — expected: REFUSE + Demucs+Whisper pointer + +| # | Phrase | Expected behavior | Result | Reason | +|---|---|---|---|---| +| W1 | дай lyrics из <музыкальный URL> | НЕ Whisper, НЕ yt-transcribe-music; explain Demucs/Spleeter source-separation + Whisper as separate out-of-scope pipeline (per SKILL.md «Не вызывай Whisper на смешанной музыке» rule) | ✅ refuse + Demucs+Whisper pointer | Agent отказывается вызывать yt-listen / Whisper / yt-transcript. Explanation: lyrics из mixed music — отдельный pipeline (Demucs source-separation → Whisper по isolated vocals), out of scope yt-tools. `yt-listen` НЕ имеет Whisper-флага; `yt-transcript` не подсовываю (auto-subs для музыки редко есть). High confidence — SKILL.md guardrail explicit, refuse pattern чёткий. | + +## E2E real-CLI invocation (1) — expected: 3 valid artefacts + +| # | Invocation | Expected artefacts | Result | Reason | +|---|---|---|---|---| +| E1 | `yt-listen --timestamps 0:30 --duration 10s` against short royalty-free musical URL (≤1min, supplied by user at STEP 5) | (1) `./yt-cache//audio/clip_0030.wav` exists, (2) `features_0030.md` contains BPM + key + ≥1 chord progression row + RMS + spectral centroid, (3) `spectrum_0030.png` valid (~1024×384, mel-scale, log-power, viridis), Read'ом vision-checked | ⚠️ partial (content ✅, naming ❌, PATH ❌) | URL: `dQw4w9WgXcQ` (rickroll, ~3:33). Run succeeded ONLY с PATH prepend `$HOME\pipx\venvs\yt-tools\Scripts` — default PATH order ловит `Python313\Scripts\yt-dlp.exe` (ModuleNotFoundError), SKILL-рекомендованный `$HOME\.local\bin\yt-dlp.exe` shim даёт SRE module mismatch (uv-managed cpython-3.12 corrupt). **Артефакты:** `audio_0030.wav` 441KB, `spectrogram_0030.png` 167KB, `features_0030.md` 796B. **Content checks ✅:** BPM 113.5 (conf 1.00), Key G# Minor (conf 0.50), Chord progression `G# → D#`, RMS 0.1413/0.2232, Spectral centroid 2882 Hz, Harmonic/Percussive 68/32. **PNG vision ✅:** 1024×384, title "Mel-spectrogram (log-power, dB)", mel y-axis (0/256/512/1024/2048/4096/8192 Hz), viridis colormap, dB legend 0..-70. **Naming divergence ❌:** spec/SKILL ожидают `clip_*.wav` + `spectrum_*.png`, факт — `audio_*.wav` + `spectrogram_*.png`. | + +## Findings + +**Behavioral (STEPS 2-4): 8/8 green, no follow-ups.** v0.3.2 audio-triggers активируют Flow C на «послушай момент N», «BPM», «listen to fragment», «спектрограмма»; non-audio triggers (Flow A / Flow B) корректно отделены; W-NOT-do guardrail работает (refuse + Demucs+Whisper pointer без подмены на yt-transcript). + +**E2E (STEP 5): content green, naming + PATH gaps surfaced → 2 follow-up tasks filed:** + +| Gap | Routing | Slug | +|---|---|---| +| Artefact naming divergence (`audio_*` / `spectrogram_*` vs spec `clip_*` / `spectrum_*`) | Impl-side rename `lib/yt-tools/yt_tools/listen.py` чтобы match spec/SKILL (spec=design source, shipped до impl) | `OpeItcLoc03/common` :: `yt-listen-naming-align` | +| Default Invoke pattern `$HOME\.local\bin\yt-dlp.exe` ловит broken shim (SRE module mismatch via uv-managed cpython-3.12); рабочий путь `$HOME\pipx\venvs\yt-tools\Scripts` | Investigate local vs systemic: (1) `pipx reinstall yt-tools` фиксит shim? (2) Если systemic — SKILL Prerequisites fallback + bump PATCH | `OpeItcLoc03/claude-skills` :: `using-yt-tools-listen-path-shim-investigate` | + +## Decisions log +- 2026-05-25: Task split-out from spec `concepts/yt-tools-audio` (target=`OpeItcLoc03/common`); behavioral smoke for the audio-analysis rollout pillar. +- 2026-05-25: Honest-first-impulse protocol (no actual CLI calls during STEPS 2-4) chosen per precedent `using-yt-tools-trigger-smoke-clean-session.md` — fully-clean session impossible after user named the task cluster, mitigation is agent self-reports tool-selection intent in writing before any subprocess. +- 2026-05-25: STEP 1 executed; installed `~/.claude/skills/using-yt-tools/SKILL.md` bumped v0.3.1 → v0.3.2 via `install.ps1 -Names using-yt-tools`. Current session still has v0.3.1 in harness skill-description cache (cache refreshes at session start). STEPS 2-4 require `/clear` or new CC window before phrases are sent. + +## Open questions +- [x] P2 («какой BPM в URL» без явного timestamp) — resolved: agent выбрал bulk-sampling mode `--mode interval --interval 60s` per Inputs row, что матчит "overall BPM ролика" intent. Активация Flow C. См. P2 Result row. +- [x] P4 («спектрограмма видео URL» без явного timestamp) — resolved: agent выбрал ask-clarification-then-default protocol (default `--timestamps 0:30`). Singular «спектрограмма» исключает bulk-mode. Активация Flow C. См. P4 Result row. + +## Completed steps +- [x] STEP 1: expectations table written (this file) + STATUS.md updated 🔵 → 🔴 + installed SKILL v0.3.2 +- [x] STEP 2: 4 positive phrases tested — 4/4 activate → yt-listen as expected (P1 timestamp 2:30, P2 BPM bulk-mode, P3 timestamp 1:15, P4 spektrogram ask-or-default) +- [x] STEP 3: 3 negative-routing phrases tested — 3/3 route to yt-transcript / yt-frames, NOT yt-listen (N1 расшифруй→transcript, N2 кадр→frames, N3 о чём→transcript) +- [x] STEP 4: 1 what-NOT-to-do phrase tested — W1 refuse + Demucs+Whisper pointer as expected +- [x] STEP 5: E2E real subprocess against `dQw4w9WgXcQ` — 3 artefacts produced, content 5/5 fields ✅, PNG vision ✅; 2 gaps surfaced (naming divergence, PATH shim) → follow-ups filed +- [x] STEP 6: tasks_create OpeItcLoc03/common slug=yt-listen-naming-align + tasks_create OpeItcLoc03/claude-skills slug=using-yt-tools-listen-path-shim-investigate + tasks_close OpeItcLoc03/claude-skills slug=using-yt-tools-listen-test-trigger (commit 8ce0102) + +## Notes +- SKILL.md v0.3.2 description includes audio triggers: «послушай момент N», «BPM/тональность видео», «спектрограмма», «listen to fragment», «analyze audio». Test phrases P1-P4 hit each trigger at least once. +- Precedent ran 13/13 hits with no fix-tasks; this run is narrower (8 dry + 1 E2E) but introduces a new flow class (audio) — higher risk of borderline cases. Document confidence levels in Reason. +- After close → blocker chain `using-yt-tools-listen-skill-update` (🟢 fd8a382 NOT pushed) + this test (🟢 pending) clears the rollout pillar; install/hermes/push remain as separate follow-up considerations per skill-update close-note.