diff --git a/.gitignore b/.gitignore index ebb4284..1fe0f9c 100644 --- a/.gitignore +++ b/.gitignore @@ -19,7 +19,7 @@ env/ .ruff_cache/ # Build output -dist/ +# (dist/ is intentionally NOT ignored — we commit built .skill archives) build/ out/ target/ diff --git a/.tasks/STATUS.md b/.tasks/STATUS.md index 7512f41..5bd3d6f 100644 --- a/.tasks/STATUS.md +++ b/.tasks/STATUS.md @@ -1,5 +1,22 @@ # Task Board _Updated: 2026-04-28_ - +## Done + +### bootstrap-skills-migration +- [x] Bootstrap repo (git, .wiki, .tasks, CLAUDE.md) +- [x] Brainstorm and document repo layout (`.wiki/source/repo-layout.md`) +- [x] Copy all 12 personal skills from `~/.claude/skills/` to `skills/` +- [x] Write `scripts/build.sh`, `scripts/build.ps1`, `scripts/install.sh` +- [x] Document ZIP/PowerShell gotcha (`.wiki/source/build-notes.md`) +- [x] Build initial `dist/*.skill` archives (forward-slash entry paths verified) +- [x] Update root README with usage +- [x] Smoke-test `install.sh` to a temp dir +- [x] Remove legacy `project-bootstrap.skill` from repo root (replaced by `dist/`) + +## Backlog + +- Resolve `compress` vs `caveman-compress` duplication (look at both, decide if one should go). +- Consider a PowerShell-native `install.ps1` (current `install.sh` already works in git-bash, so low priority). +- Revisit grouping/manifest if skill count grows past ~30. +- Decide whether to write a smoke-test that round-trips `skills//` → `dist/.skill` → unpack → diff (would catch archive-shape regressions). diff --git a/.wiki/SUMMARY.md b/.wiki/SUMMARY.md index 810d962..c7b028b 100644 --- a/.wiki/SUMMARY.md +++ b/.wiki/SUMMARY.md @@ -5,4 +5,5 @@ The agent updates this file whenever source/ changes. ## Pages - +- [repo-layout.md](source/repo-layout.md) — how the repo is organized: `skills/` sources, `dist/` builds, `scripts/` build & install +- [build-notes.md](source/build-notes.md) — why `build.ps1` exists, ZIP layout, PowerShell 5.1 backslash gotcha diff --git a/.wiki/source/build-notes.md b/.wiki/source/build-notes.md new file mode 100644 index 0000000..653054b --- /dev/null +++ b/.wiki/source/build-notes.md @@ -0,0 +1,36 @@ +# Build Notes + +Practical gotchas around building `.skill` archives. + +## Why `build.ps1` exists alongside `build.sh` + +The user is on Windows; the install target may be Linux/macOS. So archives must be portable. + +**Windows PowerShell 5.1's `Compress-Archive` (v1.0.1.0) writes ZIP entries with backslashes** (e.g. `caveman\SKILL.md`). Linux/macOS `unzip` then either fails or extracts the literal name as a single file. Verified: archives produced by `Compress-Archive` listed entries like `caveman\SKILL.md`. + +Workarounds considered: +- Install `zip.exe` → adds a Windows dep we don't want. +- PowerShell 7's `Compress-Archive` is fixed → can't assume PS7 is installed. +- **Use .NET `System.IO.Compression.ZipArchive` directly** ← chosen. Walks files, adds entries with `path -replace '\\','/'`. Spec-compliant ZIPs everywhere. + +`build.ps1` does that. `build.sh` prefers `zip` (one-liner, fast on *nix), and on Windows-without-`zip` it falls back to invoking `build.ps1` via `powershell.exe`. + +## ZIP layout the loader expects + +Looking at the canonical `~/.claude/skills//` layout, the `.skill` archive should contain a single top-level directory `/` with `SKILL.md` plus optional `assets/`, `references/`, etc. inside. + +`zip -qr dist/.skill ` (run from `skills/`) does this naturally. The .NET path adds an explicit `/` prefix to each entry. + +## Extracting `.skill` files + +A `.skill` file is just a renamed `.zip`. To extract on Windows: + +```powershell +Copy-Item project-bootstrap.skill project-bootstrap.zip +Expand-Archive -Path project-bootstrap.zip -DestinationPath ~/.claude/skills/ +Remove-Item project-bootstrap.zip +``` + +On *nix: `unzip dist/project-bootstrap.skill -d ~/.claude/skills/`. + +(Our `install.sh` skips this entirely — it copies from `skills/` directly. Extraction-from-archive is only needed when you don't have the source tree, e.g. when sharing a single `.skill` file with someone else.) diff --git a/.wiki/source/repo-layout.md b/.wiki/source/repo-layout.md new file mode 100644 index 0000000..996fb73 --- /dev/null +++ b/.wiki/source/repo-layout.md @@ -0,0 +1,53 @@ +# Repo Layout & Skill Workflow + +_Decision: 2026-04-28. Approved by Vitya in chat._ + +## Goal + +Single-source workshop for personal Claude skills. Edit here, build `.skill` archives here, install to `~/.claude/skills/` here. Repo must roll out cleanly to multiple machines. + +## Layout + +``` +claude-skills/ +├── skills/ ← editable sources (source of truth) +│ ├── caveman/ +│ ├── caveman-commit/ +│ ├── caveman-compress/ +│ ├── caveman-help/ +│ ├── caveman-review/ +│ ├── compress/ +│ ├── find-skills/ +│ ├── project-bootstrap/ +│ ├── task-status-wiki/ +│ ├── using-context7/ +│ ├── using-markitdown/ +│ └── wiki-maintainer/ +├── dist/ ← built .skill archives (committed) +├── scripts/ +│ ├── build.sh ← skills// → dist/.skill +│ └── install.sh ← skills// → ~/.claude/skills// +├── .wiki/, .tasks/, CLAUDE.md, README.md, .gitignore +``` + +## Build & Install Model + +- **Source of truth:** `skills//` — a regular folder with `SKILL.md` + optional `assets/`, `references/`, etc. Mirrors the on-disk format Claude expects in `~/.claude/skills/`. +- **Build:** `scripts/build.sh [name...]` zips each `skills//` into `dist/.skill`. No args → builds all. +- **Install:** `scripts/install.sh [name...]` copies (not symlinks) each `skills//` into `~/.claude/skills//`, replacing any existing folder. No args → installs all. +- **`.skill` archives are committed** to `dist/`. New machine: clone → run `install.sh` and you're set. No build dependency on the install path. +- **Cross-platform:** scripts are bash (works in git-bash on Windows + native on Linux/Mac). PowerShell variant added if/when needed. + +## Conventions + +- One folder per skill, name = skill name. +- Skill content authority: this repo. `~/.claude/skills/` is downstream. +- `dist/*.skill` regenerated by build script — never edited by hand. +- Editing flow: edit `skills//`, run `install.sh ` to push to live, run `build.sh ` to refresh archive, commit. + +## Out of scope (for now) + +- Versioning of individual skills (no `version` field, no changelog per skill). +- Manifest/registry file — flat folder is enough at 12 skills; revisit at ~30+. +- CI / automated builds — manual-only. +- Plugin skills (superpowers, frontend-design, etc.) — managed by plugin system, not mirrored here. diff --git a/README.md b/README.md index ff9b064..53b80d7 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,64 @@ # claude-skills +Совместное хранилище и мастерская навыков для Claude. + ## About - + +Здесь мы с Claude вместе разрабатываем, отлаживаем и храним скиллы: + +- **`skills/`** — редактируемые исходники (markdown + ассеты), source of truth +- **`dist/`** — собранные `.skill` архивы, закоммичены в репо +- **`scripts/`** — утилиты: `build.sh` (исходник → `.skill`), `install.sh` (исходник → `~/.claude/skills/`) +- **`.wiki/`**, **`.tasks/`** — рабочие материалы и доска задач ## Quick start - + +### Установить скиллы на новом компе + +```bash +git clone claude-skills +cd claude-skills +bash scripts/install.sh # копирует все skills/* в ~/.claude/skills/ +# или конкретные: +bash scripts/install.sh caveman wiki-maintainer +``` + +Цель установки можно переопределить переменной `CLAUDE_SKILLS_DIR=/path bash scripts/install.sh`. + +### Отредактировать скилл + +```bash +# 1. Правишь skills//SKILL.md (или ассеты) +# 2. Раскатываешь правки в живые скиллы: +bash scripts/install.sh +# 3. Пересобираешь архив (опц., но удобно делать перед коммитом): +bash scripts/build.sh +``` + +### Собрать `.skill` архивы + +```bash +bash scripts/build.sh # все +bash scripts/build.sh caveman # один +``` + +`build.sh` использует `zip` если он есть (Linux/macOS) или делегирует в +`scripts/build.ps1` через PowerShell (Windows без `zip`). На Windows +можно сразу запускать `powershell scripts/build.ps1`. + +## Layout + +``` +claude-skills/ +├── skills/ ← исходники (по папке на скилл) +├── dist/ ← .skill архивы (коммитятся) +├── scripts/ +│ ├── build.sh / build.ps1 +│ └── install.sh +├── .wiki/ ← дизайн-доки, заметки +├── .tasks/ ← STATUS.md +├── CLAUDE.md +└── README.md +``` + +Подробнее про принципы и решения — в [`.wiki/source/repo-layout.md`](.wiki/source/repo-layout.md). diff --git a/dist/caveman-commit.skill b/dist/caveman-commit.skill new file mode 100644 index 0000000..2a6a609 Binary files /dev/null and b/dist/caveman-commit.skill differ diff --git a/dist/caveman-compress.skill b/dist/caveman-compress.skill new file mode 100644 index 0000000..f607319 Binary files /dev/null and b/dist/caveman-compress.skill differ diff --git a/dist/caveman-help.skill b/dist/caveman-help.skill new file mode 100644 index 0000000..e8a7076 Binary files /dev/null and b/dist/caveman-help.skill differ diff --git a/dist/caveman-review.skill b/dist/caveman-review.skill new file mode 100644 index 0000000..3acc6f5 Binary files /dev/null and b/dist/caveman-review.skill differ diff --git a/dist/caveman.skill b/dist/caveman.skill new file mode 100644 index 0000000..0e71dff Binary files /dev/null and b/dist/caveman.skill differ diff --git a/dist/compress.skill b/dist/compress.skill new file mode 100644 index 0000000..c62c92e Binary files /dev/null and b/dist/compress.skill differ diff --git a/dist/find-skills.skill b/dist/find-skills.skill new file mode 100644 index 0000000..fd062a8 Binary files /dev/null and b/dist/find-skills.skill differ diff --git a/dist/project-bootstrap.skill b/dist/project-bootstrap.skill new file mode 100644 index 0000000..ade028d Binary files /dev/null and b/dist/project-bootstrap.skill differ diff --git a/dist/task-status-wiki.skill b/dist/task-status-wiki.skill new file mode 100644 index 0000000..9531555 Binary files /dev/null and b/dist/task-status-wiki.skill differ diff --git a/dist/using-context7.skill b/dist/using-context7.skill new file mode 100644 index 0000000..988207b Binary files /dev/null and b/dist/using-context7.skill differ diff --git a/dist/using-markitdown.skill b/dist/using-markitdown.skill new file mode 100644 index 0000000..5f8f2ef Binary files /dev/null and b/dist/using-markitdown.skill differ diff --git a/dist/wiki-maintainer.skill b/dist/wiki-maintainer.skill new file mode 100644 index 0000000..0e82793 Binary files /dev/null and b/dist/wiki-maintainer.skill differ diff --git a/scripts/build.ps1 b/scripts/build.ps1 new file mode 100644 index 0000000..395e94e --- /dev/null +++ b/scripts/build.ps1 @@ -0,0 +1,71 @@ +# Build .skill archives from skills// into dist/.skill +# Usage: build.ps1 [-Names ,] (no args = all) +# +# Uses .NET System.IO.Compression.ZipArchive directly to produce +# spec-compliant ZIPs with forward-slash entry names (Windows PowerShell 5.1's +# Compress-Archive writes backslashes, which break extraction on Linux/Mac). + +[CmdletBinding()] +param( + [string[]]$Names = @() +) + +$ErrorActionPreference = 'Stop' + +$root = Split-Path -Parent $PSScriptRoot +$src = Join-Path $root 'skills' +$dist = Join-Path $root 'dist' + +if (-not (Test-Path $dist)) { New-Item -Type Directory -Path $dist | Out-Null } + +if ($Names.Count -eq 0) { + $Names = Get-ChildItem -Path $src -Directory | Sort-Object Name | Select-Object -ExpandProperty Name +} + +Add-Type -AssemblyName System.IO.Compression +Add-Type -AssemblyName System.IO.Compression.FileSystem + +function New-SkillArchive { + param( + [string]$SkillName, + [string]$SourceDir, + [string]$OutPath + ) + if (Test-Path $OutPath) { Remove-Item $OutPath -Force } + + $sourceFull = (Resolve-Path $SourceDir).ProviderPath.TrimEnd('\','/') + $archive = [System.IO.Compression.ZipFile]::Open( + $OutPath, + [System.IO.Compression.ZipArchiveMode]::Create + ) + try { + $files = Get-ChildItem -Path $sourceFull -Recurse -File + foreach ($file in $files) { + $rel = $file.FullName.Substring($sourceFull.Length + 1) -replace '\\','/' + $entryName = "$SkillName/$rel" + [System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile( + $archive, + $file.FullName, + $entryName, + [System.IO.Compression.CompressionLevel]::Optimal + ) | Out-Null + } + } finally { + $archive.Dispose() + } +} + +foreach ($name in $Names) { + $srcDir = Join-Path $src $name + if (-not (Test-Path $srcDir -PathType Container)) { + Write-Warning "skip: $name (not found in skills/)" + continue + } + if (-not (Test-Path (Join-Path $srcDir 'SKILL.md'))) { + Write-Warning "skip: $name (missing SKILL.md)" + continue + } + $out = Join-Path $dist "$name.skill" + New-SkillArchive -SkillName $name -SourceDir $srcDir -OutPath $out + Write-Host "built: dist/$name.skill" +} diff --git a/scripts/build.sh b/scripts/build.sh new file mode 100644 index 0000000..9c4486f --- /dev/null +++ b/scripts/build.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# Build .skill archives from skills// into dist/.skill +# Usage: build.sh [name...] (no args = all) +# +# Uses `zip` if available; otherwise delegates to scripts/build.ps1 +# (so the script works on Linux, macOS, and Windows-with-git-bash without +# requiring `zip` on Windows). +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +SRC="$ROOT/skills" +DIST="$ROOT/dist" + +if command -v zip >/dev/null 2>&1; then + mkdir -p "$DIST" + if [ "$#" -eq 0 ]; then + mapfile -t names < <(find "$SRC" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' | sort) + else + names=("$@") + fi + for name in "${names[@]}"; do + src_dir="$SRC/$name" + if [ ! -d "$src_dir" ]; then + echo "skip: $name (not found in skills/)" >&2 + continue + fi + if [ ! -f "$src_dir/SKILL.md" ]; then + echo "skip: $name (missing SKILL.md)" >&2 + continue + fi + out="$DIST/$name.skill" + rm -f "$out" + ( cd "$SRC" && zip -qr "$out" "$name" ) + echo "built: dist/$name.skill" + done +elif command -v powershell.exe >/dev/null 2>&1; then + # Windows fallback: delegate to build.ps1 (proper ZIP via .NET API). + ps1="$SCRIPT_DIR/build.ps1" + if [ "$#" -eq 0 ]; then + powershell.exe -NoProfile -ExecutionPolicy Bypass -File "$(cygpath -w "$ps1" 2>/dev/null || echo "$ps1")" + else + # Build a comma-separated PS argument list. + joined=$(printf ",'%s'" "$@") + joined="${joined:1}" + powershell.exe -NoProfile -ExecutionPolicy Bypass -File "$(cygpath -w "$ps1" 2>/dev/null || echo "$ps1")" -Names "$joined" + fi +else + echo "error: need either 'zip' (Linux/macOS) or PowerShell (Windows) to build .skill archives" >&2 + exit 1 +fi diff --git a/scripts/install.sh b/scripts/install.sh new file mode 100644 index 0000000..5f0af82 --- /dev/null +++ b/scripts/install.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +# Install skills// into ~/.claude/skills// (or $CLAUDE_SKILLS_DIR) +# Usage: install.sh [name...] (no args = all) +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +SRC="$ROOT/skills" +TARGET="${CLAUDE_SKILLS_DIR:-$HOME/.claude/skills}" + +if [ "$#" -eq 0 ]; then + mapfile -t names < <(find "$SRC" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' | sort) +else + names=("$@") +fi + +mkdir -p "$TARGET" + +for name in "${names[@]}"; do + src_dir="$SRC/$name" + dst_dir="$TARGET/$name" + if [ ! -d "$src_dir" ]; then + echo "skip: $name (not found in skills/)" >&2 + continue + fi + if [ ! -f "$src_dir/SKILL.md" ]; then + echo "skip: $name (missing SKILL.md)" >&2 + continue + fi + rm -rf "$dst_dir" + cp -R "$src_dir" "$dst_dir" + echo "installed: $name → $dst_dir" +done diff --git a/skills/caveman-commit/SKILL.md b/skills/caveman-commit/SKILL.md new file mode 100644 index 0000000..729318c --- /dev/null +++ b/skills/caveman-commit/SKILL.md @@ -0,0 +1,65 @@ +--- +name: caveman-commit +description: > + Ultra-compressed commit message generator. Cuts noise from commit messages while preserving + intent and reasoning. Conventional Commits format. Subject ≤50 chars, body only when "why" + isn't obvious. Use when user says "write a commit", "commit message", "generate commit", + "/commit", or invokes /caveman-commit. Auto-triggers when staging changes. +--- + +Write commit messages terse and exact. Conventional Commits format. No fluff. Why over what. + +## Rules + +**Subject line:** +- `(): ` — `` optional +- Types: `feat`, `fix`, `refactor`, `perf`, `docs`, `test`, `chore`, `build`, `ci`, `style`, `revert` +- Imperative mood: "add", "fix", "remove" — not "added", "adds", "adding" +- ≤50 chars when possible, hard cap 72 +- No trailing period +- Match project convention for capitalization after the colon + +**Body (only if needed):** +- Skip entirely when subject is self-explanatory +- Add body only for: non-obvious *why*, breaking changes, migration notes, linked issues +- Wrap at 72 chars +- Bullets `-` not `*` +- Reference issues/PRs at end: `Closes #42`, `Refs #17` + +**What NEVER goes in:** +- "This commit does X", "I", "we", "now", "currently" — the diff says what +- "As requested by..." — use Co-authored-by trailer +- "Generated with Claude Code" or any AI attribution +- Emoji (unless project convention requires) +- Restating the file name when scope already says it + +## Examples + +Diff: new endpoint for user profile with body explaining the why +- ❌ "feat: add a new endpoint to get user profile information from the database" +- ✅ + ``` + feat(api): add GET /users/:id/profile + + Mobile client needs profile data without the full user payload + to reduce LTE bandwidth on cold-launch screens. + + Closes #128 + ``` + +Diff: breaking API change +- ✅ + ``` + feat(api)!: rename /v1/orders to /v1/checkout + + BREAKING CHANGE: clients on /v1/orders must migrate to /v1/checkout + before 2026-06-01. Old route returns 410 after that date. + ``` + +## Auto-Clarity + +Always include body for: breaking changes, security fixes, data migrations, anything reverting a prior commit. Never compress these into subject-only — future debuggers need the context. + +## Boundaries + +Only generates the commit message. Does not run `git commit`, does not stage files, does not amend. Output the message as a code block ready to paste. "stop caveman-commit" or "normal mode": revert to verbose commit style. \ No newline at end of file diff --git a/skills/caveman-compress/README.md b/skills/caveman-compress/README.md new file mode 100644 index 0000000..7c0e8ba --- /dev/null +++ b/skills/caveman-compress/README.md @@ -0,0 +1,163 @@ +

+ +

+ +

caveman-compress

+ +

+ shrink memory file. save token every session. +

+ +--- + +A Claude Code skill that compresses your project memory files (`CLAUDE.md`, todos, preferences) into caveman format — so every session loads fewer tokens automatically. + +Claude read `CLAUDE.md` on every session start. If file big, cost big. Caveman make file small. Cost go down forever. + +## What It Do + +``` +/caveman:compress CLAUDE.md +``` + +``` +CLAUDE.md ← compressed (Claude reads this — fewer tokens every session) +CLAUDE.original.md ← human-readable backup (you edit this) +``` + +Original never lost. You can read and edit `.original.md`. Run skill again to re-compress after edits. + +## Benchmarks + +Real results on real project files: + +| File | Original | Compressed | Saved | +|------|----------:|----------:|------:| +| `claude-md-preferences.md` | 706 | 285 | **59.6%** | +| `project-notes.md` | 1145 | 535 | **53.3%** | +| `claude-md-project.md` | 1122 | 636 | **43.3%** | +| `todo-list.md` | 627 | 388 | **38.1%** | +| `mixed-with-code.md` | 888 | 560 | **36.9%** | +| **Average** | **898** | **481** | **46%** | + +All validations passed ✅ — headings, code blocks, URLs, file paths preserved exactly. + +## Before / After + + + + + + +
+ +### 📄 Original (706 tokens) + +> "I strongly prefer TypeScript with strict mode enabled for all new code. Please don't use `any` type unless there's genuinely no way around it, and if you do, leave a comment explaining the reasoning. I find that taking the time to properly type things catches a lot of bugs before they ever make it to runtime." + + + +### 🪨 Caveman (285 tokens) + +> "Prefer TypeScript strict mode always. No `any` unless unavoidable — comment why if used. Proper types catch bugs early." + +
+ +**Same instructions. 60% fewer tokens. Every. Single. Session.** + +## Security + +`caveman-compress` is flagged as Snyk High Risk due to subprocess and file I/O patterns detected by static analysis. This is a false positive — see [SECURITY.md](./SECURITY.md) for a full explanation of what the skill does and does not do. + +## Install + +Compress is built in with the `caveman` plugin. Install `caveman` once, then use `/caveman:compress`. + +If you need local files, the compress skill lives at: + +```bash +caveman-compress/ +``` + +**Requires:** Python 3.10+ + +## Usage + +``` +/caveman:compress +``` + +Examples: +``` +/caveman:compress CLAUDE.md +/caveman:compress docs/preferences.md +/caveman:compress todos.md +``` + +### What files work + +| Type | Compress? | +|------|-----------| +| `.md`, `.txt`, `.rst` | ✅ Yes | +| Extensionless natural language | ✅ Yes | +| `.py`, `.js`, `.ts`, `.json`, `.yaml` | ❌ Skip (code/config) | +| `*.original.md` | ❌ Skip (backup files) | + +## How It Work + +``` +/caveman:compress CLAUDE.md + ↓ +detect file type (no tokens) + ↓ +Claude compresses (tokens — one call) + ↓ +validate output (no tokens) + checks: headings, code blocks, URLs, file paths, bullets + ↓ +if errors: Claude fixes cherry-picked issues only (tokens — targeted fix) + does NOT recompress — only patches broken parts + ↓ +retry up to 2 times + ↓ +write compressed → CLAUDE.md +write original → CLAUDE.original.md +``` + +Only two things use tokens: initial compression + targeted fix if validation fails. Everything else is local Python. + +## What Is Preserved + +Caveman compress natural language. It never touch: + +- Code blocks (` ``` ` fenced or indented) +- Inline code (`` `backtick content` ``) +- URLs and links +- File paths (`/src/components/...`) +- Commands (`npm install`, `git commit`) +- Technical terms, library names, API names +- Headings (exact text preserved) +- Tables (structure preserved, cell text compressed) +- Dates, version numbers, numeric values + +## Why This Matter + +`CLAUDE.md` loads on **every session start**. A 1000-token project memory file costs tokens every single time you open a project. Over 100 sessions that's 100,000 tokens of overhead — just for context you already wrote. + +Caveman cut that by ~46% on average. Same instructions. Same accuracy. Less waste. + +``` +┌────────────────────────────────────────────┐ +│ TOKEN SAVINGS PER FILE █████ 46% │ +│ SESSIONS THAT BENEFIT ██████████ 100% │ +│ INFORMATION PRESERVED ██████████ 100% │ +│ SETUP TIME █ 1x │ +└────────────────────────────────────────────┘ +``` + +## Part of Caveman + +This skill is part of the [caveman](https://github.com/JuliusBrussee/caveman) toolkit — making Claude use fewer tokens without losing accuracy. + +- **caveman** — make Claude *speak* like caveman (cuts response tokens ~65%) +- **caveman-compress** — make Claude *read* less (cuts context tokens ~46%) diff --git a/skills/caveman-compress/SECURITY.md b/skills/caveman-compress/SECURITY.md new file mode 100644 index 0000000..693108c --- /dev/null +++ b/skills/caveman-compress/SECURITY.md @@ -0,0 +1,31 @@ +# Security + +## Snyk High Risk Rating + +`caveman-compress` receives a Snyk High Risk rating due to static analysis heuristics. This document explains what the skill does and does not do. + +### What triggers the rating + +1. **subprocess usage**: The skill calls the `claude` CLI via `subprocess.run()` as a fallback when `ANTHROPIC_API_KEY` is not set. The subprocess call uses a fixed argument list — no shell interpolation occurs. User file content is passed via stdin, not as a shell argument. + +2. **File read/write**: The skill reads the file the user explicitly points it at, compresses it, and writes the result back to the same path. A `.original.md` backup is saved alongside it. No files outside the user-specified path are read or written. + +### What the skill does NOT do + +- Does not execute user file content as code +- Does not make network requests except to Anthropic's API (via SDK or CLI) +- Does not access files outside the path the user provides +- Does not use shell=True or string interpolation in subprocess calls +- Does not collect or transmit any data beyond the file being compressed + +### Auth behavior + +If `ANTHROPIC_API_KEY` is set, the skill uses the Anthropic Python SDK directly (no subprocess). If not set, it falls back to the `claude` CLI, which uses the user's existing Claude desktop authentication. + +### File size limit + +Files larger than 500KB are rejected before any API call is made. + +### Reporting a vulnerability + +If you believe you've found a genuine security issue, please open a GitHub issue with the label `security`. diff --git a/skills/caveman-compress/SKILL.md b/skills/caveman-compress/SKILL.md new file mode 100644 index 0000000..7b3e3aa --- /dev/null +++ b/skills/caveman-compress/SKILL.md @@ -0,0 +1,111 @@ +--- +name: caveman-compress +description: > + Compress natural language memory files (CLAUDE.md, todos, preferences) into caveman format + to save input tokens. Preserves all technical substance, code, URLs, and structure. + Compressed version overwrites the original file. Human-readable backup saved as FILE.original.md. + Trigger: /caveman:compress or "compress memory file" +--- + +# Caveman Compress + +## Purpose + +Compress natural language files (CLAUDE.md, todos, preferences) into caveman-speak to reduce input tokens. Compressed version overwrites original. Human-readable backup saved as `.original.md`. + +## Trigger + +`/caveman:compress ` or when user asks to compress a memory file. + +## Process + +1. The compression scripts live in `caveman-compress/scripts/` (adjacent to this SKILL.md). If the path is not immediately available, search for `caveman-compress/scripts/__main__.py`. + +2. Run: + +cd caveman-compress && python3 -m scripts + +3. The CLI will: +- detect file type (no tokens) +- call Claude to compress +- validate output (no tokens) +- if errors: cherry-pick fix with Claude (targeted fixes only, no recompression) +- retry up to 2 times +- if still failing after 2 retries: report error to user, leave original file untouched + +4. Return result to user + +## Compression Rules + +### Remove +- Articles: a, an, the +- Filler: just, really, basically, actually, simply, essentially, generally +- Pleasantries: "sure", "certainly", "of course", "happy to", "I'd recommend" +- Hedging: "it might be worth", "you could consider", "it would be good to" +- Redundant phrasing: "in order to" → "to", "make sure to" → "ensure", "the reason is because" → "because" +- Connective fluff: "however", "furthermore", "additionally", "in addition" + +### Preserve EXACTLY (never modify) +- Code blocks (fenced ``` and indented) +- Inline code (`backtick content`) +- URLs and links (full URLs, markdown links) +- File paths (`/src/components/...`, `./config.yaml`) +- Commands (`npm install`, `git commit`, `docker build`) +- Technical terms (library names, API names, protocols, algorithms) +- Proper nouns (project names, people, companies) +- Dates, version numbers, numeric values +- Environment variables (`$HOME`, `NODE_ENV`) + +### Preserve Structure +- All markdown headings (keep exact heading text, compress body below) +- Bullet point hierarchy (keep nesting level) +- Numbered lists (keep numbering) +- Tables (compress cell text, keep structure) +- Frontmatter/YAML headers in markdown files + +### Compress +- Use short synonyms: "big" not "extensive", "fix" not "implement a solution for", "use" not "utilize" +- Fragments OK: "Run tests before commit" not "You should always run tests before committing" +- Drop "you should", "make sure to", "remember to" — just state the action +- Merge redundant bullets that say the same thing differently +- Keep one example where multiple examples show the same pattern + +CRITICAL RULE: +Anything inside ``` ... ``` must be copied EXACTLY. +Do not: +- remove comments +- remove spacing +- reorder lines +- shorten commands +- simplify anything + +Inline code (`...`) must be preserved EXACTLY. +Do not modify anything inside backticks. + +If file contains code blocks: +- Treat code blocks as read-only regions +- Only compress text outside them +- Do not merge sections around code + +## Pattern + +Original: +> You should always make sure to run the test suite before pushing any changes to the main branch. This is important because it helps catch bugs early and prevents broken builds from being deployed to production. + +Compressed: +> Run tests before push to main. Catch bugs early, prevent broken prod deploys. + +Original: +> The application uses a microservices architecture with the following components. The API gateway handles all incoming requests and routes them to the appropriate service. The authentication service is responsible for managing user sessions and JWT tokens. + +Compressed: +> Microservices architecture. API gateway route all requests to services. Auth service manage user sessions + JWT tokens. + +## Boundaries + +- ONLY compress natural language files (.md, .txt, extensionless) +- NEVER modify: .py, .js, .ts, .json, .yaml, .yml, .toml, .env, .lock, .css, .html, .xml, .sql, .sh +- If file has mixed content (prose + code), compress ONLY the prose sections +- If unsure whether something is code or prose, leave it unchanged +- Original file is backed up as FILE.original.md before overwriting +- Never compress FILE.original.md (skip it) diff --git a/skills/caveman-compress/scripts/__init__.py b/skills/caveman-compress/scripts/__init__.py new file mode 100644 index 0000000..16b8c53 --- /dev/null +++ b/skills/caveman-compress/scripts/__init__.py @@ -0,0 +1,9 @@ +"""Caveman compress scripts. + +This package provides tools to compress natural language markdown files +into caveman format to save input tokens. +""" + +__all__ = ["cli", "compress", "detect", "validate"] + +__version__ = "1.0.0" diff --git a/skills/caveman-compress/scripts/__main__.py b/skills/caveman-compress/scripts/__main__.py new file mode 100644 index 0000000..4e28416 --- /dev/null +++ b/skills/caveman-compress/scripts/__main__.py @@ -0,0 +1,3 @@ +from .cli import main + +main() diff --git a/skills/caveman-compress/scripts/benchmark.py b/skills/caveman-compress/scripts/benchmark.py new file mode 100644 index 0000000..eac927d --- /dev/null +++ b/skills/caveman-compress/scripts/benchmark.py @@ -0,0 +1,78 @@ +#!/usr/bin/env python3 +from pathlib import Path +import sys + +# Support both direct execution and module import +try: + from .validate import validate +except ImportError: + sys.path.insert(0, str(Path(__file__).parent)) + from validate import validate + +try: + import tiktoken + _enc = tiktoken.get_encoding("o200k_base") +except ImportError: + _enc = None + + +def count_tokens(text): + if _enc is None: + return len(text.split()) # fallback: word count + return len(_enc.encode(text)) + + +def benchmark_pair(orig_path: Path, comp_path: Path): + orig_text = orig_path.read_text() + comp_text = comp_path.read_text() + + orig_tokens = count_tokens(orig_text) + comp_tokens = count_tokens(comp_text) + saved = 100 * (orig_tokens - comp_tokens) / orig_tokens if orig_tokens > 0 else 0.0 + result = validate(orig_path, comp_path) + + return (comp_path.name, orig_tokens, comp_tokens, saved, result.is_valid) + + +def print_table(rows): + print("\n| File | Original | Compressed | Saved % | Valid |") + print("|------|----------|------------|---------|-------|") + for r in rows: + print(f"| {r[0]} | {r[1]} | {r[2]} | {r[3]:.1f}% | {'✅' if r[4] else '❌'} |") + + +def main(): + # Direct file pair: python3 benchmark.py original.md compressed.md + if len(sys.argv) == 3: + orig = Path(sys.argv[1]).resolve() + comp = Path(sys.argv[2]).resolve() + if not orig.exists(): + print(f"❌ Not found: {orig}") + sys.exit(1) + if not comp.exists(): + print(f"❌ Not found: {comp}") + sys.exit(1) + print_table([benchmark_pair(orig, comp)]) + return + + # Glob mode: repo_root/tests/caveman-compress/ + tests_dir = Path(__file__).parent.parent.parent / "tests" / "caveman-compress" + if not tests_dir.exists(): + print(f"❌ Tests dir not found: {tests_dir}") + sys.exit(1) + + rows = [] + for orig in sorted(tests_dir.glob("*.original.md")): + comp = orig.with_name(orig.stem.removesuffix(".original") + ".md") + if comp.exists(): + rows.append(benchmark_pair(orig, comp)) + + if not rows: + print("No compressed file pairs found.") + return + + print_table(rows) + + +if __name__ == "__main__": + main() diff --git a/skills/caveman-compress/scripts/cli.py b/skills/caveman-compress/scripts/cli.py new file mode 100644 index 0000000..428fd86 --- /dev/null +++ b/skills/caveman-compress/scripts/cli.py @@ -0,0 +1,73 @@ +#!/usr/bin/env python3 +""" +Caveman Compress CLI + +Usage: + caveman +""" + +import sys +from pathlib import Path + +from .compress import compress_file +from .detect import detect_file_type, should_compress + + +def print_usage(): + print("Usage: caveman ") + + +def main(): + if len(sys.argv) != 2: + print_usage() + sys.exit(1) + + filepath = Path(sys.argv[1]) + + # Check file exists + if not filepath.exists(): + print(f"❌ File not found: {filepath}") + sys.exit(1) + + if not filepath.is_file(): + print(f"❌ Not a file: {filepath}") + sys.exit(1) + + filepath = filepath.resolve() + + # Detect file type + file_type = detect_file_type(filepath) + + print(f"Detected: {file_type}") + + # Check if compressible + if not should_compress(filepath): + print("Skipping: file is not natural language (code/config)") + sys.exit(0) + + print("Starting caveman compression...\n") + + try: + success = compress_file(filepath) + + if success: + print("\nCompression completed successfully") + backup_path = filepath.with_name(filepath.stem + ".original.md") + print(f"Compressed: {filepath}") + print(f"Original: {backup_path}") + sys.exit(0) + else: + print("\n❌ Compression failed after retries") + sys.exit(2) + + except KeyboardInterrupt: + print("\nInterrupted by user") + sys.exit(130) + + except Exception as e: + print(f"\n❌ Error: {e}") + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/skills/caveman-compress/scripts/compress.py b/skills/caveman-compress/scripts/compress.py new file mode 100644 index 0000000..70aeb40 --- /dev/null +++ b/skills/caveman-compress/scripts/compress.py @@ -0,0 +1,227 @@ +#!/usr/bin/env python3 +""" +Caveman Memory Compression Orchestrator + +Usage: + python scripts/compress.py +""" + +import os +import re +import subprocess +from pathlib import Path +from typing import List + +OUTER_FENCE_REGEX = re.compile( + r"\A\s*(`{3,}|~{3,})[^\n]*\n(.*)\n\1\s*\Z", re.DOTALL +) + +# Filenames and paths that almost certainly hold secrets or PII. Compressing +# them ships raw bytes to the Anthropic API — a third-party data boundary that +# developers on sensitive codebases cannot cross. detect.py already skips .env +# by extension, but credentials.md / secrets.txt / ~/.aws/credentials would +# slip through the natural-language filter. This is a hard refuse before read. +SENSITIVE_BASENAME_REGEX = re.compile( + r"(?ix)^(" + r"\.env(\..+)?" + r"|\.netrc" + r"|credentials(\..+)?" + r"|secrets?(\..+)?" + r"|passwords?(\..+)?" + r"|id_(rsa|dsa|ecdsa|ed25519)(\.pub)?" + r"|authorized_keys" + r"|known_hosts" + r"|.*\.(pem|key|p12|pfx|crt|cer|jks|keystore|asc|gpg)" + r")$" +) + +SENSITIVE_PATH_COMPONENTS = frozenset({".ssh", ".aws", ".gnupg", ".kube", ".docker"}) + +SENSITIVE_NAME_TOKENS = ( + "secret", "credential", "password", "passwd", + "apikey", "accesskey", "token", "privatekey", +) + + +def is_sensitive_path(filepath: Path) -> bool: + """Heuristic denylist for files that must never be shipped to a third-party API.""" + name = filepath.name + if SENSITIVE_BASENAME_REGEX.match(name): + return True + lowered_parts = {p.lower() for p in filepath.parts} + if lowered_parts & SENSITIVE_PATH_COMPONENTS: + return True + # Normalize separators so "api-key" and "api_key" both match "apikey". + lower = re.sub(r"[_\-\s.]", "", name.lower()) + return any(tok in lower for tok in SENSITIVE_NAME_TOKENS) + + +def strip_llm_wrapper(text: str) -> str: + """Strip outer ```markdown ... ``` fence when it wraps the entire output.""" + m = OUTER_FENCE_REGEX.match(text) + if m: + return m.group(2) + return text + +from .detect import should_compress +from .validate import validate + +MAX_RETRIES = 2 + + +# ---------- Claude Calls ---------- + + +def call_claude(prompt: str) -> str: + api_key = os.environ.get("ANTHROPIC_API_KEY") + if api_key: + try: + import anthropic + + client = anthropic.Anthropic(api_key=api_key) + msg = client.messages.create( + model=os.environ.get("CAVEMAN_MODEL", "claude-sonnet-4-5"), + max_tokens=8192, + messages=[{"role": "user", "content": prompt}], + ) + return strip_llm_wrapper(msg.content[0].text.strip()) + except ImportError: + pass # anthropic not installed, fall back to CLI + # Fallback: use claude CLI (handles desktop auth) + try: + result = subprocess.run( + ["claude", "--print"], + input=prompt, + text=True, + capture_output=True, + check=True, + ) + return strip_llm_wrapper(result.stdout.strip()) + except subprocess.CalledProcessError as e: + raise RuntimeError(f"Claude call failed:\n{e.stderr}") + + +def build_compress_prompt(original: str) -> str: + return f""" +Compress this markdown into caveman format. + +STRICT RULES: +- Do NOT modify anything inside ``` code blocks +- Do NOT modify anything inside inline backticks +- Preserve ALL URLs exactly +- Preserve ALL headings exactly +- Preserve file paths and commands +- Return ONLY the compressed markdown body — do NOT wrap the entire output in a ```markdown fence or any other fence. Inner code blocks from the original stay as-is; do not add a new outer fence around the whole file. + +Only compress natural language. + +TEXT: +{original} +""" + + +def build_fix_prompt(original: str, compressed: str, errors: List[str]) -> str: + errors_str = "\n".join(f"- {e}" for e in errors) + return f"""You are fixing a caveman-compressed markdown file. Specific validation errors were found. + +CRITICAL RULES: +- DO NOT recompress or rephrase the file +- ONLY fix the listed errors — leave everything else exactly as-is +- The ORIGINAL is provided as reference only (to restore missing content) +- Preserve caveman style in all untouched sections + +ERRORS TO FIX: +{errors_str} + +HOW TO FIX: +- Missing URL: find it in ORIGINAL, restore it exactly where it belongs in COMPRESSED +- Code block mismatch: find the exact code block in ORIGINAL, restore it in COMPRESSED +- Heading mismatch: restore the exact heading text from ORIGINAL into COMPRESSED +- Do not touch any section not mentioned in the errors + +ORIGINAL (reference only): +{original} + +COMPRESSED (fix this): +{compressed} + +Return ONLY the fixed compressed file. No explanation. +""" + + +# ---------- Core Logic ---------- + + +def compress_file(filepath: Path) -> bool: + # Resolve and validate path + filepath = filepath.resolve() + MAX_FILE_SIZE = 500_000 # 500KB + if not filepath.exists(): + raise FileNotFoundError(f"File not found: {filepath}") + if filepath.stat().st_size > MAX_FILE_SIZE: + raise ValueError(f"File too large to compress safely (max 500KB): {filepath}") + + # Refuse files that look like they contain secrets or PII. Compressing ships + # the raw bytes to the Anthropic API — a third-party boundary — so we fail + # loudly rather than silently exfiltrate credentials or keys. Override is + # intentional: the user must rename the file if the heuristic is wrong. + if is_sensitive_path(filepath): + raise ValueError( + f"Refusing to compress {filepath}: filename looks sensitive " + "(credentials, keys, secrets, or known private paths). " + "Compression sends file contents to the Anthropic API. " + "Rename the file if this is a false positive." + ) + + print(f"Processing: {filepath}") + + if not should_compress(filepath): + print("Skipping (not natural language)") + return False + + original_text = filepath.read_text(errors="ignore") + backup_path = filepath.with_name(filepath.stem + ".original.md") + + # Check if backup already exists to prevent accidental overwriting + if backup_path.exists(): + print(f"⚠️ Backup file already exists: {backup_path}") + print("The original backup may contain important content.") + print("Aborting to prevent data loss. Please remove or rename the backup file if you want to proceed.") + return False + + # Step 1: Compress + print("Compressing with Claude...") + compressed = call_claude(build_compress_prompt(original_text)) + + # Save original as backup, write compressed to original path + backup_path.write_text(original_text) + filepath.write_text(compressed) + + # Step 2: Validate + Retry + for attempt in range(MAX_RETRIES): + print(f"\nValidation attempt {attempt + 1}") + + result = validate(backup_path, filepath) + + if result.is_valid: + print("Validation passed") + break + + print("❌ Validation failed:") + for err in result.errors: + print(f" - {err}") + + if attempt == MAX_RETRIES - 1: + # Restore original on failure + filepath.write_text(original_text) + backup_path.unlink(missing_ok=True) + print("❌ Failed after retries — original restored") + return False + + print("Fixing with Claude...") + compressed = call_claude( + build_fix_prompt(original_text, compressed, result.errors) + ) + filepath.write_text(compressed) + + return True diff --git a/skills/caveman-compress/scripts/detect.py b/skills/caveman-compress/scripts/detect.py new file mode 100644 index 0000000..5f50fd3 --- /dev/null +++ b/skills/caveman-compress/scripts/detect.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python3 +"""Detect whether a file is natural language (compressible) or code/config (skip).""" + +import json +import re +from pathlib import Path + +# Extensions that are natural language and compressible +COMPRESSIBLE_EXTENSIONS = {".md", ".txt", ".markdown", ".rst"} + +# Extensions that are code/config and should be skipped +SKIP_EXTENSIONS = { + ".py", ".js", ".ts", ".tsx", ".jsx", ".json", ".yaml", ".yml", + ".toml", ".env", ".lock", ".css", ".scss", ".html", ".xml", + ".sql", ".sh", ".bash", ".zsh", ".go", ".rs", ".java", ".c", + ".cpp", ".h", ".hpp", ".rb", ".php", ".swift", ".kt", ".lua", + ".dockerfile", ".makefile", ".csv", ".ini", ".cfg", +} + +# Patterns that indicate a line is code +CODE_PATTERNS = [ + re.compile(r"^\s*(import |from .+ import |require\(|const |let |var )"), + re.compile(r"^\s*(def |class |function |async function |export )"), + re.compile(r"^\s*(if\s*\(|for\s*\(|while\s*\(|switch\s*\(|try\s*\{)"), + re.compile(r"^\s*[\}\]\);]+\s*$"), # closing braces/brackets + re.compile(r"^\s*@\w+"), # decorators/annotations + re.compile(r'^\s*"[^"]+"\s*:\s*'), # JSON-like key-value + re.compile(r"^\s*\w+\s*=\s*[{\[\(\"']"), # assignment with literal +] + + +def _is_code_line(line: str) -> bool: + """Check if a line looks like code.""" + return any(p.match(line) for p in CODE_PATTERNS) + + +def _is_json_content(text: str) -> bool: + """Check if content is valid JSON.""" + try: + json.loads(text) + return True + except (json.JSONDecodeError, ValueError): + return False + + +def _is_yaml_content(lines: list[str]) -> bool: + """Heuristic: check if content looks like YAML.""" + yaml_indicators = 0 + for line in lines[:30]: + stripped = line.strip() + if stripped.startswith("---"): + yaml_indicators += 1 + elif re.match(r"^\w[\w\s]*:\s", stripped): + yaml_indicators += 1 + elif stripped.startswith("- ") and ":" in stripped: + yaml_indicators += 1 + # If most non-empty lines look like YAML + non_empty = sum(1 for l in lines[:30] if l.strip()) + return non_empty > 0 and yaml_indicators / non_empty > 0.6 + + +def detect_file_type(filepath: Path) -> str: + """Classify a file as 'natural_language', 'code', 'config', or 'unknown'. + + Returns: + One of: 'natural_language', 'code', 'config', 'unknown' + """ + ext = filepath.suffix.lower() + + # Extension-based classification + if ext in COMPRESSIBLE_EXTENSIONS: + return "natural_language" + if ext in SKIP_EXTENSIONS: + return "code" if ext not in {".json", ".yaml", ".yml", ".toml", ".ini", ".cfg", ".env"} else "config" + + # Extensionless files (like CLAUDE.md, TODO) — check content + if not ext: + try: + text = filepath.read_text(errors="ignore") + except (OSError, PermissionError): + return "unknown" + + lines = text.splitlines()[:50] + + if _is_json_content(text[:10000]): + return "config" + if _is_yaml_content(lines): + return "config" + + code_lines = sum(1 for l in lines if l.strip() and _is_code_line(l)) + non_empty = sum(1 for l in lines if l.strip()) + if non_empty > 0 and code_lines / non_empty > 0.4: + return "code" + + return "natural_language" + + return "unknown" + + +def should_compress(filepath: Path) -> bool: + """Return True if the file is natural language and should be compressed.""" + if not filepath.is_file(): + return False + # Skip backup files + if filepath.name.endswith(".original.md"): + return False + return detect_file_type(filepath) == "natural_language" + + +if __name__ == "__main__": + import sys + + if len(sys.argv) < 2: + print("Usage: python detect.py [file2] ...") + sys.exit(1) + + for path_str in sys.argv[1:]: + p = Path(path_str).resolve() + file_type = detect_file_type(p) + compress = should_compress(p) + print(f" {p.name:30s} type={file_type:20s} compress={compress}") diff --git a/skills/caveman-compress/scripts/validate.py b/skills/caveman-compress/scripts/validate.py new file mode 100644 index 0000000..3c4d4c1 --- /dev/null +++ b/skills/caveman-compress/scripts/validate.py @@ -0,0 +1,189 @@ +#!/usr/bin/env python3 +import re +from pathlib import Path + +URL_REGEX = re.compile(r"https?://[^\s)]+") +FENCE_OPEN_REGEX = re.compile(r"^(\s{0,3})(`{3,}|~{3,})(.*)$") +HEADING_REGEX = re.compile(r"^(#{1,6})\s+(.*)", re.MULTILINE) +BULLET_REGEX = re.compile(r"^\s*[-*+]\s+", re.MULTILINE) + +# crude but effective path detection +# Requires either a path prefix (./ ../ / or drive letter) or a slash/backslash within the match +PATH_REGEX = re.compile(r"(?:\./|\.\./|/|[A-Za-z]:\\)[\w\-/\\\.]+|[\w\-\.]+[/\\][\w\-/\\\.]+") + + +class ValidationResult: + def __init__(self): + self.is_valid = True + self.errors = [] + self.warnings = [] + + def add_error(self, msg): + self.is_valid = False + self.errors.append(msg) + + def add_warning(self, msg): + self.warnings.append(msg) + + +def read_file(path: Path) -> str: + return path.read_text(errors="ignore") + + +# ---------- Extractors ---------- + + +def extract_headings(text): + return [(level, title.strip()) for level, title in HEADING_REGEX.findall(text)] + + +def extract_code_blocks(text): + """Line-based fenced code block extractor. + + Handles ``` and ~~~ fences with variable length (CommonMark: closing + fence must use same char and be at least as long as opening). Supports + nested fences (e.g. an outer 4-backtick block wrapping inner 3-backtick + content). + """ + blocks = [] + lines = text.split("\n") + i = 0 + n = len(lines) + while i < n: + m = FENCE_OPEN_REGEX.match(lines[i]) + if not m: + i += 1 + continue + fence_char = m.group(2)[0] + fence_len = len(m.group(2)) + open_line = lines[i] + block_lines = [open_line] + i += 1 + closed = False + while i < n: + close_m = FENCE_OPEN_REGEX.match(lines[i]) + if ( + close_m + and close_m.group(2)[0] == fence_char + and len(close_m.group(2)) >= fence_len + and close_m.group(3).strip() == "" + ): + block_lines.append(lines[i]) + closed = True + i += 1 + break + block_lines.append(lines[i]) + i += 1 + if closed: + blocks.append("\n".join(block_lines)) + # Unclosed fences are silently skipped — they indicate malformed markdown + # and including them would cause false-positive validation failures. + return blocks + + +def extract_urls(text): + return set(URL_REGEX.findall(text)) + + +def extract_paths(text): + return set(PATH_REGEX.findall(text)) + + +def count_bullets(text): + return len(BULLET_REGEX.findall(text)) + + +# ---------- Validators ---------- + + +def validate_headings(orig, comp, result): + h1 = extract_headings(orig) + h2 = extract_headings(comp) + + if len(h1) != len(h2): + result.add_error(f"Heading count mismatch: {len(h1)} vs {len(h2)}") + + if h1 != h2: + result.add_warning("Heading text/order changed") + + +def validate_code_blocks(orig, comp, result): + c1 = extract_code_blocks(orig) + c2 = extract_code_blocks(comp) + + if c1 != c2: + result.add_error("Code blocks not preserved exactly") + + +def validate_urls(orig, comp, result): + u1 = extract_urls(orig) + u2 = extract_urls(comp) + + if u1 != u2: + result.add_error(f"URL mismatch: lost={u1 - u2}, added={u2 - u1}") + + +def validate_paths(orig, comp, result): + p1 = extract_paths(orig) + p2 = extract_paths(comp) + + if p1 != p2: + result.add_warning(f"Path mismatch: lost={p1 - p2}, added={p2 - p1}") + + +def validate_bullets(orig, comp, result): + b1 = count_bullets(orig) + b2 = count_bullets(comp) + + if b1 == 0: + return + + diff = abs(b1 - b2) / b1 + + if diff > 0.15: + result.add_warning(f"Bullet count changed too much: {b1} -> {b2}") + + +# ---------- Main ---------- + + +def validate(original_path: Path, compressed_path: Path) -> ValidationResult: + result = ValidationResult() + + orig = read_file(original_path) + comp = read_file(compressed_path) + + validate_headings(orig, comp, result) + validate_code_blocks(orig, comp, result) + validate_urls(orig, comp, result) + validate_paths(orig, comp, result) + validate_bullets(orig, comp, result) + + return result + + +# ---------- CLI ---------- + +if __name__ == "__main__": + import sys + + if len(sys.argv) != 3: + print("Usage: python validate.py ") + sys.exit(1) + + orig = Path(sys.argv[1]).resolve() + comp = Path(sys.argv[2]).resolve() + + res = validate(orig, comp) + + print(f"\nValid: {res.is_valid}") + + if res.errors: + print("\nErrors:") + for e in res.errors: + print(f" - {e}") + + if res.warnings: + print("\nWarnings:") + for w in res.warnings: + print(f" - {w}") diff --git a/skills/caveman-help/SKILL.md b/skills/caveman-help/SKILL.md new file mode 100644 index 0000000..078e487 --- /dev/null +++ b/skills/caveman-help/SKILL.md @@ -0,0 +1,59 @@ +--- +name: caveman-help +description: > + Quick-reference card for all caveman modes, skills, and commands. + One-shot display, not a persistent mode. Trigger: /caveman-help, + "caveman help", "what caveman commands", "how do I use caveman". +--- + +# Caveman Help + +Display this reference card when invoked. One-shot — do NOT change mode, write flag files, or persist anything. Output in caveman style. + +## Modes + +| Mode | Trigger | What change | +|------|---------|-------------| +| **Lite** | `/caveman lite` | Drop filler. Keep sentence structure. | +| **Full** | `/caveman` | Drop articles, filler, pleasantries, hedging. Fragments OK. Default. | +| **Ultra** | `/caveman ultra` | Extreme compression. Bare fragments. Tables over prose. | +| **Wenyan-Lite** | `/caveman wenyan-lite` | Classical Chinese style, light compression. | +| **Wenyan-Full** | `/caveman wenyan` | Full 文言文. Maximum classical terseness. | +| **Wenyan-Ultra** | `/caveman wenyan-ultra` | Extreme. Ancient scholar on a budget. | + +Mode stick until changed or session end. + +## Skills + +| Skill | Trigger | What it do | +|-------|---------|-----------| +| **caveman-commit** | `/caveman-commit` | Terse commit messages. Conventional Commits. ≤50 char subject. | +| **caveman-review** | `/caveman-review` | One-line PR comments: `L42: bug: user null. Add guard.` | +| **caveman-compress** | `/caveman:compress ` | Compress .md files to caveman prose. Saves ~46% input tokens. | +| **caveman-help** | `/caveman-help` | This card. | + +## Deactivate + +Say "stop caveman" or "normal mode". Resume anytime with `/caveman`. + +## Configure Default Mode + +Default mode = `full`. Change it: + +**Environment variable** (highest priority): +```bash +export CAVEMAN_DEFAULT_MODE=ultra +``` + +**Config file** (`~/.config/caveman/config.json`): +```json +{ "defaultMode": "lite" } +``` + +Set `"off"` to disable auto-activation on session start. User can still activate manually with `/caveman`. + +Resolution: env var > config file > `full`. + +## More + +Full docs: https://github.com/JuliusBrussee/caveman diff --git a/skills/caveman-review/SKILL.md b/skills/caveman-review/SKILL.md new file mode 100644 index 0000000..48f4adb --- /dev/null +++ b/skills/caveman-review/SKILL.md @@ -0,0 +1,55 @@ +--- +name: caveman-review +description: > + Ultra-compressed code review comments. Cuts noise from PR feedback while preserving + the actionable signal. Each comment is one line: location, problem, fix. Use when user + says "review this PR", "code review", "review the diff", "/review", or invokes + /caveman-review. Auto-triggers when reviewing pull requests. +--- + +Write code review comments terse and actionable. One line per finding. Location, problem, fix. No throat-clearing. + +## Rules + +**Format:** `L: . .` — or `:L: ...` when reviewing multi-file diffs. + +**Severity prefix (optional, when mixed):** +- `🔴 bug:` — broken behavior, will cause incident +- `🟡 risk:` — works but fragile (race, missing null check, swallowed error) +- `🔵 nit:` — style, naming, micro-optim. Author can ignore +- `❓ q:` — genuine question, not a suggestion + +**Drop:** +- "I noticed that...", "It seems like...", "You might want to consider..." +- "This is just a suggestion but..." — use `nit:` instead +- "Great work!", "Looks good overall but..." — say it once at the top, not per comment +- Restating what the line does — the reviewer can read the diff +- Hedging ("perhaps", "maybe", "I think") — if unsure use `q:` + +**Keep:** +- Exact line numbers +- Exact symbol/function/variable names in backticks +- Concrete fix, not "consider refactoring this" +- The *why* if the fix isn't obvious from the problem statement + +## Examples + +❌ "I noticed that on line 42 you're not checking if the user object is null before accessing the email property. This could potentially cause a crash if the user is not found in the database. You might want to add a null check here." + +✅ `L42: 🔴 bug: user can be null after .find(). Add guard before .email.` + +❌ "It looks like this function is doing a lot of things and might benefit from being broken up into smaller functions for readability." + +✅ `L88-140: 🔵 nit: 50-line fn does 4 things. Extract validate/normalize/persist.` + +❌ "Have you considered what happens if the API returns a 429? I think we should probably handle that case." + +✅ `L23: 🟡 risk: no retry on 429. Wrap in withBackoff(3).` + +## Auto-Clarity + +Drop terse mode for: security findings (CVE-class bugs need full explanation + reference), architectural disagreements (need rationale, not just a one-liner), and onboarding contexts where the author is new and needs the "why". In those cases write a normal paragraph, then resume terse for the rest. + +## Boundaries + +Reviews only — does not write the code fix, does not approve/request-changes, does not run linters. Output the comment(s) ready to paste into the PR. "stop caveman-review" or "normal mode": revert to verbose review style. \ No newline at end of file diff --git a/skills/caveman/SKILL.md b/skills/caveman/SKILL.md new file mode 100644 index 0000000..2ab498b --- /dev/null +++ b/skills/caveman/SKILL.md @@ -0,0 +1,67 @@ +--- +name: caveman +description: > + Ultra-compressed communication mode. Cuts token usage ~75% by speaking like caveman + while keeping full technical accuracy. Supports intensity levels: lite, full (default), ultra, + wenyan-lite, wenyan-full, wenyan-ultra. + Use when user says "caveman mode", "talk like caveman", "use caveman", "less tokens", + "be brief", or invokes /caveman. Also auto-triggers when token efficiency is requested. +--- + +Respond terse like smart caveman. All technical substance stay. Only fluff die. + +## Persistence + +ACTIVE EVERY RESPONSE. No revert after many turns. No filler drift. Still active if unsure. Off only: "stop caveman" / "normal mode". + +Default: **full**. Switch: `/caveman lite|full|ultra`. + +## Rules + +Drop: articles (a/an/the), filler (just/really/basically/actually/simply), pleasantries (sure/certainly/of course/happy to), hedging. Fragments OK. Short synonyms (big not extensive, fix not "implement a solution for"). Technical terms exact. Code blocks unchanged. Errors quoted exact. + +Pattern: `[thing] [action] [reason]. [next step].` + +Not: "Sure! I'd be happy to help you with that. The issue you're experiencing is likely caused by..." +Yes: "Bug in auth middleware. Token expiry check use `<` not `<=`. Fix:" + +## Intensity + +| Level | What change | +|-------|------------| +| **lite** | No filler/hedging. Keep articles + full sentences. Professional but tight | +| **full** | Drop articles, fragments OK, short synonyms. Classic caveman | +| **ultra** | Abbreviate (DB/auth/config/req/res/fn/impl), strip conjunctions, arrows for causality (X → Y), one word when one word enough | +| **wenyan-lite** | Semi-classical. Drop filler/hedging but keep grammar structure, classical register | +| **wenyan-full** | Maximum classical terseness. Fully 文言文. 80-90% character reduction. Classical sentence patterns, verbs precede objects, subjects often omitted, classical particles (之/乃/為/其) | +| **wenyan-ultra** | Extreme abbreviation while keeping classical Chinese feel. Maximum compression, ultra terse | + +Example — "Why React component re-render?" +- lite: "Your component re-renders because you create a new object reference each render. Wrap it in `useMemo`." +- full: "New object ref each render. Inline object prop = new ref = re-render. Wrap in `useMemo`." +- ultra: "Inline obj prop → new ref → re-render. `useMemo`." +- wenyan-lite: "組件頻重繪,以每繪新生對象參照故。以 useMemo 包之。" +- wenyan-full: "物出新參照,致重繪。useMemo .Wrap之。" +- wenyan-ultra: "新參照→重繪。useMemo Wrap。" + +Example — "Explain database connection pooling." +- lite: "Connection pooling reuses open connections instead of creating new ones per request. Avoids repeated handshake overhead." +- full: "Pool reuse open DB connections. No new connection per request. Skip handshake overhead." +- ultra: "Pool = reuse DB conn. Skip handshake → fast under load." +- wenyan-full: "池reuse open connection。不每req新開。skip handshake overhead。" +- wenyan-ultra: "池reuse conn。skip handshake → fast。" + +## Auto-Clarity + +Drop caveman for: security warnings, irreversible action confirmations, multi-step sequences where fragment order risks misread, user asks to clarify or repeats question. Resume caveman after clear part done. + +Example — destructive op: +> **Warning:** This will permanently delete all rows in the `users` table and cannot be undone. +> ```sql +> DROP TABLE users; +> ``` +> Caveman resume. Verify backup exist first. + +## Boundaries + +Code/commits/PRs: write normal. "stop caveman" or "normal mode": revert. Level persist until changed or session end. \ No newline at end of file diff --git a/skills/compress/SKILL.md b/skills/compress/SKILL.md new file mode 100644 index 0000000..1a6652f --- /dev/null +++ b/skills/compress/SKILL.md @@ -0,0 +1,111 @@ +--- +name: compress +description: > + Compress natural language memory files (CLAUDE.md, todos, preferences) into caveman format + to save input tokens. Preserves all technical substance, code, URLs, and structure. + Compressed version overwrites the original file. Human-readable backup saved as FILE.original.md. + Trigger: /caveman:compress or "compress memory file" +--- + +# Caveman Compress + +## Purpose + +Compress natural language files (CLAUDE.md, todos, preferences) into caveman-speak to reduce input tokens. Compressed version overwrites original. Human-readable backup saved as `.original.md`. + +## Trigger + +`/caveman:compress ` or when user asks to compress a memory file. + +## Process + +1. This SKILL.md lives alongside `scripts/` in the same directory. Find that directory. + +2. Run: + +cd && python3 -m scripts + +3. The CLI will: +- detect file type (no tokens) +- call Claude to compress +- validate output (no tokens) +- if errors: cherry-pick fix with Claude (targeted fixes only, no recompression) +- retry up to 2 times +- if still failing after 2 retries: report error to user, leave original file untouched + +4. Return result to user + +## Compression Rules + +### Remove +- Articles: a, an, the +- Filler: just, really, basically, actually, simply, essentially, generally +- Pleasantries: "sure", "certainly", "of course", "happy to", "I'd recommend" +- Hedging: "it might be worth", "you could consider", "it would be good to" +- Redundant phrasing: "in order to" → "to", "make sure to" → "ensure", "the reason is because" → "because" +- Connective fluff: "however", "furthermore", "additionally", "in addition" + +### Preserve EXACTLY (never modify) +- Code blocks (fenced ``` and indented) +- Inline code (`backtick content`) +- URLs and links (full URLs, markdown links) +- File paths (`/src/components/...`, `./config.yaml`) +- Commands (`npm install`, `git commit`, `docker build`) +- Technical terms (library names, API names, protocols, algorithms) +- Proper nouns (project names, people, companies) +- Dates, version numbers, numeric values +- Environment variables (`$HOME`, `NODE_ENV`) + +### Preserve Structure +- All markdown headings (keep exact heading text, compress body below) +- Bullet point hierarchy (keep nesting level) +- Numbered lists (keep numbering) +- Tables (compress cell text, keep structure) +- Frontmatter/YAML headers in markdown files + +### Compress +- Use short synonyms: "big" not "extensive", "fix" not "implement a solution for", "use" not "utilize" +- Fragments OK: "Run tests before commit" not "You should always run tests before committing" +- Drop "you should", "make sure to", "remember to" — just state the action +- Merge redundant bullets that say the same thing differently +- Keep one example where multiple examples show the same pattern + +CRITICAL RULE: +Anything inside ``` ... ``` must be copied EXACTLY. +Do not: +- remove comments +- remove spacing +- reorder lines +- shorten commands +- simplify anything + +Inline code (`...`) must be preserved EXACTLY. +Do not modify anything inside backticks. + +If file contains code blocks: +- Treat code blocks as read-only regions +- Only compress text outside them +- Do not merge sections around code + +## Pattern + +Original: +> You should always make sure to run the test suite before pushing any changes to the main branch. This is important because it helps catch bugs early and prevents broken builds from being deployed to production. + +Compressed: +> Run tests before push to main. Catch bugs early, prevent broken prod deploys. + +Original: +> The application uses a microservices architecture with the following components. The API gateway handles all incoming requests and routes them to the appropriate service. The authentication service is responsible for managing user sessions and JWT tokens. + +Compressed: +> Microservices architecture. API gateway route all requests to services. Auth service manage user sessions + JWT tokens. + +## Boundaries + +- ONLY compress natural language files (.md, .txt, extensionless) +- NEVER modify: .py, .js, .ts, .json, .yaml, .yml, .toml, .env, .lock, .css, .html, .xml, .sql, .sh +- If file has mixed content (prose + code), compress ONLY the prose sections +- If unsure whether something is code or prose, leave it unchanged +- Original file is backed up as FILE.original.md before overwriting +- Never compress FILE.original.md (skip it) diff --git a/skills/compress/scripts/__init__.py b/skills/compress/scripts/__init__.py new file mode 100644 index 0000000..16b8c53 --- /dev/null +++ b/skills/compress/scripts/__init__.py @@ -0,0 +1,9 @@ +"""Caveman compress scripts. + +This package provides tools to compress natural language markdown files +into caveman format to save input tokens. +""" + +__all__ = ["cli", "compress", "detect", "validate"] + +__version__ = "1.0.0" diff --git a/skills/compress/scripts/__main__.py b/skills/compress/scripts/__main__.py new file mode 100644 index 0000000..4e28416 --- /dev/null +++ b/skills/compress/scripts/__main__.py @@ -0,0 +1,3 @@ +from .cli import main + +main() diff --git a/skills/compress/scripts/benchmark.py b/skills/compress/scripts/benchmark.py new file mode 100644 index 0000000..eac927d --- /dev/null +++ b/skills/compress/scripts/benchmark.py @@ -0,0 +1,78 @@ +#!/usr/bin/env python3 +from pathlib import Path +import sys + +# Support both direct execution and module import +try: + from .validate import validate +except ImportError: + sys.path.insert(0, str(Path(__file__).parent)) + from validate import validate + +try: + import tiktoken + _enc = tiktoken.get_encoding("o200k_base") +except ImportError: + _enc = None + + +def count_tokens(text): + if _enc is None: + return len(text.split()) # fallback: word count + return len(_enc.encode(text)) + + +def benchmark_pair(orig_path: Path, comp_path: Path): + orig_text = orig_path.read_text() + comp_text = comp_path.read_text() + + orig_tokens = count_tokens(orig_text) + comp_tokens = count_tokens(comp_text) + saved = 100 * (orig_tokens - comp_tokens) / orig_tokens if orig_tokens > 0 else 0.0 + result = validate(orig_path, comp_path) + + return (comp_path.name, orig_tokens, comp_tokens, saved, result.is_valid) + + +def print_table(rows): + print("\n| File | Original | Compressed | Saved % | Valid |") + print("|------|----------|------------|---------|-------|") + for r in rows: + print(f"| {r[0]} | {r[1]} | {r[2]} | {r[3]:.1f}% | {'✅' if r[4] else '❌'} |") + + +def main(): + # Direct file pair: python3 benchmark.py original.md compressed.md + if len(sys.argv) == 3: + orig = Path(sys.argv[1]).resolve() + comp = Path(sys.argv[2]).resolve() + if not orig.exists(): + print(f"❌ Not found: {orig}") + sys.exit(1) + if not comp.exists(): + print(f"❌ Not found: {comp}") + sys.exit(1) + print_table([benchmark_pair(orig, comp)]) + return + + # Glob mode: repo_root/tests/caveman-compress/ + tests_dir = Path(__file__).parent.parent.parent / "tests" / "caveman-compress" + if not tests_dir.exists(): + print(f"❌ Tests dir not found: {tests_dir}") + sys.exit(1) + + rows = [] + for orig in sorted(tests_dir.glob("*.original.md")): + comp = orig.with_name(orig.stem.removesuffix(".original") + ".md") + if comp.exists(): + rows.append(benchmark_pair(orig, comp)) + + if not rows: + print("No compressed file pairs found.") + return + + print_table(rows) + + +if __name__ == "__main__": + main() diff --git a/skills/compress/scripts/cli.py b/skills/compress/scripts/cli.py new file mode 100644 index 0000000..428fd86 --- /dev/null +++ b/skills/compress/scripts/cli.py @@ -0,0 +1,73 @@ +#!/usr/bin/env python3 +""" +Caveman Compress CLI + +Usage: + caveman +""" + +import sys +from pathlib import Path + +from .compress import compress_file +from .detect import detect_file_type, should_compress + + +def print_usage(): + print("Usage: caveman ") + + +def main(): + if len(sys.argv) != 2: + print_usage() + sys.exit(1) + + filepath = Path(sys.argv[1]) + + # Check file exists + if not filepath.exists(): + print(f"❌ File not found: {filepath}") + sys.exit(1) + + if not filepath.is_file(): + print(f"❌ Not a file: {filepath}") + sys.exit(1) + + filepath = filepath.resolve() + + # Detect file type + file_type = detect_file_type(filepath) + + print(f"Detected: {file_type}") + + # Check if compressible + if not should_compress(filepath): + print("Skipping: file is not natural language (code/config)") + sys.exit(0) + + print("Starting caveman compression...\n") + + try: + success = compress_file(filepath) + + if success: + print("\nCompression completed successfully") + backup_path = filepath.with_name(filepath.stem + ".original.md") + print(f"Compressed: {filepath}") + print(f"Original: {backup_path}") + sys.exit(0) + else: + print("\n❌ Compression failed after retries") + sys.exit(2) + + except KeyboardInterrupt: + print("\nInterrupted by user") + sys.exit(130) + + except Exception as e: + print(f"\n❌ Error: {e}") + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/skills/compress/scripts/compress.py b/skills/compress/scripts/compress.py new file mode 100644 index 0000000..70aeb40 --- /dev/null +++ b/skills/compress/scripts/compress.py @@ -0,0 +1,227 @@ +#!/usr/bin/env python3 +""" +Caveman Memory Compression Orchestrator + +Usage: + python scripts/compress.py +""" + +import os +import re +import subprocess +from pathlib import Path +from typing import List + +OUTER_FENCE_REGEX = re.compile( + r"\A\s*(`{3,}|~{3,})[^\n]*\n(.*)\n\1\s*\Z", re.DOTALL +) + +# Filenames and paths that almost certainly hold secrets or PII. Compressing +# them ships raw bytes to the Anthropic API — a third-party data boundary that +# developers on sensitive codebases cannot cross. detect.py already skips .env +# by extension, but credentials.md / secrets.txt / ~/.aws/credentials would +# slip through the natural-language filter. This is a hard refuse before read. +SENSITIVE_BASENAME_REGEX = re.compile( + r"(?ix)^(" + r"\.env(\..+)?" + r"|\.netrc" + r"|credentials(\..+)?" + r"|secrets?(\..+)?" + r"|passwords?(\..+)?" + r"|id_(rsa|dsa|ecdsa|ed25519)(\.pub)?" + r"|authorized_keys" + r"|known_hosts" + r"|.*\.(pem|key|p12|pfx|crt|cer|jks|keystore|asc|gpg)" + r")$" +) + +SENSITIVE_PATH_COMPONENTS = frozenset({".ssh", ".aws", ".gnupg", ".kube", ".docker"}) + +SENSITIVE_NAME_TOKENS = ( + "secret", "credential", "password", "passwd", + "apikey", "accesskey", "token", "privatekey", +) + + +def is_sensitive_path(filepath: Path) -> bool: + """Heuristic denylist for files that must never be shipped to a third-party API.""" + name = filepath.name + if SENSITIVE_BASENAME_REGEX.match(name): + return True + lowered_parts = {p.lower() for p in filepath.parts} + if lowered_parts & SENSITIVE_PATH_COMPONENTS: + return True + # Normalize separators so "api-key" and "api_key" both match "apikey". + lower = re.sub(r"[_\-\s.]", "", name.lower()) + return any(tok in lower for tok in SENSITIVE_NAME_TOKENS) + + +def strip_llm_wrapper(text: str) -> str: + """Strip outer ```markdown ... ``` fence when it wraps the entire output.""" + m = OUTER_FENCE_REGEX.match(text) + if m: + return m.group(2) + return text + +from .detect import should_compress +from .validate import validate + +MAX_RETRIES = 2 + + +# ---------- Claude Calls ---------- + + +def call_claude(prompt: str) -> str: + api_key = os.environ.get("ANTHROPIC_API_KEY") + if api_key: + try: + import anthropic + + client = anthropic.Anthropic(api_key=api_key) + msg = client.messages.create( + model=os.environ.get("CAVEMAN_MODEL", "claude-sonnet-4-5"), + max_tokens=8192, + messages=[{"role": "user", "content": prompt}], + ) + return strip_llm_wrapper(msg.content[0].text.strip()) + except ImportError: + pass # anthropic not installed, fall back to CLI + # Fallback: use claude CLI (handles desktop auth) + try: + result = subprocess.run( + ["claude", "--print"], + input=prompt, + text=True, + capture_output=True, + check=True, + ) + return strip_llm_wrapper(result.stdout.strip()) + except subprocess.CalledProcessError as e: + raise RuntimeError(f"Claude call failed:\n{e.stderr}") + + +def build_compress_prompt(original: str) -> str: + return f""" +Compress this markdown into caveman format. + +STRICT RULES: +- Do NOT modify anything inside ``` code blocks +- Do NOT modify anything inside inline backticks +- Preserve ALL URLs exactly +- Preserve ALL headings exactly +- Preserve file paths and commands +- Return ONLY the compressed markdown body — do NOT wrap the entire output in a ```markdown fence or any other fence. Inner code blocks from the original stay as-is; do not add a new outer fence around the whole file. + +Only compress natural language. + +TEXT: +{original} +""" + + +def build_fix_prompt(original: str, compressed: str, errors: List[str]) -> str: + errors_str = "\n".join(f"- {e}" for e in errors) + return f"""You are fixing a caveman-compressed markdown file. Specific validation errors were found. + +CRITICAL RULES: +- DO NOT recompress or rephrase the file +- ONLY fix the listed errors — leave everything else exactly as-is +- The ORIGINAL is provided as reference only (to restore missing content) +- Preserve caveman style in all untouched sections + +ERRORS TO FIX: +{errors_str} + +HOW TO FIX: +- Missing URL: find it in ORIGINAL, restore it exactly where it belongs in COMPRESSED +- Code block mismatch: find the exact code block in ORIGINAL, restore it in COMPRESSED +- Heading mismatch: restore the exact heading text from ORIGINAL into COMPRESSED +- Do not touch any section not mentioned in the errors + +ORIGINAL (reference only): +{original} + +COMPRESSED (fix this): +{compressed} + +Return ONLY the fixed compressed file. No explanation. +""" + + +# ---------- Core Logic ---------- + + +def compress_file(filepath: Path) -> bool: + # Resolve and validate path + filepath = filepath.resolve() + MAX_FILE_SIZE = 500_000 # 500KB + if not filepath.exists(): + raise FileNotFoundError(f"File not found: {filepath}") + if filepath.stat().st_size > MAX_FILE_SIZE: + raise ValueError(f"File too large to compress safely (max 500KB): {filepath}") + + # Refuse files that look like they contain secrets or PII. Compressing ships + # the raw bytes to the Anthropic API — a third-party boundary — so we fail + # loudly rather than silently exfiltrate credentials or keys. Override is + # intentional: the user must rename the file if the heuristic is wrong. + if is_sensitive_path(filepath): + raise ValueError( + f"Refusing to compress {filepath}: filename looks sensitive " + "(credentials, keys, secrets, or known private paths). " + "Compression sends file contents to the Anthropic API. " + "Rename the file if this is a false positive." + ) + + print(f"Processing: {filepath}") + + if not should_compress(filepath): + print("Skipping (not natural language)") + return False + + original_text = filepath.read_text(errors="ignore") + backup_path = filepath.with_name(filepath.stem + ".original.md") + + # Check if backup already exists to prevent accidental overwriting + if backup_path.exists(): + print(f"⚠️ Backup file already exists: {backup_path}") + print("The original backup may contain important content.") + print("Aborting to prevent data loss. Please remove or rename the backup file if you want to proceed.") + return False + + # Step 1: Compress + print("Compressing with Claude...") + compressed = call_claude(build_compress_prompt(original_text)) + + # Save original as backup, write compressed to original path + backup_path.write_text(original_text) + filepath.write_text(compressed) + + # Step 2: Validate + Retry + for attempt in range(MAX_RETRIES): + print(f"\nValidation attempt {attempt + 1}") + + result = validate(backup_path, filepath) + + if result.is_valid: + print("Validation passed") + break + + print("❌ Validation failed:") + for err in result.errors: + print(f" - {err}") + + if attempt == MAX_RETRIES - 1: + # Restore original on failure + filepath.write_text(original_text) + backup_path.unlink(missing_ok=True) + print("❌ Failed after retries — original restored") + return False + + print("Fixing with Claude...") + compressed = call_claude( + build_fix_prompt(original_text, compressed, result.errors) + ) + filepath.write_text(compressed) + + return True diff --git a/skills/compress/scripts/detect.py b/skills/compress/scripts/detect.py new file mode 100644 index 0000000..5f50fd3 --- /dev/null +++ b/skills/compress/scripts/detect.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python3 +"""Detect whether a file is natural language (compressible) or code/config (skip).""" + +import json +import re +from pathlib import Path + +# Extensions that are natural language and compressible +COMPRESSIBLE_EXTENSIONS = {".md", ".txt", ".markdown", ".rst"} + +# Extensions that are code/config and should be skipped +SKIP_EXTENSIONS = { + ".py", ".js", ".ts", ".tsx", ".jsx", ".json", ".yaml", ".yml", + ".toml", ".env", ".lock", ".css", ".scss", ".html", ".xml", + ".sql", ".sh", ".bash", ".zsh", ".go", ".rs", ".java", ".c", + ".cpp", ".h", ".hpp", ".rb", ".php", ".swift", ".kt", ".lua", + ".dockerfile", ".makefile", ".csv", ".ini", ".cfg", +} + +# Patterns that indicate a line is code +CODE_PATTERNS = [ + re.compile(r"^\s*(import |from .+ import |require\(|const |let |var )"), + re.compile(r"^\s*(def |class |function |async function |export )"), + re.compile(r"^\s*(if\s*\(|for\s*\(|while\s*\(|switch\s*\(|try\s*\{)"), + re.compile(r"^\s*[\}\]\);]+\s*$"), # closing braces/brackets + re.compile(r"^\s*@\w+"), # decorators/annotations + re.compile(r'^\s*"[^"]+"\s*:\s*'), # JSON-like key-value + re.compile(r"^\s*\w+\s*=\s*[{\[\(\"']"), # assignment with literal +] + + +def _is_code_line(line: str) -> bool: + """Check if a line looks like code.""" + return any(p.match(line) for p in CODE_PATTERNS) + + +def _is_json_content(text: str) -> bool: + """Check if content is valid JSON.""" + try: + json.loads(text) + return True + except (json.JSONDecodeError, ValueError): + return False + + +def _is_yaml_content(lines: list[str]) -> bool: + """Heuristic: check if content looks like YAML.""" + yaml_indicators = 0 + for line in lines[:30]: + stripped = line.strip() + if stripped.startswith("---"): + yaml_indicators += 1 + elif re.match(r"^\w[\w\s]*:\s", stripped): + yaml_indicators += 1 + elif stripped.startswith("- ") and ":" in stripped: + yaml_indicators += 1 + # If most non-empty lines look like YAML + non_empty = sum(1 for l in lines[:30] if l.strip()) + return non_empty > 0 and yaml_indicators / non_empty > 0.6 + + +def detect_file_type(filepath: Path) -> str: + """Classify a file as 'natural_language', 'code', 'config', or 'unknown'. + + Returns: + One of: 'natural_language', 'code', 'config', 'unknown' + """ + ext = filepath.suffix.lower() + + # Extension-based classification + if ext in COMPRESSIBLE_EXTENSIONS: + return "natural_language" + if ext in SKIP_EXTENSIONS: + return "code" if ext not in {".json", ".yaml", ".yml", ".toml", ".ini", ".cfg", ".env"} else "config" + + # Extensionless files (like CLAUDE.md, TODO) — check content + if not ext: + try: + text = filepath.read_text(errors="ignore") + except (OSError, PermissionError): + return "unknown" + + lines = text.splitlines()[:50] + + if _is_json_content(text[:10000]): + return "config" + if _is_yaml_content(lines): + return "config" + + code_lines = sum(1 for l in lines if l.strip() and _is_code_line(l)) + non_empty = sum(1 for l in lines if l.strip()) + if non_empty > 0 and code_lines / non_empty > 0.4: + return "code" + + return "natural_language" + + return "unknown" + + +def should_compress(filepath: Path) -> bool: + """Return True if the file is natural language and should be compressed.""" + if not filepath.is_file(): + return False + # Skip backup files + if filepath.name.endswith(".original.md"): + return False + return detect_file_type(filepath) == "natural_language" + + +if __name__ == "__main__": + import sys + + if len(sys.argv) < 2: + print("Usage: python detect.py [file2] ...") + sys.exit(1) + + for path_str in sys.argv[1:]: + p = Path(path_str).resolve() + file_type = detect_file_type(p) + compress = should_compress(p) + print(f" {p.name:30s} type={file_type:20s} compress={compress}") diff --git a/skills/compress/scripts/validate.py b/skills/compress/scripts/validate.py new file mode 100644 index 0000000..3c4d4c1 --- /dev/null +++ b/skills/compress/scripts/validate.py @@ -0,0 +1,189 @@ +#!/usr/bin/env python3 +import re +from pathlib import Path + +URL_REGEX = re.compile(r"https?://[^\s)]+") +FENCE_OPEN_REGEX = re.compile(r"^(\s{0,3})(`{3,}|~{3,})(.*)$") +HEADING_REGEX = re.compile(r"^(#{1,6})\s+(.*)", re.MULTILINE) +BULLET_REGEX = re.compile(r"^\s*[-*+]\s+", re.MULTILINE) + +# crude but effective path detection +# Requires either a path prefix (./ ../ / or drive letter) or a slash/backslash within the match +PATH_REGEX = re.compile(r"(?:\./|\.\./|/|[A-Za-z]:\\)[\w\-/\\\.]+|[\w\-\.]+[/\\][\w\-/\\\.]+") + + +class ValidationResult: + def __init__(self): + self.is_valid = True + self.errors = [] + self.warnings = [] + + def add_error(self, msg): + self.is_valid = False + self.errors.append(msg) + + def add_warning(self, msg): + self.warnings.append(msg) + + +def read_file(path: Path) -> str: + return path.read_text(errors="ignore") + + +# ---------- Extractors ---------- + + +def extract_headings(text): + return [(level, title.strip()) for level, title in HEADING_REGEX.findall(text)] + + +def extract_code_blocks(text): + """Line-based fenced code block extractor. + + Handles ``` and ~~~ fences with variable length (CommonMark: closing + fence must use same char and be at least as long as opening). Supports + nested fences (e.g. an outer 4-backtick block wrapping inner 3-backtick + content). + """ + blocks = [] + lines = text.split("\n") + i = 0 + n = len(lines) + while i < n: + m = FENCE_OPEN_REGEX.match(lines[i]) + if not m: + i += 1 + continue + fence_char = m.group(2)[0] + fence_len = len(m.group(2)) + open_line = lines[i] + block_lines = [open_line] + i += 1 + closed = False + while i < n: + close_m = FENCE_OPEN_REGEX.match(lines[i]) + if ( + close_m + and close_m.group(2)[0] == fence_char + and len(close_m.group(2)) >= fence_len + and close_m.group(3).strip() == "" + ): + block_lines.append(lines[i]) + closed = True + i += 1 + break + block_lines.append(lines[i]) + i += 1 + if closed: + blocks.append("\n".join(block_lines)) + # Unclosed fences are silently skipped — they indicate malformed markdown + # and including them would cause false-positive validation failures. + return blocks + + +def extract_urls(text): + return set(URL_REGEX.findall(text)) + + +def extract_paths(text): + return set(PATH_REGEX.findall(text)) + + +def count_bullets(text): + return len(BULLET_REGEX.findall(text)) + + +# ---------- Validators ---------- + + +def validate_headings(orig, comp, result): + h1 = extract_headings(orig) + h2 = extract_headings(comp) + + if len(h1) != len(h2): + result.add_error(f"Heading count mismatch: {len(h1)} vs {len(h2)}") + + if h1 != h2: + result.add_warning("Heading text/order changed") + + +def validate_code_blocks(orig, comp, result): + c1 = extract_code_blocks(orig) + c2 = extract_code_blocks(comp) + + if c1 != c2: + result.add_error("Code blocks not preserved exactly") + + +def validate_urls(orig, comp, result): + u1 = extract_urls(orig) + u2 = extract_urls(comp) + + if u1 != u2: + result.add_error(f"URL mismatch: lost={u1 - u2}, added={u2 - u1}") + + +def validate_paths(orig, comp, result): + p1 = extract_paths(orig) + p2 = extract_paths(comp) + + if p1 != p2: + result.add_warning(f"Path mismatch: lost={p1 - p2}, added={p2 - p1}") + + +def validate_bullets(orig, comp, result): + b1 = count_bullets(orig) + b2 = count_bullets(comp) + + if b1 == 0: + return + + diff = abs(b1 - b2) / b1 + + if diff > 0.15: + result.add_warning(f"Bullet count changed too much: {b1} -> {b2}") + + +# ---------- Main ---------- + + +def validate(original_path: Path, compressed_path: Path) -> ValidationResult: + result = ValidationResult() + + orig = read_file(original_path) + comp = read_file(compressed_path) + + validate_headings(orig, comp, result) + validate_code_blocks(orig, comp, result) + validate_urls(orig, comp, result) + validate_paths(orig, comp, result) + validate_bullets(orig, comp, result) + + return result + + +# ---------- CLI ---------- + +if __name__ == "__main__": + import sys + + if len(sys.argv) != 3: + print("Usage: python validate.py ") + sys.exit(1) + + orig = Path(sys.argv[1]).resolve() + comp = Path(sys.argv[2]).resolve() + + res = validate(orig, comp) + + print(f"\nValid: {res.is_valid}") + + if res.errors: + print("\nErrors:") + for e in res.errors: + print(f" - {e}") + + if res.warnings: + print("\nWarnings:") + for w in res.warnings: + print(f" - {w}") diff --git a/skills/find-skills/SKILL.md b/skills/find-skills/SKILL.md new file mode 100644 index 0000000..114c663 --- /dev/null +++ b/skills/find-skills/SKILL.md @@ -0,0 +1,142 @@ +--- +name: find-skills +description: Helps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skill should be used when the user is looking for functionality that might exist as an installable skill. +--- + +# Find Skills + +This skill helps you discover and install skills from the open agent skills ecosystem. + +## When to Use This Skill + +Use this skill when the user: + +- Asks "how do I do X" where X might be a common task with an existing skill +- Says "find a skill for X" or "is there a skill for X" +- Asks "can you do X" where X is a specialized capability +- Expresses interest in extending agent capabilities +- Wants to search for tools, templates, or workflows +- Mentions they wish they had help with a specific domain (design, testing, deployment, etc.) + +## What is the Skills CLI? + +The Skills CLI (`npx skills`) is the package manager for the open agent skills ecosystem. Skills are modular packages that extend agent capabilities with specialized knowledge, workflows, and tools. + +**Key commands:** + +- `npx skills find [query]` - Search for skills interactively or by keyword +- `npx skills add ` - Install a skill from GitHub or other sources +- `npx skills check` - Check for skill updates +- `npx skills update` - Update all installed skills + +**Browse skills at:** https://skills.sh/ + +## How to Help Users Find Skills + +### Step 1: Understand What They Need + +When a user asks for help with something, identify: + +1. The domain (e.g., React, testing, design, deployment) +2. The specific task (e.g., writing tests, creating animations, reviewing PRs) +3. Whether this is a common enough task that a skill likely exists + +### Step 2: Check the Leaderboard First + +Before running a CLI search, check the [skills.sh leaderboard](https://skills.sh/) to see if a well-known skill already exists for the domain. The leaderboard ranks skills by total installs, surfacing the most popular and battle-tested options. + +For example, top skills for web development include: +- `vercel-labs/agent-skills` — React, Next.js, web design (100K+ installs each) +- `anthropics/skills` — Frontend design, document processing (100K+ installs) + +### Step 3: Search for Skills + +If the leaderboard doesn't cover the user's need, run the find command: + +```bash +npx skills find [query] +``` + +For example: + +- User asks "how do I make my React app faster?" → `npx skills find react performance` +- User asks "can you help me with PR reviews?" → `npx skills find pr review` +- User asks "I need to create a changelog" → `npx skills find changelog` + +### Step 4: Verify Quality Before Recommending + +**Do not recommend a skill based solely on search results.** Always verify: + +1. **Install count** — Prefer skills with 1K+ installs. Be cautious with anything under 100. +2. **Source reputation** — Official sources (`vercel-labs`, `anthropics`, `microsoft`) are more trustworthy than unknown authors. +3. **GitHub stars** — Check the source repository. A skill from a repo with <100 stars should be treated with skepticism. + +### Step 5: Present Options to the User + +When you find relevant skills, present them to the user with: + +1. The skill name and what it does +2. The install count and source +3. The install command they can run +4. A link to learn more at skills.sh + +Example response: + +``` +I found a skill that might help! The "react-best-practices" skill provides +React and Next.js performance optimization guidelines from Vercel Engineering. +(185K installs) + +To install it: +npx skills add vercel-labs/agent-skills@react-best-practices + +Learn more: https://skills.sh/vercel-labs/agent-skills/react-best-practices +``` + +### Step 6: Offer to Install + +If the user wants to proceed, you can install the skill for them: + +```bash +npx skills add -g -y +``` + +The `-g` flag installs globally (user-level) and `-y` skips confirmation prompts. + +## Common Skill Categories + +When searching, consider these common categories: + +| Category | Example Queries | +| --------------- | ---------------------------------------- | +| Web Development | react, nextjs, typescript, css, tailwind | +| Testing | testing, jest, playwright, e2e | +| DevOps | deploy, docker, kubernetes, ci-cd | +| Documentation | docs, readme, changelog, api-docs | +| Code Quality | review, lint, refactor, best-practices | +| Design | ui, ux, design-system, accessibility | +| Productivity | workflow, automation, git | + +## Tips for Effective Searches + +1. **Use specific keywords**: "react testing" is better than just "testing" +2. **Try alternative terms**: If "deploy" doesn't work, try "deployment" or "ci-cd" +3. **Check popular sources**: Many skills come from `vercel-labs/agent-skills` or `ComposioHQ/awesome-claude-skills` + +## When No Skills Are Found + +If no relevant skills exist: + +1. Acknowledge that no existing skill was found +2. Offer to help with the task directly using your general capabilities +3. Suggest the user could create their own skill with `npx skills init` + +Example: + +``` +I searched for skills related to "xyz" but didn't find any matches. +I can still help you with this task directly! Would you like me to proceed? + +If this is something you do often, you could create your own skill: +npx skills init my-xyz-skill +``` diff --git a/skills/project-bootstrap/SKILL.md b/skills/project-bootstrap/SKILL.md new file mode 100644 index 0000000..682de92 --- /dev/null +++ b/skills/project-bootstrap/SKILL.md @@ -0,0 +1,212 @@ +--- +name: project-bootstrap +description: > + Initializes or upgrades a project in the current folder: git, .gitignore, README.md, + .wiki/ using Karpathy's method, .tasks/ for task tracking, CLAUDE.md with skill triggers. + Use this skill when the user says "initialize project", "bootstrap", "setup project", + "upgrade project", "add wiki", "add tasks", "start project", "set everything up", + or launches the agent in a new or existing folder and wants to configure the workspace. + Trigger even if the user just says "let's start a project" or "set it all up". +--- + +# Project Bootstrap + +Sets up a complete working environment for a monorepo project in one pass. +Operates in two modes: **init** (new project) and **upgrade** (existing project). + +--- + +## Step 0 — Detect mode + +Check what already exists in the current directory: + +```bash +ls -la +git rev-parse --git-dir 2>/dev/null && echo "git:yes" || echo "git:no" +[ -d .wiki ] && echo "wiki:yes" || echo "wiki:no" +[ -d .tasks ] && echo "tasks:yes" || echo "tasks:no" +[ -f CLAUDE.md ] && echo "claude:yes" || echo "claude:no" +[ -f README.md ] && echo "readme:yes" || echo "readme:no" +``` + +Show the user a summary in one block — what was found, what will be created: + +``` +Found: ✅ git ❌ .wiki ✅ .tasks ❌ CLAUDE.md ✅ README.md +Create: .wiki CLAUDE.md +Skip: git (exists) .tasks (exists) README.md (exists) +``` + +Ask one question: "Looks right? Shall we proceed?" — and wait for confirmation. +**Create nothing before confirmation.** + +--- + +## Step 1 — Git + +If git is not initialized: + +```bash +git init +``` + +If `.gitignore` does not exist — create from template `assets/.gitignore.template`. +If it exists — leave it untouched. + +--- + +## Step 2 — README.md + +If it does not exist — create a minimal one: + +```markdown +# + +## About + + +## Quick start + +``` + +If it exists — leave it untouched. + +--- + +## Step 3 — .wiki/ + +If `.wiki/` already exists — skip it, notify the user. + +If it does not exist — create the structure: + +``` +.wiki/ + raw/ ← raw inputs: transcripts, docs, links — anything not yet processed + source/ ← compiled knowledge: wiki pages maintained by the agent + SUMMARY.md ← table of contents: all source/ pages with one-line descriptions + WORKFLOW.md ← how to work with the wiki: when to read, when to update +``` + +Contents of `WORKFLOW.md`: + +```markdown +# Wiki Workflow + +This wiki follows Karpathy's method: knowledge accumulates rather than being re-derived. + +## Rules for the agent + +- **Before starting work** — read relevant pages from source/ +- **After an important decision** — update or create a page in source/ +- **New raw materials** (links, docs, transcripts) — put in raw/ +- **Processed knowledge** — write as markdown pages in source/ +- **SUMMARY.md** — update whenever a page is added to source/ + +## Trigger + +Skill is activated by the phrase: **use project wiki** +``` + +Contents of `SUMMARY.md`: + +```markdown +# Wiki Summary + +List of all pages in source/ with one-line descriptions. +The agent updates this file whenever source/ changes. + +## Pages + + +``` + +--- + +## Step 4 — .tasks/ + +If `.tasks/` already exists — skip it, notify the user. + +If it does not exist — create the structure: + +``` +.tasks/ + STATUS.md ← task board +``` + +Contents of `STATUS.md`: + +```markdown +# Task Board +_Updated: _ + + +``` + +--- + +## Step 5 — CLAUDE.md + +If `CLAUDE.md` already exists — show its contents and ask: +"CLAUDE.md already exists. Append skill triggers to the end, or leave as is?" + +If it does not exist — create from template `assets/CLAUDE.md.template`. + +Template contents: +```markdown +# CLAUDE.md +# Agent instructions. Each line is a trigger for an installed skill. + +talk like a caveman +use superpowers +use project wiki +use task management system +``` + +--- + +## Step 6 — Commit + +```bash +git add . +git commit -m "chore: bootstrap project structure" +``` + +If the repo already had commits — commit only the files just created: + +```bash +git add .wiki/ .tasks/ CLAUDE.md .gitignore README.md +git commit -m "chore: upgrade project structure" +``` + +--- + +## Step 7 — Summary + +Print a final report: + +``` +✅ Done! Created: + .wiki/ — project wiki (Karpathy method) + .tasks/ — task tracking system + CLAUDE.md — skill triggers + .gitignore — standard template + README.md — starter file + +Skipped (already existed): + git — left untouched + +Next step: describe the project in README.md and start your first task — +say "use task management system". +``` + +--- + +## Rules + +- **Never overwrite** existing files without explicit user confirmation +- **Always show the plan first** — one question, one confirmation +- **Never invent details** — if the project already exists, read what's there +- **Commit only what was just created** — do not touch the rest of the file tree +- **Commit automatically** after each successful step, no extra questions +- **Push only after explicit user confirmation** — ask "Push to remote?" and wait for "yes" diff --git a/skills/project-bootstrap/assets/.gitignore.template b/skills/project-bootstrap/assets/.gitignore.template new file mode 100644 index 0000000..de37d73 --- /dev/null +++ b/skills/project-bootstrap/assets/.gitignore.template @@ -0,0 +1,29 @@ +# Dependencies +node_modules/ +.venv/ +__pycache__/ +*.pyc + +# Build outputs +dist/ +build/ +*.egg-info/ + +# Environment +.env +.env.local +.env.*.local + +# IDE +.idea/ +.vscode/ +*.swp +*.swo + +# OS +.DS_Store +Thumbs.db + +# Logs +*.log +logs/ diff --git a/skills/project-bootstrap/assets/CLAUDE.md.template b/skills/project-bootstrap/assets/CLAUDE.md.template new file mode 100644 index 0000000..6c25120 --- /dev/null +++ b/skills/project-bootstrap/assets/CLAUDE.md.template @@ -0,0 +1,7 @@ +# CLAUDE.md +# Agent instructions. Each line is a trigger for an installed skill. + +talk like a caveman +use superpowers +use project wiki +use task management system diff --git a/skills/task-status-wiki/SKILL.md b/skills/task-status-wiki/SKILL.md new file mode 100644 index 0000000..53f4f1e --- /dev/null +++ b/skills/task-status-wiki/SKILL.md @@ -0,0 +1,146 @@ +--- +name: task-status-wiki +description: > + Manage a persistent task status wiki inside a monorepo using the .tasks/ folder pattern. + Use this skill whenever the user is switching between tasks, resuming a paused task, + starting a new task, asking "where were we", or wanting to track progress across multiple + parallel workstreams. Also use when the user says "pause", "switch to X", "what's the status", + "update status", or "initialize task tracking". Trigger even if the user doesn't explicitly + mention the wiki — any context-switching or multi-task coordination question in a code project + is a signal to use this skill. +--- + +# Task Status Wiki + +A pattern for maintaining compressed working context across parallel tasks in a monorepo. +The agent reads and updates a small wiki in `.tasks/` so that every session starts oriented +and every switch costs seconds, not minutes. + +## Structure + +``` +/ + .tasks/ + STATUS.md ← board: one block per task, sorted by priority + .md ← deep context per task, one file each +``` + +Commit `.tasks/` to git. Decision history is valuable; diffs show how thinking evolved. + +--- + +## STATUS.md format + +```markdown +# Task Board +_Updated: YYYY-MM-DD_ + +## 🔴 [task-slug] — short description +**Status:** active | paused | blocked | done +**Where I stopped:** one sentence — the exact thought or action interrupted +**Next action:** one concrete step to resume immediately +**Blocker:** (only if blocked) what is preventing progress +**Branch:** git branch name + +--- +``` + +**Emoji convention:** +- 🔴 Active — currently worked on (only one at a time) +- 🟡 Paused — in progress, resumable +- ⚪ Ready — not started, fully defined +- 🟢 Done — completed, kept until merged +- 🔵 Blocked — waiting on external input + +--- + +## Per-task file format (`.md`) + +```markdown +# + +## Goal +One paragraph. What this achieves and why it matters in the monorepo. + +## Key files +- `path/to/file.ts` — role in this task +- `path/to/other.ts:42` — specific line if relevant + +## Decisions log +Reverse-chronological. Append only — never rewrite past entries. +- YYYY-MM-DD: Why X was chosen over Y +- YYYY-MM-DD: Constraint Z discovered, approach adjusted + +## Open questions +- [ ] unresolved design or dependency questions + +## Completed steps +- [x] steps finished this or previous sessions + +## Notes +Temporary hypotheses, links, names of people to consult. +``` + +--- + +## Agent operations + +### Session start +1. Check if `.tasks/STATUS.md` exists. If not → run **Initialization** below. +2. Read `STATUS.md`. +3. If user names a task, read its `.md`. +4. Confirm in one sentence: "We're in the middle of X, next step is Y." +5. Ask if the plan is still correct before doing anything. +6. If STATUS.md `_Updated` date is >3 days ago, flag it and ask user to confirm current state. + +### Session end / pause / switch +1. Update `STATUS.md`: set current task to 🟡, update "Where I stopped" and "Next action". +2. Append to `.md` Decisions log any non-obvious choices made this session. +3. Move finished items to "Completed steps". +4. Commit: `git add .tasks/ && git commit -m "chore: update task status []"` + +### Task switch +1. Perform session-end operations for the current task. +2. Read the target `.md`. +3. Set it to 🔴 in STATUS.md (demote previous active to 🟡). +4. Confirm orientation before starting work. + +### New task creation +1. Ask: task name (slug), goal, known key files, branch name. +2. Create `.md` with Goal and Key files populated. +3. Add ⚪ block to `STATUS.md`. +4. Create and checkout branch if it doesn't exist. + +### Task completion +1. Resolve or drop all open questions. +2. Set status to 🟢 in STATUS.md. +3. Append final summary line to Decisions log. +4. Remind user to delete the branch after merge. + +--- + +## Initialization (first time in a project) + +Run when `.tasks/` doesn't exist yet. + +1. Confirm understanding in one sentence. +2. Ask: "What tasks are currently in flight? Give me each name and a one-line description." +3. For each task: ask for branch, current status, and where work stopped last. +4. Create `.tasks/STATUS.md` with all tasks filled in. +5. Create `.md` for each active or paused task. +6. Commit: `git add .tasks/ && git commit -m "chore: init task status wiki"` +7. Print path to STATUS.md and confirm it's ready. + +**Do not invent task details — ask for what is unknown.** + +--- + +## Rules + +- **Never lose "Where I stopped"** — most critical field. If unclear, ask before ending session. +- **One sentence per STATUS.md field** — compress, don't write prose. +- **Key files must be specific** — not "auth module" but `packages/auth/src/useAuth.ts:87`. +- **Decisions log is append-only** — past entries are immutable. +- **Commit after every session end** — git log is the history of thinking. +- **Always confirm orientation at session start** — state understanding before acting. +- **One active task at a time** — only one 🔴 in STATUS.md. diff --git a/skills/using-context7/SKILL.md b/skills/using-context7/SKILL.md new file mode 100644 index 0000000..0e0e7fa --- /dev/null +++ b/skills/using-context7/SKILL.md @@ -0,0 +1,114 @@ +--- +name: using-context7 +description: Use when answering questions about a specific library, framework, SDK, API, or CLI tool — including setup/install, config, API syntax, version-specific behavior, migration between versions, or library-specific errors. Training data is often stale; context7 returns current docs. Skip for general programming concepts, refactoring, business-logic debugging, or when the codebase already answers the question. +--- + +# Using the context7 MCP server + +## Overview + +`context7` is an MCP server that fetches **current** documentation for named libraries and frameworks. Two tools: `mcp__context7__resolve-library-id` (name → library ID) and `mcp__context7__query-docs` (library ID + question → doc snippets). + +Your training data has a cutoff. Library APIs change. If a question names a library, **reach for context7 before answering from memory**, even for libraries you "know" — your recall may be one or two majors behind. + +## When to use + +Use when the user asks about any of these in the context of a specific library: + +- Install / setup / init commands +- Config file shape (`nuxt.config.ts`, `next.config.mjs`, `tsconfig.json` extends, `vite.config`, etc.) +- API / component / hook / composable syntax +- Migration between versions (v3 → v4, v14 → v15) +- Library-specific errors / warnings +- CLI flags +- Feature availability ("does X support Y?") +- Plugin / module ecosystem questions + +Common triggers: "how do I …", "what's the right way to … in ", "is there a way to …", any error message containing a library's name, any config file snippet. + +**Prefer context7 over WebSearch / WebFetch for library docs** — it returns curated snippets, not rendered marketing pages. + +## When NOT to use + +- General programming concepts (closures, concurrency, algorithms) +- Refactoring / code review / business-logic debugging +- Writing new code from scratch where the stack isn't named +- Questions the current codebase answers (read the repo first) +- Your own prior-conversation context (use wiki / memory instead) + +## Workflow + +``` +1. Identify the library (and version, if the user mentioned one) +2. resolve-library-id → pick best match by name + reputation + snippet count +3. query-docs with the ID + a specific question +4. Cite what you found; fall back only if context7 returned nothing useful +``` + +**Budget: 3 calls per question, max.** After 3, use what you have — don't loop. + +If the user already gave a library ID in `/org/project` or `/org/project/version` form, skip step 2 and go straight to `query-docs`. + +## Tool quick reference + +| Tool | Required args | Purpose | +|---|---|---| +| `mcp__context7__resolve-library-id` | `libraryName`, `query` | Name → `/org/project` ID. Use official casing ("Next.js", not "nextjs"). | +| `mcp__context7__query-docs` | `libraryId`, `query` | ID → doc snippets. `query` must be specific. | + +Library ID format: `/org/project` (e.g. `/vercel/next.js`) or `/org/project/version` (e.g. `/vercel/next.js/v14.3.0`). + +## Good vs bad queries + +**`resolve-library-id` — pick official names:** + +``` +libraryName: "Nuxt" query: "Nuxt 4 config and route rules" ✅ +libraryName: "nuxt4" query: "nuxt" ❌ (wrong casing, vague query) +``` + +**`query-docs` — be specific:** + +``` +query: "How to set up @nuxtjs/i18n with prefix_except_default and ru default locale in Nuxt 4" ✅ +query: "i18n" ❌ +query: "How to configure YooKassa payment provider in Medusa v2 core flows" ✅ +query: "payments" ❌ +``` + +A specific query returns targeted snippets; a vague one returns a grab bag you'll ignore. + +## Example + +User: "How do `routeRules` work in Nuxt 4?" + +``` +1. mcp__context7__resolve-library-id + libraryName: "Nuxt" + query: "Nuxt 4 routeRules hybrid rendering" + → /nuxt/nuxt (or /nuxt/nuxt/v4.x.x if version known) + +2. mcp__context7__query-docs + libraryId: "/nuxt/nuxt" + query: "routeRules for hybrid rendering: ssr, prerender, isr, swr — syntax and examples" + → doc snippets + +3. Answer using the snippets. Cite the library + version. +``` + +## Common mistakes + +| Mistake | Fix | +|---|---| +| Answering from memory on a library question | Run `resolve-library-id` first. Your training data is stale. | +| Calling `query-docs` without resolving first | Required unless user already gave `/org/project` ID. | +| Vague queries ("auth", "hooks", "config") | Include the specific task, version, and constraints. | +| Looping until you find the "perfect" answer | 3-call hard cap. Take the best result and move on. | +| Using context7 for codebase questions | Read the code. context7 doesn't know your repo. | +| Using context7 for general concepts | Answer from training data. context7 is for libraries. | + +## Red flags + +- "I already know this library" → your recall may be one major behind. Resolve anyway if the user is about to act on your answer. +- "This will take too many calls" → you have 3. Use them. +- "The error message looks obvious" → error messages that include a library name are a strong context7 signal. diff --git a/skills/using-markitdown/SKILL.md b/skills/using-markitdown/SKILL.md new file mode 100644 index 0000000..20bcf77 --- /dev/null +++ b/skills/using-markitdown/SKILL.md @@ -0,0 +1,80 @@ +--- +name: using-markitdown +description: Use when capturing external content into a markdown-based knowledge base, wiki `raw/` directory, or any pipeline that must preserve the source's full text — for web pages, PDFs, DOCX/PPTX/XLSX, EPUB, CSV/JSON/XML, ZIP archives, images (with OCR/EXIF), audio (with transcription), or YouTube URLs. Also use when WebFetch returned an LLM-summarized version but the raw content is what's needed. +--- + +# using-markitdown + +> Convert almost any URI to plain markdown using Microsoft's `markitdown` MCP server. Returns **raw textual content**, not an LLM summary. + +## Tool + +``` +mcp__markitdown__convert_to_markdown(uri: string) → markdown string +``` + +`uri` accepts: `http://`, `https://`, `file://`, `data:`. + +## Local files — Docker-mount caveat (READ FIRST) + +The markitdown MCP usually runs in a **Docker container** with a single host directory bind-mounted. The container does **not** see your full host filesystem. `file://` URIs must point to the **in-container path**, not the host path. + +1. Open `~/.claude.json` and find `mcpServers.markitdown.args`. Look for the `-v` flag — e.g. `-v C:\Users\vitya:/workdir` means host `C:\Users\vitya` is mounted at `/workdir` inside the container. +2. Translate the host path to the container path before forming the URI. +3. Forward slashes only inside the container path. + +**Example.** Host file at `C:\Users\vitya\modular\heart-and-mask\.wiki\raw\foo.html` with mount `C:\Users\vitya:/workdir`: + +``` +file:///workdir/modular/heart-and-mask/.wiki/raw/foo.html +``` + +**Symptom of getting this wrong:** `[Errno 2] No such file or directory: '/c:/Users/...'` — the container literally tried to open the host-shaped path. The fix is path translation, not URL encoding. + +**If the file falls outside the mount:** either copy it into the mounted tree, or extend the mount in `~/.claude.json` (a Claude restart is required for MCP changes to take effect — MCP servers are spawned at session start). + +**Filenames.** Non-ASCII filenames (Cyrillic, etc.) inside `file://` URIs are flaky across the URL-encode → urllib → Docker → host-FS chain. Rename to Latin kebab-case **before** calling markitdown. + +## When to use + +- Filling a wiki's `raw/` directory from a URL or local PDF/DOCX. +- Datasheets, papers, blog posts, GitHub READMEs, Obsidian Web Clipper outputs, KiCad netlist exports — anything where the source text matters and lossy summarization would break later ingest steps. +- Any time the next step is "save the source verbatim before summarizing". + +## When NOT to use + +- You only need a *summary* or an *answer about* a page → use **WebFetch** (cheaper, runs through a small model, returns prose). +- The URI is GitHub/PR/issue/release content → use `gh` CLI (richer metadata, structured output). +- The URI is private/authenticated (GDocs, Confluence, Jira, Slack, Notion, `share.google/*` sign-in walls) → markitdown receives the **public-facing fallback page** (sign-in screen, cookie banner) and returns *that* as markdown. Verify the result is real content before saving. +- **The URI is a browser-rendered web page the user is already viewing** → ask the user to capture it via **Obsidian Web Clipper** (browser extension, runs Readability extraction client-side) and drop the resulting `.md` into `raw/`. Web Clipper output is dramatically cleaner than markitdown's HTML pass — no nav chrome, no sidebar history, no cookie banners — plus it carries YAML frontmatter (title / source URL / date) out of the box. Reserves markitdown for things browsers can't easily save (PDF, DOCX, PPTX, XLSX, EPUB, file:// resources). Note: rename the resulting file to Latin kebab-case before ingest (Web Clipper preserves the page `` verbatim, often non-ASCII). + +## Pattern: ingest a remote source into a wiki + +``` +1. mcp__markitdown__convert_to_markdown(uri="https://example.com/foo.pdf") +2. Inspect the head of the result. If it looks like a sign-in/cookie/consent page, abort — ask the user for an alternative (manual save, paste, authenticated MCP). +3. Write the result to .wiki/raw/<slug>.md (kebab-case, Latin only). +4. Register the new file in .wiki/raw/README.md. +5. Hand off to the wiki ingest workflow (creates sources/<slug>.md summary + entity/concept updates). +``` + +## Common gotchas + +| Symptom | Cause | Fix | +|---|---|---| +| Output is a Google/Microsoft sign-in page in some random language | URI behind auth wall | Ask user to export the content manually (Save as PDF, copy-paste) and put it in `raw/` | +| Output is mostly nav/cookie banner text | Site is JS-rendered or anti-bot | Try the cached or print URL; or ask user for HTML export | +| Output lacks images / diagrams | Markdown is text-only by design | Save the original asset separately under `raw/assets/`; reference it from the `sources/` summary | +| Tool not available in session | MCP server not loaded | Confirm `mcp__markitdown__convert_to_markdown` appears via ToolSearch; load with `select:mcp__markitdown__convert_to_markdown` | +| Huge output (book-length) | Whole document converted in one call | Save raw, then summarize *from the saved file* — do not hold the entire markdown in working context | + +## Quick contrast with WebFetch and Web Clipper + +| | markitdown | WebFetch | Obsidian Web Clipper | +|---|---|---|---| +| Returns | raw markdown of the source | LLM answer about the source | Readability-extracted markdown with YAML frontmatter | +| Use for | PDF / DOCX / non-browser-friendly | one-shot Q&A | any browser-viewable web page | +| Auth-aware | no | no | **yes** (uses the user's logged-in browser session) | +| Chrome / nav stripped | partial (still verbose) | n/a (model picks signal) | **yes** (clean) | +| Handles PDFs / DOCX | yes | text-only HTML extraction | no (web pages only) | +| Who triggers it | Claude | Claude | the user (manual click) | diff --git a/skills/wiki-maintainer/SKILL.md b/skills/wiki-maintainer/SKILL.md new file mode 100644 index 0000000..26baefa --- /dev/null +++ b/skills/wiki-maintainer/SKILL.md @@ -0,0 +1,127 @@ +--- +name: wiki-maintainer +description: Use when a repo has a `.wiki/` directory following Karpathy's LLM Wiki pattern and the user asks to ingest a document, answer from the wiki, lint/health-check it, or says "заингесть", "обнови вики", "проверь вики", "запроси вики", "query the wiki". Also use when modifying any file under `.wiki/` — the workflow and formats below are mandatory, and project-specific conventions live in `.wiki/CLAUDE.md`. +--- + +# wiki-maintainer + +> Maintain an LLM Wiki (Karpathy pattern: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f). Knowledge is **compiled once and kept current** across three layers, via three named operations, with strict file formats that make the wiki parseable and grep-friendly. + +## Three layers (do not blur) + +1. **Raw sources** — `.wiki/raw/` (or external paths registered in `raw/README.md`). **Immutable.** Read, never edit. The only exception is appending a `> Status` blockquote when the user explicitly asks for a status audit. +2. **Wiki** — everything else under `.wiki/`. Agent-owned. Entity / concept / package / source summary pages. +3. **Schema** — `.wiki/CLAUDE.md`. Project-specific conventions (what entities, what packages, naming). Always read it first if present; it overrides this skill when it conflicts. + +## First step on every operation + +1. Read `.wiki/CLAUDE.md` if it exists. +2. Read `.wiki/index.md` to locate relevant pages. +3. Only then act. + +If `.wiki/CLAUDE.md` is missing, treat it as a greenfield wiki and offer to bootstrap one (schema + empty index/log + `raw/README.md`). + +## Three operations + +### Ingest — «заингесть X» + +1. Read the raw source fully. +2. Extract: entities, concepts, packages, cross-cutting patterns. +3. Create `sources/<slug>.md` (one summary page per source, ~50–150 lines). +4. For each affected entity/concept/package page: + - If it exists → update it. **Flag contradictions explicitly** with `> **Противоречие:** источник A говорит X, источник B — Y`. Don't silently overwrite. + - If not → create it. +5. Update `index.md` — add or move entries. +6. Append one line to `log.md` (format below). +7. Report to the user: what created, what updated, what contradictions found. + +**One ingest may touch 10–15 pages. This is normal — that's why LLMs do it.** + +### Query — вопрос по wiki + +1. Read `index.md` first, then drill into relevant pages. +2. Answer with citations as markdown links to wiki pages. +3. **Compound the wiki.** If the answer is a real synthesis (comparison, analysis, new connection) — ask the user: "Сохранить как страницу wiki?" Good queries become durable pages under `concepts/`, `analyses/`, or similar. +4. Append one line to `log.md`. + +### Lint — «проверь wiki» + +Scan for: +- **Contradictions** between pages. +- **Orphans** — pages with no inbound links. +- **Stale claims** — git `log -p` on the raw source shows it was updated after the summary's `ingested:` date. +- **Missing entities** — concepts mentioned in prose but without their own page. +- **Empty/TODO sections.** + +Report as a punch list. Don't delete anything automatically. +Append one line to `log.md` summarizing the findings. + +## File formats (MANDATORY) + +### Page frontmatter + +```yaml +--- +title: Человекочитаемое имя +type: entity | concept | package | source | overview +tags: [short, tokens] +sources: [../sources/foo.md, ../sources/bar.md] +updated: 2026-04-21 +--- +``` + +Source pages also carry `ingested: YYYY-MM-DD` and `raw_path: ../raw/...`. + +### File naming + +- `kebab-case.md`, **Latin only**. Transliterate Cyrillic / other scripts in filenames (`план переписывания` → `ozon-client-rewrite.md`). Keep the original title in the H1 and frontmatter. +- `entities/<name>.md`, `concepts/<name>.md`, `packages/<name>.md` (no `@org/` prefix), `sources/<slug>.md`. + +### `log.md` — append-only, grep-parseable + +Every entry **must** start with: + +``` +## [YYYY-MM-DD] <operation> | <short description> +``` + +Operations: `ingest`, `query`, `lint`, `refactor`, `decision`, `init`. + +Parseable with: `grep "^## \[" .wiki/log.md | tail -20`. + +### `index.md` + +Catalog, not narrative. One line per page: `- [Title](path) — hook.` Sections by type (entities / concepts / packages / sources). Update on every ingest. + +### Cross-references + +- Wiki → wiki: relative markdown links, `[Name](../entities/x.md)`. +- Wiki → code: relative path from repo root: `[foo.js](../../packages/api/foo.js)`. +- Wiki → raw: `../raw/<file>`. +- URL-encode spaces in paths (`%20`) and Cyrillic when needed. + +## Quick reference + +| Situation | Files touched | +|---|---| +| Ingest one doc | `sources/<slug>.md` (new) + 3–15 entity/concept/package pages + `index.md` + `log.md` | +| Query | (read only) + optionally new wiki page + `log.md` | +| Lint | (read only) + `log.md` | +| Bootstrap empty wiki | `CLAUDE.md`, `index.md`, `log.md`, `overview.md`, `raw/README.md` + empty `entities/ concepts/ packages/ sources/` | + +## Common mistakes + +- **Editing `raw/`.** Don't. Only allowed: status blockquote when user explicitly asks. +- **Dumping raw content into `sources/`.** Summaries are summaries. Link to raw, don't copy it. +- **Silent overwrites.** When a new source contradicts an existing page, flag it with a `> **Противоречие:**` block; don't just overwrite. +- **Narrative `log.md`.** `Today I added…` is wrong. Use `## [YYYY-MM-DD] ingest | <what>`. +- **Non-ASCII file names.** Breaks greppability and cross-platform. Transliterate. +- **Forgetting `index.md`.** Pages not listed there are effectively invisible for future queries. +- **Skipping contradictions in lint.** The wiki's value grows from surfaced tensions, not from false consensus. +- **Bootstrapping a new wiki without asking for domain.** Directory layout depends on whether this is a code-project wiki, a research wiki, or a reading-companion wiki. Ask briefly before creating. + +## When NOT to use this skill + +- Project has CLAUDE.md / AGENTS.md docs but no `.wiki/` — that's regular project documentation, not an LLM Wiki. +- User wants a single-file README or ADR — this skill is for persistent interlinked knowledge bases. +- One-off questions about code — use regular file reading, not wiki workflow.