Watch Skill

Give every AI agent eyes for video—and a way to inspect its own work.

Watch Skill is a local-first video intelligence layer for agents. It turns videos, live streams, meetings, and screen recordings into searchable, timestamped evidence. Agents can understand what happened, retain it across sessions, and cite the exact moment behind an answer.

When the video is the agent's own browser or desktop session, THE LOOP closes the feedback cycle: record the work, evaluate it against plain-language criteria, guide the fix, and verify the result. The same engine is available through skills, 23 MCP tools, a CLI, REST, and native framework adapters.

Watch. Remember. Fix. Verify.

One video layer, three agent capabilities

Capability What the agent gains

Watch Scene-aware frames, on-screen text, and local-first transcription from 1,800+ sites, live HLS/DASH streams, local media, meetings, browsers, windows, and desktops.

Remember A persistent, searchable index with timestamp citations, hybrid retrieval, cross-video synthesis, and reusable lessons.

Verify A capture → critique → fix → proof loop for browser flows, interfaces, generated video, gameplay, and monitored streams.

Start in 60 seconds

Claude Code

/plugin marketplace add oxbshw/watch-skill
/plugin install watch-skill@watch-skill

Run /watch-skill:setup-watch-skill once after installation. It installs the engine, checks the binary dependencies, registers the MCP server, and offers to configure a vision provider.

macOS and Linux

curl -fsSL https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.sh | sh

Windows

powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.ps1 | iex"

Then watch a video and ask a follow-up:

watch-skill watch "https://youtu.be/..." "Summarize the important moments."
watch-skill ask <video_id> "When does the demo first fail?"
watch-skill search "pricing decision"       # search every indexed video
watch-skill serve                           # MCP over stdio

Transcription, OCR, and search can run locally without an API key. For visual Q&A, use Gemini, Anthropic, OpenAI, OpenRouter, or a local Ollama model. See Getting started for manual installation and Configuration for provider and privacy settings.

Why use it

• Evidence instead of frame dumps. Scene detection and perceptual deduplication spend the frame budget on distinct moments. Answers include timestamps, confidence, and the evidence used to support them.

• Persistent video memory. Analyze once, ask again without downloading or transcribing the same video. Hybrid full-text and vector retrieval works within one video or across the entire library.

• Local-first processing. Original-language captions are preferred, local Whisper is the default fallback, and cloud speech-to-text is opt-in. An Ollama configuration keeps the complete pipeline on the machine.

• Flow verification. THE LOOP records an agent's browser, screen, or window; checks the result against plain-language criteria; and produces before/after proof after a fix.

• Corrections that persist. report_mistake stores a local lesson, applies it to related questions, and turns it into a replayable evaluation.

• Measured cost controls. Text-first answers, semantic caching, configurable token budgets, and explicit cheapest, quality_first, and offline_only policies keep the trade-offs visible.

• Multilingual retrieval. Script-aware OCR routing, Arabic normalization, CJK substring matching, and multilingual embeddings support questions across languages.

The repository includes reproducible cost and perception benchmarks. Product claims in this README link to the relevant implementation notes or testable example rather than relying on unqualified marketing numbers.

Works with your agent

The setup command detects supported clients and updates their configuration with a backup. Manual guides are available for every entry below.

https://github.com/oxbshw/watch-skill/blob/main/docs/agents/claude-code.md[Claude Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/claude-code.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/claude-desktop.md[Claude Desktop](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/claude-desktop.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cursor.md[Cursor](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cursor.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/codex-cli.md[Codex CLI](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/codex-cli.md)

https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cline.md[Cline](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cline.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/windsurf.md[Windsurf](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/windsurf.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/gemini-cli.md[Gemini CLI](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/gemini-cli.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/vscode.md[VS Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/vscode.md)

https://github.com/oxbshw/watch-skill/blob/main/docs/agents/github-copilot-cli.md[GitHub Copilot CLI](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/github-copilot-cli.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kimi-code.md[Kimi Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kimi-code.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qwen-code.md[Qwen Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qwen-code.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/opencode.md[OpenCode](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/opencode.md)

https://github.com/oxbshw/watch-skill/blob/main/docs/agents/goose.md[Goose](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/goose.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openhands.md[OpenHands](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openhands.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kilocode.md[Kilo Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kilocode.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qodo.md[Qodo](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qodo.md)

https://github.com/oxbshw/watch-skill/blob/main/docs/agents/agent-zero.md[Agent Zero](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/agent-zero.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openclaw.md[OpenClaw](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openclaw.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/pi.md[Pi](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/pi.md) https://github.com/oxbshw/watch-skill/blob/main/docs/agents/hermes.md[Hermes](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/hermes.md)

https://github.com/oxbshw/watch-skill/blob/main/docs/agents/frameworks.md

Native tools are also available for LangChain/LangGraph, CrewAI, OpenAI Agents SDK, LlamaIndex, and AutoGen; any other framework can use REST or MCP.

Connection Supported agents and frameworks

Plugin and skills Claude Code, OpenClaw, Pi, Hermes-style agents

MCP Claude Desktop, Cursor, Codex CLI, Cline, Windsurf, Gemini CLI, VS Code, GitHub Copilot CLI, Kimi Code, Qwen Code, OpenCode, Goose, OpenHands, Kilo Code, Qodo, Agent Zero

Native Python tools LangChain/LangGraph, CrewAI, OpenAI Agents SDK, LlamaIndex, and AutoGen

HTTP Vercel AI SDK, n8n, and any client that can call REST/OpenAPI

The full compatibility matrix separates machine-tested, machine-configured, and documentation-verified integrations. If your agent is missing, the adapter template provides a short contribution path.

Common workflows

Build a searchable video library

watch-skill batch ./recordings --limit 50
watch-skill library overview
watch-skill library ask "What did the team decide about authentication?"

library ask synthesizes evidence across videos and retains per-video timestamp provenance. The [library example](exam