01
See images inside your existing Agent
Once connected through MCP, Codex, Claude Code, Cursor, and other clients can work with images in the same conversation.
Product overview
Open Source Multimodal Vision Service for AI Agents (MCP)
Shadow Vision
v0.1.1 Open Source MCP
Help a text-first AI understand images
When a text model replies “I can’t see the image” after you share a screenshot, Shadow Vision adds vision tools through MCP so you can keep working in the Agent you already use.
01 / The outcome
How it helps
02
Do not get locked to one model
Switch between Ollama, OpenAI-compatible, Anthropic, and Gemini backends as the task, cost, or quality changes.
03
Keep sensitive images on your device
With Ollama, the image and inference stay local, which gives internal screenshots and private documents a clearer boundary.
04
Bring images from where they already are
Use a local file, base64 data, or a remote image URL, with safety limits around remote requests.
03 / What it can do
From seeing to doing
The Agent gets more than a description: it gets structured results it can inspect, compare, and turn into the next step.
01
vision_ocrPull text from screenshots, receipts, documents, and tables so the Agent can search, organize, and cite it.
02
vision_inspectAsk “what is wrong with this interface?” and let the Agent understand the scene before it explains or checks it.
03
vision_annotateRead circles, arrows, underlines, and highlights so the Agent can connect your marked point to the right object.
04
vision_layoutBreak an interface into elements, positions, text styles, and relationships for review, debugging, or code generation.
05
vision_reconstructGenerate an HTML, React, or SVG draft from a reference screenshot instead of starting from a blank page.
06
vision_comparePut before-and-after images, frames, or related documents together and surface what changed and why it matters.
04 / Choose your data boundary
Choose the vision source that fits your data
Local models, cloud models, and compatible services all use the same MCP entry point.
01
Ollama
Keep images and inference local for code screenshots, internal documents, and other content you do not want to send out.
02
OpenAI Compatible
Already using LM Studio, vLLM, or another OpenAI-protocol service? Change the configuration, not the client.
03
Anthropic
Choose Claude vision models when the task needs deeper image understanding and reasoning.
04
Gemini
Choose Gemini when the work benefits from broader multimodal understanding.
05 / Works with your Agent
Keep your Agent; give it sight
Connect once, then use the same visual entry point from Codex, Claude Code, Cursor, and more.
One shared start command: uv run shadow-vision
Codex
Claude Code
Cursor
VS Code Copilot
Windsurf
Claude Desktop
OpenCode
02 / Use cases
Use Cases
01
Developers
Give an Agent an error screenshot, wireframe, or design and let it understand, review, explain, or turn it into code.
02
Research and document work
Extract information from paper charts, scanned documents, and tables without the usual screenshot-and-copy loop.
03
When images should stay private
Use a local Ollama backend for sensitive screenshots and internal documents, keeping the data boundary on your device.
06 / Questions
Frequently Asked Questions
Clear answers regarding features, compatibility, and privacy
What is Shadow Vision and how does it integrate with AI Agents?
Shadow Vision is an open-source Model Context Protocol (MCP) visual service. It equips AI coding agents (Cursor, Claude Code, Codex) with 6 structured vision tools: screen OCR, UI layout parsing, visual difference inspection, annotation reading, and UI-to-code reconstruction.
Can Shadow Vision operate completely offline for sensitive code and UI data?
Yes. Shadow Vision natively supports local Ollama vision backends (e.g. Qwen2-VL, LLaVA). All screenshots and inferences remain strictly on your local machine, keeping sensitive internal designs and code safe.
Which vision model backends are supported?
It supports pluggable backends including local Ollama, OpenAI-compatible servers (vLLM, LM Studio, DeepSeek), Anthropic Claude 3.5 Sonnet, and Google Gemini multimodal models, selectable via straightforward configuration.
“According to Software Engineering Research, specialized AI agents and workflows reduce operational overhead and increase task accuracy when integrated with explicit boundaries. Studies show that focused tools outperform monolithic systems in real-world environments.”