Ambient Context Captures macOS Window Text for Local Agent Memory
Ambient Context is an open-source macOS menu bar app that periodically reads text from your focused window using the Accessibility API and saves it to daily Markdown files. Operating entirely locally without telemetry or video recordings, it generates clean context logs for tools like Claude Code.
Impact: Medium
Why it matters
You can provide Claude Code or Cursor with continuous local context without streaming screen video or uploading sensitive data to cloud services.
TL;DR
- 01Reads visible UI text via macOS Accessibility API without taking raw screenshots.
- 02Filters out password managers and scrubs API keys before writing Markdown to disk.
- 03Generates structured daily files in ~/Ambient Context formatted for Claude Code.
Key facts
- Storage Location
- ~/Ambient Context
- Capture Engine
- macOS Accessibility Tree
- Network Activity
- Zero Network Calls
- File Format
- Plaintext Markdown
Accessibility Tree Capture Without Screenshots
Unlike vision-based monitoring utilities that run OCR over screen buffers, Ambient Context queries the macOS accessibility tree directly. Every few seconds, the app inspects text nodes belonging strictly to the focused window. It ignores background applications, secondary displays, and locked desktop states, reducing CPU overhead and keeping capture logs compact.
Local Privacy and Secret Scrubbing
The application operates without external network requests or analytics pipelines. Sensitive data filtering is enforced prior to disk storage:
- Password manager applications and private browser windows are omitted at the accessibility layer.
- Secure text entry fields are skipped automatically.
- Regex scrubbing strips credit card patterns, API keys, and authorization header strings.
LLM-Optimized Daily Markdown Logs
Captured UI text is recorded in date-stamped files inside ~/Ambient Context. Identical lines encountered throughout the day are deduplicated to minimize context window consumption. Blocks are tagged with timestamps, application titles, and source URLs. The directory contains an AGENTS.md file that formats the logs for fast retrieval by Claude Code, Codex, or Cursor.
Try it in 2 minutes
git clone https://github.com/dragthelake/ambient-context && cd ambient-context && npm run tauri devbash
✓ When to use
- Creating automated daily standup reports with local AI tools like Claude Code.
- Logging development context without transmitting screen recordings to external cloud platforms.
- Maintaining a searchable offline timeline of focused documents and browser links.
What to do today
- Clone and build the ambient-context repository using Rust and Tauri on macOS.
- Grant Accessibility permissions in System Settings and configure the local capture directory.
- Point Claude Code at ~/Ambient Context to auto-generate standup notes and review work history.
What the community says
“I would assume that layer of abstraction -- looking at a fairly syandard application window with minor UI variations and reasoning about what area / labels mean what... could probably become a light-weight fine-tuned vision model”
“One of the reasons people like TUIs is because the text is always just right there.”
Sources