The Problem
Switching between apps, searching files, running commands, and automating desktop tasks breaks focus and consumes time—especially for repetitive or complex workflows.
MULTI-AGENT · VOICE-FIRST · ACCESSIBILITY
ARIA – Desktop Vision Assistant
A voice-first, multi-agent AI assistant that sees your screen, understands natural language commands, and gets real work done on your desktop.
GitHub ↗Live Demo ↗
Stack
VOICE & SPEECH
Deepgram (STT)Cartesia (TTS)
AGENTS & ORCHESTRATION
LangGraphLangChainuAgents
LLM PROVIDERS
OpenAIClaudeGeminiOllama (Local)
DESKTOP & BROWSER
Stagehand (Browser Automation)AppleScript / JXA (macOS Control)
BACKEND & FRONTEND
Node.jsExpressSocket.ioReactTypeScriptViteElectronNext.js
MEMORY & STORAGE
ElasticsearchJSON (Session Store)
What I Built
ARIA is a voice-first, multi-agent assistant that listens, sees your screen, plans, acts, and remembers context—delivering accurate results through natural conversation.
The Difference
Combines computer vision, natural language understanding, and action orchestration with safety confirmations and persistent memory across sessions.
Numbers That Matter
Sub-secondvoice response experience
5specialized agents
Real-timescreen understanding & interaction
Safety Firstspoken confirmations for risky actions
Persistentmemory across sessions
Engineering Notes ✎
Vision Agent
Captures and analyzes screens to find UI elements, text, files, and visual context.
Planner Agent
Breaks complex goals into ordered steps with reasoning and execution plan.
Action Agent
Executes actions on the system or browser using safe, controlled tools and APIs.
Memory Agent
Stores context, files, and summaries for continuity across sessions.
Documentation Agent
Explains results, creates structured summaries, and answers follow-up questions.