V2

MistAI Desktop Assistant

Voice control, screen vision, and verified desktop automation — every action is checked against reality, not assumed to have worked.

Voice Control Screen Vision Real Browser Automation Verified Execution Live Captions

What is MistAI Desktop?

MistAI Desktop brings your AI assistant out of the browser and onto your system — listening for your voice, watching your screen, and taking action in real time.

V2 is a rebuild of the core execution model: every action now goes through a single path that settles, verifies against ground truth (was the file actually written? does the window actually have focus?), and only then reports success back to the model — instead of trusting that a click or a launch worked.

Built by Kristian, MistAI is intentionally not a fixed menu of hardcoded functions. The agent reasons about each task and decides its own steps; the actions below are the tools it has available, not a script it follows.

The Three Personalities

N

Nova Available

Fast, lightweight, creative. Powered by Gemini 2.5 Flash.

S

Sage Available

Strong reasoning and memory. Runs on Cohere's Command models, and is currently the desktop assistant's default brain.

F

Flux Temporarily degraded

Balanced speed and performance, powered by Mistral. Mistral's free tier is currently disabled upstream, so Flux requests are automatically served by Sage until that's restored.

Core Features

Wake Detection

Say "Mist" or "Hey Mist" to activate hands-free, with word-boundary matching so it doesn't trigger on unrelated words that merely contain similar sounds.

Screen Vision

Reads and clicks on native-app UI elements using OCR, computer vision, and the Windows accessibility tree — with an on-screen indicator showing exactly where and how a click was resolved.

Real Browser Automation

A dedicated, persistent browser instance reads and interacts with the actual page structure (DOM) instead of guessing from a screenshot.

Verified Execution

Every action is checked against ground truth after it runs — a claimed success that didn't actually happen is caught and reported, not trusted blindly.

Live Captions

Real-time overlays of what MistAI is saying and doing at the bottom of your screen.

App & Window Control

Opens, focuses, and manages applications and windows, distinguishing between "already open and visible" and "running but hidden" rather than treating both as success.

Multi-Phase Tasks

Larger goals are broken into a dependency graph of phases that can be paused and resumed by name — a failed phase only blocks the work that actually depends on it.

Persistent Memory

Remembers conversations, facts, and reminders, and tracks which strategies have worked or failed before so it doesn't repeat known dead ends.

Installation

Download MistAI Desktop

Available as a standalone executable for Windows. No Python installation required — just download and run.

Download Latest Release

Windows 10/11 (64-bit) · Built with PyInstaller

Platform support: V2's automation layer relies on Windows-specific APIs (window management, the accessibility tree, the registry) for desktop control. macOS/Linux support is a planned future project, not a current guarantee — see the Roadmap below.

Requirements

  • Windows 10/11 (64-bit)
  • Microphone — for voice control
  • Internet connection — required for AI processing
  • Vision OCR — bundled Tesseract engine, works out of the box on Windows
About screen vision: OCR is used as a fallback for native apps when the accessibility tree doesn't expose an element. Browser pages are read directly from the DOM and don't need OCR at all.

Caption System

The caption system provides real-time visual feedback for everything MistAI says and does. Captions appear at the bottom of your screen in non-intrusive overlays.

  • Assistant messages — green background for MistAI responses
  • System notifications — yellow background for status updates
  • Suggestions — purple background for smart tips
  • Privacy first — user messages are never shown in captions
Assistant: Opening Firefox now
System: Wake word detection enabled
Suggestion: Would you like me to maximize this window?

How to Use

Wake words

Mist Hey Mist MistAI Mist AI

Example commands

"Hey Mist, open Discord"
"Mist, what's my battery level?"
"Search YouTube for AI tutorials"
"Go to that site and tell me what it says"

Ending or interrupting

"Stop" — halts whatever MistAI is currently doing
"That's all" / "Nevermind" — ends the conversation, back to wake-word listening

What MistAI Can Do

A sample of the tools available to the agent — it chooses which to use and in what order for a given task, rather than following a fixed script.

open_app / switch_app — Open or focus an application
browser_navigate / click / fill — Real DOM-based browser control
uia_click — Click via the accessibility tree
click_on_text — OCR/vision fallback click
run_command — Direct PowerShell for system/file tasks
write_file — Create or edit files directly
read_screen_text — Answer a specific question about what's visible
remember / recall — Long-term memory of facts
window management — List, activate, close, resize windows
volume / brightness — System-level controls

MistAI API

Integrate MistAI into your applications using a simple REST API. Send messages, select a model, and receive intelligent responses.

🔐 Authentication

All API requests require an API key. Include it in the Authorization header using the Bearer scheme.

Base URL
https://api.mistai.org/api/v1
Headers
Authorization: Bearer YOUR_API_KEY Content-Type: application/json
You can generate and manage your API key from the dashboard.

⚡ Rate Limits

Free tier includes:

  • 30 requests per minute
  • No fixed monthly cap (fair use applies)
model: "mistral" is temporarily routed through Cohere while Mistral's free tier is disabled upstream. Nothing changes on your end — same request shape, same response shape.

💬 POST /api/v1/chat

Send a message to MistAI and receive a response from the selected model. This is the primary endpoint for interacting with the API.

Request Body
{ "message": "Explain quantum computing simply", "model": "cohere" }
Response
{ "success": true, "response": "Here's a simple explanation of quantum computing...", "model": "cohere", "timestamp": "2026-09-09T13:48:31.123456" }

🖥️ Using the API with MistAI Desktop

MistAI Desktop is powered by the same backend, meaning your API key can be used seamlessly across both the web and desktop environments.

  • Use your API key to connect the desktop assistant to your account
  • All voice commands and automation requests are processed through the API
  • Switch between Nova, Sage, and Flux directly from the desktop app
The desktop app sends its full task context and available tools as a structured request, so the model reasons over the actual current state of your screen and windows, not just a flattened prompt.

Frequently asked questions

Is MistAI free to use?
Yes — MistAI Desktop is completely free and open-source. The backend API is also free to use.
Does MistAI work offline?
No, an internet connection is required for the AI backend. Some local system actions (window management, file operations) still work without one, but nothing that requires reasoning does.
Is my data private?
MistAI sends voice commands, screen text, and relevant task context to the AI backend to decide what to do next. Screenshots used for on-device OCR are processed locally. Conversations are not stored permanently.
Can I use MistAI on Mac or Linux?
Not yet. V2's automation core depends on Windows-specific APIs for window and accessibility control. Cross-platform support is on the roadmap, not a current guarantee — see Roadmap below.
How accurate is wake word detection?
MistAI uses Google's speech recognition with word-boundary-aware matching, tuned to avoid triggering on unrelated words. It's most reliable in quiet environments; heavy background noise can still cause missed or mistranscribed wake attempts.
What's the difference between Nova, Sage, and Flux?
Nova: Fast and efficient for general tasks (Gemini 2.5 Flash)
Sage: Strong reasoning, the desktop assistant's default (Cohere Command)
Flux: Normally Mistral-backed; currently served by Sage while Mistral's free tier is down upstream.
Do I need to know Python?
No — the executable requires zero coding knowledge. Just download and run.
Can I contribute to the project?
Absolutely. MistAI is open-source — check the GitHub repo, submit issues, or open a pull request.

Troubleshooting

Wake word not responding

Check microphone permissions and ensure the wake word toggle is enabled in the UI. A single short word ("Mist" alone) is the hardest case for speech recognition — "Hey Mist" or "Mist AI" tend to be picked up more reliably.

OCR not finding text

OCR is a fallback for native apps only — browser pages are read from the DOM directly and don't use it. If a native-app click is missing, try describing the element differently, or ask MistAI to use the accessibility tree explicitly.

No text-to-speech

Check system volume settings. MistAI falls back to a secondary speech engine if the primary one can't reach the network, and will log an explicit warning if neither is available rather than staying silently mute.

Connection errors

Verify your internet connection. The status indicator in the UI shows connectivity. If one model provider is degraded, MistAI can route requests to a different one automatically.

Roadmap

Now — Stabilizing V2: concurrency safety, verified action execution, wake-word accuracy
Next — More generic perception tools (read / search files directly) so the agent relies less on guessing at system internals
Later — Local wake-word detection, replacing general speech recognition for the trigger word specifically
Later — Cross-platform support (macOS/Linux)
Exploring — Plugin system for community-added capabilities