NewsMacroAndrew Ng Announces OpenWorker, an Open-Source Local-First Desktop AI Coworker

Andrew Ng Announces OpenWorker, an Open-Source Local-First Desktop AI Coworker

Author: MarkTechPost·

Key Takeaways

  • OpenWorker is an open-source, MIT-licensed desktop agent created by Andrew Ng that produces finished deliverables rather than chat replies.
  • The system runs entirely on the user's machine using a four-layer architecture comprising a Tauri 2 desktop shell, a local Python FastAPI agent server, a capability and connector layer, and a model router.
  • Users must supply their own API keys or local runtimes to access 30 curated models from providers such as OpenAI, Anthropic, Google, and Ollama.
  • A typed risk engine assigns every tool call to one of four risk classes—read, write_local, exec, or external—and governs execution across five permission modes.
  • The local-first privacy model keeps conversations, connector tokens, and model keys on the user's device, with an optional cloud broker used solely for OAuth handshakes.
Andrew Ng Announces OpenWorker, an Open-Source Local-First Desktop AI Coworker

Andrew Ng has announced OpenWorker, an open-source desktop agent designed to produce finished work rather than chat-based responses. Ng is the founder of DeepLearning.AI and previously led AI research at Google Brain and Baidu, and he co-founded Coursera. The announcement was shared on X at https://x.com/AndrewYNg/status/2080333504446108104.

OpenWorker is built around outcomes rather than prompts. A user can ask for a polished document, a Slack reply containing actual numbers, an updated calendar, or a triaged inbox. The system then breaks the requested outcome into steps, operates across local files and connected applications, and checks in with the user before taking consequential actions. This places OpenWorker in the emerging category of agentic AI tools that complete multi-step tasks across applications, rather than only generating text responses.

Four-layer architecture running on the local machine

OpenWorker's architecture is organized into four layers, all of which run on the user's machine. The repository contains 119 Python files, totaling approximately 32,400 lines, under coworker/; 149 TypeScript/TSX files under surfaces/gui/; and 78 backend test modules.

The desktop shell is a Tauri 2 native window that wraps a React 18 user interface. Tauri is an open-source framework for building lightweight desktop applications using web frontends with native system access. Its bundle identifier is com.openworker.desktop, and the shell supervises the Python server itself.

The local agent server uses Python 3.10+ with FastAPI and uvicorn. By default, it binds to 127.0.0.1:8765. The example configuration limits a single turn to 12 model tool iterations.

The capability and connector layer includes vetted local tools for files, git, ripgrep-backed search, shell access, and todo management, along with hosted integrations and MCP, the Model Context Protocol: MCP is an open standard introduced by Anthropic in late 2024 for connecting AI assistants to external data sources and tools through a common protocol.

The model router provides one interface across native, OpenAI-compatible, reseller, and local providers. The engine is built on aisuite, Andrew Ng's provider-agnostic LLM library: https://github.com/andrewyng/aisuite. aisuite provides a unified interface for calling models from multiple providers without changing application code.

Bring-your-own-model access

OpenWorker does not provide its own inference service. Users either paste an API key into the app or point it to a local runtime, which means users pay providers directly and retain control over which models process their data.

The curated model matrix contains exactly 30 entries. Native providers include OpenAI, with GPT-5.6 Sol, GPT-5.6 Terra, GPT-5.6 Luna, and GPT-5.5; Anthropic, with Claude Fable 5, Opus 4.8, Sonnet 4.6, and Haiku 4.5; and Google, with Gemini 3.1 Pro, 3.6 Flash, 2.5 Pro, and 2.5 Flash.

OpenAI-compatible vendors add GLM-5.2, DeepSeek V4, Kimi K2.6, MiniMax M2.5, Qwen3 Max, Grok 4.3, and Mistral Large. Open-weight models are available through Together AI, and Fireworks, Fully local models are available through Ollama, which requires no API key.

Permission system and risk classes

OpenWorker treats approvals as a typed layer rather than only a user-interface feature. Each tool call is assigned to one of four risk classes: read, for actions with no side effects; write_local, for path-scoped changes to the workspace; exec, for running commands; and external, for actions with side effects outside the machine.

Five permission modes determine how those actions are handled. discuss and plan are read-only modes. interactive is the default and requests approval before writes, commands, and external actions. auto allows all actions while remaining path-scoped. custom auto-approves a user-defined list of tools.

Two design choices define the approval model. First, unattended mode does not increase the level of autonomy; it only changes where the user is contacted. Prompts that would normally appear inline are routed to an Inbox, and the session is suspended until the user responds.

Second, task-scoped standing rules are limited to external risk. Shell commands continue to require approval by design.

The built-in ops persona also directs the model to treat content from tools, logs, the web, files, and incoming messages as untrusted data rather than instructions. That prompt-injection posture is written into the shipped persona. Treating external content as untrusted data is a defense against indirect prompt injection, an attack vector in which malicious instructions embedded in data sources attempt to manipulate an agent's behavior.

Local-first privacy model

Model calls are sent directly from the user's machine to the configured provider. Conversations, connector tokens, and model keys remain local. The secret store is designed so that secrets do not enter the model's context, prompts, or traces.

The only cloud component is an optional broker for OAuth handshakes used by one-click connectors. It uses Auth0 Authorization Code with PKCE. Connector tokens are handed directly to the user's machine and are not stored in the cloud. The app remains fully functional when signed out by using manually pasted credentials.

OpenWorker is MIT-licensed and positioned as a desktop AI coworker that returns completed deliverables rather than chat replies. Its stack combines a Tauri 2 and React shell with a local Python FastAPI agent server built on aisuite. Model access is bring-your-own-key across 30 curated tool-calling models, with fully local access available through Ollama. A typed risk engine using read, write_local, exec, and external gates actions across five permission modes.

The OpenWorker GitHub repository is available at https://github.com/andrewyng/openworker, and the project site is