Omlx AI

Omlx AI, listed as oMLX, is a native macOS inference server designed for Apple Silicon Macs and built on MLX. It uses paged SSD KV caching to cut first response time for agents from 30 to 90 seconds down to under 5 seconds, which avoids the cache invalidation and recomputation that context switching usually causes. Continuous batching improves throughput for concurrent requests by up to 4.14 times. The server exposes OpenAI and Anthropic compatible APIs, so existing tools connect without rewrites, and it can serve LLMs, vision language models, embeddings, and re rankers at the same time. The project uses the Apache 2.0 license, and the page does not list pricing. The focus is local performance rather than cloud hosting. For developers who run agents locally, that removes the wait that usually makes local models painful for interactive work.

- Serve large language models locally on a Mac

- Cut first response time for agent workloads

- Run several model types from one server

- Connect tools through OpenAI compatible APIs

- Mac users who want local model inference

- Developers building coding agent workflows

- Teams that need compatible local APIs

It only runs on macOS with Apple Silicon, so Windows and Linux machines are out of scope. The page does not list pricing, so treat cost questions as unconfirmed and check the project terms yourself. Performance gains depend on your hardware and the models you load, and SSD space matters because the cache lives there. Setup assumes some familiarity with local servers and API endpoints. Confirm compatibility with your current toolchain before moving work onto it, and test response times on your own machine. Caching benefits shrink if storage is nearly full. Model downloads take disk space, and larger models need more memory, so check your machine before loading everything at once. Keep models updated as the project changes.