Local AI on a Mac Studio with oMLX and MLX
Reviewed by the Contractor AI Shop research team · Last updated Oct 22, 2026 · How we research
Why: keep bids, specs and pricing private, with no per-token fees. Apple silicon unified memory lets a Mac Studio hold large models. This guide uses oMLX, a free, open-source (Apache-2.0) model server built on Apple's MLX framework, with mlx-lm as a lighter option. We run local models this way in our own contracting business.
1. Hardware
oMLX needs Apple silicon (M1 or later) and macOS 15 or later. Its site lists 16GB RAM as the minimum and recommends 64GB or more for larger models (omlx.ai). For which model sizes fit in your memory, see Which local AI models fit on which hardware. Hardware: Apple Mac Studio.
2. Install oMLX
Either download the .dmg from oMLX Releases and drag it to Applications (the app includes in-app updates and a menu-bar control), or install with Homebrew, as documented in the oMLX README:
brew tap jundot/omlx https://github.com/jundot/omlx brew install jundot/omlx/omlx
The README also offers brew install jundot/omlx/omlx --with-custom-kernel, which builds native custom kernels and needs full Xcode. To upgrade later: brew update && brew upgrade omlx.
3. Download an MLX model
oMLX and mlx-lm run models in MLX format. Thousands of ready-converted, quantized models are in the mlx-community organization on Hugging Face, for example mlx-community/Llama-3.2-3B-Instruct-4bit (the mlx-lm default model). Two ways to get one:
- From the oMLX dashboard. The admin dashboard (at
http://localhost:8000/adminonce the server runs) can search Hugging Face and download models directly. In the Mac app, the Welcome screen walks you through choosing a model directory, starting the server and downloading a first model. - From the terminal with the Hugging Face CLI (
brew install hf), into a subfolder of your model directory:
hf download mlx-community/Llama-3.2-3B-Instruct-4bit --local-dir ~/models/Llama-3.2-3B-Instruct-4bit
oMLX treats each subfolder of the model directory as one model, and also accepts two-level folders such as mlx-community/model-name/.
4. Serve an OpenAI-compatible endpoint
Run the server in the foreground, pointed at your model directory:
omlx serve --model-dir ~/models
Or run it as a managed background service with omlx start (and omlx stop, omlx restart). The background service uses ~/.omlx/models and port 8000 by default. Any OpenAI-compatible client can then connect to http://localhost:8000/v1, and oMLX also has a built-in chat page at http://localhost:8000/admin/chat. To check which model IDs the server exposes:
curl http://127.0.0.1:8000/v1/models
To share it with other computers in the office, the README says to set an API key first; oMLX refuses to start on a non-loopback address without one:
OMLX_API_KEY=your-secret-key omlx serve --model-dir ~/models --host 0.0.0.0
5. Connect Open WebUI
Install Open WebUI with Python (it supports Python 3.11 and 3.12, not 3.13). Its default port is 8080, so this uses 3000 to stay clear of mlx-lm's default port in step 6:
pip install open-webui open-webui serve --port 3000
Open http://localhost:3000, create the admin account, then go to your avatar > Settings > Admin > Connections. Click the + next to Manage OpenAI API Connections, enter http://localhost:8000/v1 (and your oMLX API key if you set one), then Save. If you run Open WebUI in Docker instead, its quick start uses --add-host=host.docker.internal:host-gateway so the container can reach servers on your Mac; use http://host.docker.internal:8000/v1 as the URL in that case. If the model list stays empty, the Open WebUI docs say some providers need model IDs entered on the connection; use the IDs from /v1/models. Then upload specs or RFIs to a chat for Q&A, and check every answer against the document.
6. Lighter option: mlx-lm on its own
mlx-lm (MIT license, from Apple's ml-explore team) is a Python package for running and fine-tuning models with MLX. Install and chat in the terminal:
pip install mlx-lm mlx_lm.chat
To serve an OpenAI-style HTTP API (it downloads the model from Hugging Face if it is not already cached):
mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit
This listens on localhost port 8080, so in Open WebUI use http://localhost:8080/v1. The mlx-lm server docs say it is not recommended for production because it only implements basic security checks. Run mlx_lm.server --help for all options.
7. If your model is not supported yet
oMLX runs any text model that mlx-lm supports (vision models come through mlx-vlm). A brand-new model may not be supported until mlx-lm adds its architecture; the mlx-lm README asks you to file an issue in that case. Check mlx-community for an MLX conversion first. If the model is supported but nobody has converted it, mlx-lm can convert and quantize it, for example mlx_lm.convert --model mistralai/Mistral-7B-Instruct-v0.3 -q. Some models also need --trust-remote-code.
8. Alternatives
Ollama and LM Studio also run local models on a Mac and work with Open WebUI.