# Local AI on a Mac Studio with oMLX and MLX

Reviewed by the Contractor AI Shop research team · Last updated Oct 28, 2026  
HTML: https://contractoraishop.com/guides/local-ai-mac-studio-mlx

**Why:** keep bids, specs and pricing private, with no per-token fees. Apple silicon unified memory lets a Mac Studio hold large models. This guide uses [oMLX](https://github.com/jundot/omlx), a free, open-source (Apache-2.0) model server built on Apple's [MLX](https://github.com/ml-explore/mlx) framework, with [mlx-lm](https://github.com/ml-explore/mlx-lm) as a lighter option. We run local models this way in our own contracting business.

## 1. Hardware

oMLX needs Apple silicon (M1 or later) and macOS 15 or later. Its site lists 16GB RAM as the minimum and recommends 64GB or more for larger models ([omlx.ai](https://omlx.ai/)). For which model sizes fit in your memory, see [Which local AI models fit on which hardware](https://contractoraishop.com/guides/local-ai-models-hardware-fit). Hardware: [Apple Mac Studio](https://www.apple.com/mac-studio/).

## 2. Install oMLX

Either download the `.dmg` from [oMLX Releases](https://github.com/jundot/omlx/releases) and drag it to Applications (the app includes in-app updates and a menu-bar control), or install with Homebrew, as documented in the [oMLX README](https://github.com/jundot/omlx#install):

```
brew tap jundot/omlx https://github.com/jundot/omlx
brew install jundot/omlx/omlx
```

The README also offers `brew install jundot/omlx/omlx --with-custom-kernel`, which builds native custom kernels and needs full Xcode. To upgrade later: `brew update && brew upgrade omlx`.

## 3. Download an MLX model

oMLX and mlx-lm run models in MLX format. Thousands of ready-converted, quantized models are in the [mlx-community](https://huggingface.co/mlx-community) organization on Hugging Face, for example [mlx-community/Llama-3.2-3B-Instruct-4bit](https://huggingface.co/mlx-community/Llama-3.2-3B-Instruct-4bit) (the mlx-lm default model). Two ways to get one:

- **From the oMLX dashboard.** The admin dashboard (at `http://localhost:8000/admin` once the server runs) can search Hugging Face and download models directly. In the Mac app, the Welcome screen walks you through choosing a model directory, starting the server and downloading a first model.
- **From the terminal** with the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) (`brew install hf`), into a subfolder of your model directory:

```
hf download mlx-community/Llama-3.2-3B-Instruct-4bit --local-dir ~/models/Llama-3.2-3B-Instruct-4bit
```

oMLX treats each subfolder of the model directory as one model, and also accepts two-level folders such as `mlx-community/model-name/`.

## 4. Serve an OpenAI-compatible endpoint

Run the server in the foreground, pointed at your model directory:

```
omlx serve --model-dir ~/models
```

Or run it as a managed background service with `omlx start` (and `omlx stop`, `omlx restart`). The background service uses `~/.omlx/models` and port 8000 by default. Any OpenAI-compatible client can then connect to `http://localhost:8000/v1`, and oMLX also has a built-in chat page at `http://localhost:8000/admin/chat`. To check which model IDs the server exposes:

```
curl http://127.0.0.1:8000/v1/models
```

To share it with other computers in the office, the README says to set an API key first; oMLX refuses to start on a non-loopback address without one:

```
OMLX_API_KEY=your-secret-key omlx serve --model-dir ~/models --host 0.0.0.0
```

## 5. Connect Open WebUI

Install [Open WebUI](https://docs.openwebui.com/getting-started/quick-start/) with Python (it supports Python 3.11 and 3.12, not 3.13). Its default port is 8080, so this uses 3000 to stay clear of mlx-lm's default port in step 6:

```
pip install open-webui
open-webui serve --port 3000
```

Open `http://localhost:3000`, create the admin account, then go to your avatar > Settings > Admin > Connections. Click the + next to Manage OpenAI API Connections, enter `http://localhost:8000/v1` (and your oMLX API key if you set one), then Save. If you run Open WebUI in Docker instead, its quick start uses `--add-host=host.docker.internal:host-gateway` so the container can reach servers on your Mac; use `http://host.docker.internal:8000/v1` as the URL in that case. If the model list stays empty, the Open WebUI docs say some providers need model IDs entered on the connection; use the IDs from `/v1/models`. Then upload specs or RFIs to a chat for Q&A, and check every answer against the document.

## 6. Lighter option: mlx-lm on its own

[mlx-lm](https://github.com/ml-explore/mlx-lm) (MIT license, from Apple's ml-explore team) is a Python package for running and fine-tuning models with MLX. Install and chat in the terminal:

```
pip install mlx-lm
mlx_lm.chat
```

To serve an OpenAI-style HTTP API (it downloads the model from Hugging Face if it is not already cached):

```
mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit
```

This listens on `localhost` port 8080, so in Open WebUI use `http://localhost:8080/v1`. The [mlx-lm server docs](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md) say it is not recommended for production because it only implements basic security checks. Run `mlx_lm.server --help` for all options.

## 7. If your model is not supported yet

oMLX runs any text model that mlx-lm supports (vision models come through mlx-vlm). A brand-new model may not be supported until mlx-lm adds its architecture; the mlx-lm README asks you to file an issue in that case. Check [mlx-community](https://huggingface.co/mlx-community) for an MLX conversion first. If the model is supported but nobody has converted it, mlx-lm can convert and quantize it, for example `mlx_lm.convert --model mistralai/Mistral-7B-Instruct-v0.3 -q`. Some models also need `--trust-remote-code`.

## 8. Alternatives

[Ollama](https://ollama.com/download/mac) and [LM Studio](https://lmstudio.ai) also run local models on a Mac and work with Open WebUI.

Independent guide by Contractor AI Shop. Not affiliated with listed vendors. Follow the linked official docs for current steps.