Chat With Your Specs and Bid Docs Locally (Ollama + Open WebUI)

Reviewed by the Contractor AI Shop research team · Last updated Oct 22, 2026 · How we research

By Contractor AI Shop · Last verified Oct 20, 2026 (PT)

Short answer: Yes. You can ask questions about your spec book and bid documents without uploading them to anyone's cloud: install Ollama, which runs an AI model on your own computer, and put Open WebUI in front of it as a ChatGPT-style screen that can search a "knowledge base" of your files. Both are free to run on your own hardware, and answers link to the passages they came from, but a local model can still miss or invent things, so test it on a spec you know and check every paragraph number before you rely on it.

How it works

Ollama runs the model and serves it on your machine at http://localhost:11434. Open WebUI is the chat screen. When you upload a document, it extracts the text, splits it into chunks and indexes them. When you ask a question, it finds the chunks that match and hands them to the model along with your question. This is called retrieval-augmented generation (RAG). The model works from the retrieved chunks, not the whole spec book at once, so the quality of the answer depends on what gets retrieved.

1. Install Ollama

• Mac: download from ollama.com/download/mac, open the file and drag Ollama into Applications, which is the method Ollama's macOS docs prefer. On first launch it offers to set up the ollama command.
• Windows: download the installer from ollama.com/download/windows, or run this in PowerShell: irm https://ollama.com/install.ps1 | iex (Windows docs).
• Linux or DGX Spark: run curl -fsSL https://ollama.com/install.sh | sh (Linux docs). On a Spark, NVIDIA's playbooks cover the same setup; see our DGX Spark setup guide.

Check that it installed:

ollama -v

2. Pull a model sized to your machine

Start small to prove the setup works, then step up. The picks below come from the document Q&A column of our models vs hardware guide. They are editorial suggestions, not benchmark results. Sizes are Ollama download sizes, checked Oct 20, 2026.

Pull the model and ask it a simple question:

ollama pull llama3.1:8b
ollama run llama3.1:8b

If it answers, your local AI is working. Then run ollama ps while the model is loaded. "100% GPU" in the PROCESSOR column means the model fits in GPU memory, and the CONTEXT column shows how much text it can hold at once.

Stay local: Ollama also offers cloud models, whose names end in :cloud. Those run on Ollama's servers, so don't use them for bid documents. To turn Ollama's cloud features off completely, set OLLAMA_NO_CLOUD=1 (or "disable_ollama_cloud": true in ~/.ollama/server.json) and restart Ollama (Ollama FAQ).

3. Run Open WebUI

With Docker, generate a secret key with openssl rand -hex 32, put it in place of your-secret-key, and run the command from Open WebUI's quick start:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Then open http://localhost:3000 in your browser. Your chats, users, settings and uploaded files live in the open-webui volume, so keep it and back it up before every update. Keep the same secret key too; a new key signs everyone out.

• On a Mac, run Ollama as the normal app and only Open WebUI in Docker. Ollama's docs say GPU acceleration isn't available inside Docker Desktop on macOS.
• On Windows, Open WebUI's docs run Docker Desktop with the WSL 2 backend.
• Without Docker, Open WebUI supports Python 3.11 and 3.12 (not 3.13). One option from its docs is DATA_DIR=~/.open-webui uvx --python 3.11 open-webui@latest serve, which serves it at http://localhost:8080.

4. Create the admin account and connect the model

The first account you create is the administrator, and sign-ups switch off after that. Open WebUI looks for Ollama on its own: at http://host.docker.internal:11434 from Docker, or http://localhost:11434 from Python. Check under your avatar > Settings > Admin > Connections. If no models show up, Open WebUI's docs say Ollama on the host has to listen on 0.0.0.0 when Open WebUI runs in Docker. Read the privacy notes below before you change that.

Raise the context length. Ollama picks a default context length from GPU memory: 4K tokens under 24GiB, 32K from 24 to 48GiB and 256K at 48GiB and up (Ollama context length docs). Four thousand tokens leaves little room for retrieved spec text. In Open WebUI, go to Settings > Admin > Models, click the pencil on your model, open Advanced Params and set the context length to at least 8192, or above 16,000 if your model and memory allow. A larger context uses more memory, so check ollama ps again afterward.

5. Put a spec section or bid docs into a knowledge base

1. Go to Workspace > Knowledge and click Create. Give it a name you'll recognize, such as the job name and "Div 07 specs."
2. Upload one spec section or bid document you know well. Addenda, bid forms and scope sheets work the same way. Wait until processing finishes.
3. Open the file and switch to Indexed text. That is the text Open WebUI extracted, and it's what the model works from. If it's blank or scrambled, the answers will be too. Scanned or image-only pages can come through empty with the default extractor; Open WebUI's docs suggest switching to Apache Tika or Docling, or turning on PDF image extraction (OCR), under Settings > Admin > Documents. Check tables such as schedules and bid forms here as well.

6. Test it with questions you already know the answers to

Before you ask anything, write down the answers you expect. Then start a new chat, pick your model, type # and select your knowledge base so it's attached to the chat. Ask three questions you can check:

• A submittal requirement ("What submittals does this section require?")
• A scope exclusion ("What does this section say is by others or not in contract?")
• A testing requirement ("What field testing or mock-ups are required?")

Then try a full scope check:

You are helping a [trade] estimator. Read the spec section below. List: (1) every item of work that falls in our scope, (2) items that mention another trade or "by others," (3) submittals, testing or mock-ups required, (4) anything ambiguous we should send as an RFI. Quote the paragraph number for each item. If something is unclear, say so instead of guessing.

7. Check every cited paragraph number

Click the sources under each answer and compare them with the original document. Mark each item right, wrong, missed or made up. A source link shows which passage was retrieved; it doesn't prove the model read it correctly.

If it fails your test questions, try these in order:

1. A larger model, if your memory allows (step 2).
2. A longer context length (step 4).
3. Full Context for one short section. Click the attached knowledge base and choose "Use Entire Document" so the whole section is sent instead of retrieved chunks. It only works if the section fits in the model's context.
4. Smaller chunks. For local models with about 8K of context, Open WebUI's RAG troubleshooting page suggests the token text splitter, chunk size 1000, overlap 100 and Top K of 3 to 5, under Settings > Admin > Documents.

Only move on to the next spec once it passes your three questions. If it never does, keep it for first-pass summaries and read the spec yourself.

8. Optional: share it with the office

• Teammates on the same network or VPN can open http://<that computer's IP address>:3000.
• Turn on New Sign Ups under Settings > Admin > Authentication. New accounts wait as Pending until you approve them in the Admin Panel.
• Knowledge bases aren't shared automatically, and by default regular users can't create them. Create groups and grant access to the right knowledge bases and models.
• Sign in as a regular user to confirm they see what they should, and nothing more.

What it gets wrong (limits)

• It only knows what was retrieved. If the answer depends on two places, such as Division 01 requirements that change a technical section, check whether it pulled both.
• Paragraph numbers can be wrong or invented. Small local models misread and make things up. That's why step 7 exists.
• It reads extracted text, not drawings. This setup is for specs and bid documents. It isn't a takeoff tool, and we haven't tested it on plan sheets.
• The default context is short on machines with less GPU memory (step 4).
• You're the IT department. Open WebUI updates often: it shipped four releases between Oct 10 and Oct 16, 2026. Back up the data volume before every update, and re-test your three questions after one.
• Speed depends on your hardware. We don't quote speeds here; our models vs hardware guide lists measured ones with their sources.
• It doesn't replace reading the spec. The bid is still yours.

Privacy notes

• Ollama runs locally. Its FAQ says it doesn't see your prompts or data when you run models locally.
• Ollama's local API has no login. It listens only on your own computer (127.0.0.1, port 11434) by default. If you change OLLAMA_HOST to 0.0.0.0, anyone who can reach that port on your network can use your models, so keep it behind your firewall.
• Keep Open WebUI on your LAN or a VPN, not the open internet. Open WebUI's hardening guide says it's built for private, trusted networks and recommends a VPN (such as WireGuard or Tailscale), a zero-trust access proxy, or a reverse proxy with authentication. Don't port-forward it from your router.
• Use real user accounts. Approve each person, use groups, and tell your team that admins can view all user chats by default (ENABLE_ADMIN_CHAT_ACCESS).
• Keep every step local. Where document text is processed depends on the extraction, embedding and model providers you choose. With the standard image, the default embedding model runs locally; leave the extractor, embeddings and model all on your hardware.
• Turn off what you don't need. A stock Open WebUI install checks GitHub for new versions, asks the default OpenAI connection for a model list and checks Hugging Face for embedding-model updates. The quick start explains how to switch each one off (ENABLE_VERSION_UPDATE_CHECK=false, delete the OpenAI connection or set ENABLE_OPENAI_API=false, and RAG_EMBEDDING_MODEL_AUTO_UPDATE=false). Set ENABLE_COMMUNITY_SHARING=false to hide the options that send chats to the Open WebUI Community site.
• Encrypt the disk (FileVault, BitLocker or LUKS), as Open WebUI's hardening guide recommends for the default SQLite database.
• Check your documents' terms. Some owners and GCs put confidentiality terms on bid documents. Read them before you load the files anywhere, local or not.

When it's worth paying

Local AI is private and has no per-question fees, but it costs setup time, upkeep and hardware. Paying may make sense when:

• Nobody in the office wants to maintain it. Someone has to install updates, approve accounts and re-test.
• You need more than your machine can run. Larger models and more simultaneous users need more memory. A Mac Studio or NVIDIA DGX Spark is the hardware route.
• You want a tool built for construction documents. Our directory lists paid tools such as Trunk Tools (answers questions from project docs, specs and drawings), Document Crunch (flags risk in contracts and specs) and Datagrid (agents that read specs and RFPs). We haven't tested them against this setup. Check pricing and data terms on each vendor's site.
• You'd rather use a cloud assistant. ChatGPT and Claude need no setup on your end. Read their data-handling terms for your plan before you upload bid documents.

Try the free route on one spec first. That tells you what a paid tool would need to do better.

FAQ

Can I chat with a PDF locally without uploading it to the cloud?

Yes. Ollama runs the model on your computer and Open WebUI stores and searches your PDFs on the machine you install it on, so the files don't leave your hardware as long as you use a local model (not one ending in :cloud) and keep document extraction and embeddings local.

Are Ollama and Open WebUI really free for a contracting business?

The software is free to run on your own hardware. Ollama is MIT-licensed. Open WebUI is free under its own license as long as you keep its branding visible. Docker Desktop, if you use it, is free for businesses with fewer than 250 employees and under $10 million in annual revenue. Each model has its own license; for example, Ollama lists gpt-oss under Apache 2.0. Your costs are the computer, the electricity and your time.

What computer do I need to run AI on my spec book?

Start with the computer you have. llama3.1:8b is a 4.9GB download, and our models vs hardware guide shows it fitting on a 16GB Mac. For longer documents and better answers, more memory helps: that guide matches models to machines from 16GB Macs up to 128GB systems such as the NVIDIA DGX Spark.

How accurate is local AI at reading construction specs?

We don't publish an accuracy number, because it depends on the model, the document and how the text was extracted. Test it yourself: ask three questions you already know the answers to, and check every paragraph number it cites against the spec before you trust it on a bid.

Can my whole office use one local AI setup?

Yes. Open WebUI has user accounts: the first account is the admin, new accounts wait for approval, and you grant groups access to specific knowledge bases. Keep it on your office network or a VPN, not the open internet.

Can it read scanned specs or drawings?

Not reliably out of the box. Open WebUI reads the text it can extract, and scanned or image-only pages can come through empty with the default extractor. Check the Indexed text view, and switch to an OCR-capable extractor such as Apache Tika or Docling if pages are blank. This setup isn't built for reading plan sheets.

Related guides and tools

• Local AI on a Mac Studio with oMLX and MLX
• NVIDIA DGX Spark setup for local AI
• Which local AI models fit on which hardware
• Tools: Ollama · Open WebUI · AnythingLLM (another local option with document chat) · LM Studio · n8n (to automate filing new specs later)

Last verified: Oct 20, 2026 (PT), against the Ollama docs (latest release v0.40.6) and Open WebUI docs (latest release v0.12.3). Commands and menu names change between releases; follow the linked official docs if something looks different.

Independent guide by Contractor AI Shop. Not affiliated with Ollama, Open WebUI, Docker or any model publisher. This page has no sponsored or affiliate links.