I Installed an AI Brain on My Computer in 20 Minutes
I’ve been planning a local AI infrastructure project for months. Researching models, benchmarking hardware, writing strategy documents. The whole enterprise planning exercise.
Then I sat down one afternoon and just… did it.
Twenty minutes later, I had a 7-billion-parameter language model running on my desktop. No cloud. No subscription. No data leaving my building.
Here’s exactly how.
What You Need
- A computer with at least 16GB of RAM (I have 64GB, but 16 works)
- An internet connection (for the initial download only)
- 20 minutes
That’s it. No special hardware. No GPU required. My workstation has a basic Radeon with 4GB — everything runs on CPU.
Step 1: Install Ollama (2 minutes)
Go to ollama.com. Download the installer. Run it. Done.
Ollama is the engine that runs AI models locally. It’s free, open source, and works on Windows, Mac, and Linux.
Step 2: Download a Model (5-10 minutes)
Open a terminal and type:
ollama pull qwen2.5:7b
This downloads a 4.7GB model called Qwen 2.5, built by Alibaba. It’s free, open-source, and one of the best models in its size class.
The download takes a few minutes depending on your internet speed.
Step 3: Talk to It (immediately)
ollama run qwen2.5:7b
That’s it. You’re now chatting with a local AI.
I asked it to summarize a meeting transcript. It did. I asked it to categorize a list of expenses. It did. I asked it to draft an email. It did.
Not as eloquently as Claude Opus. But faster, and without sending a single byte of data to anyone’s server.
What Surprised Me
Speed. On my 16-core Ryzen, responses come back in seconds. Not instant like a cloud API with dedicated hardware, but fast enough for real work.
Quality. The 7B model handles summarization, classification, data extraction, and simple drafting better than I expected. The 14B model (also free, also local) handles complex reasoning.
Simplicity. I was prepared for dependency hell, configuration files, Docker containers. Instead: one install, one command, working AI. The ecosystem grew up while I was planning.
What It Won’t Do
Be honest about limitations:
- It’s not Claude. Complex multi-step reasoning, creative writing, and nuanced analysis — that still belongs to frontier cloud models.
- No internet. It can’t search the web or access external APIs (though you can build that).
- No memory between sessions. Each conversation starts fresh (unless you build persistence around it).
Why This Matters for Business
Three words: air-gapped AI capability.
Some industries — healthcare, legal, financial services — have legitimate concerns about sending data to cloud AI providers. “What happens to my client data?” is a valid question with no simple answer.
Local AI eliminates the question entirely. The data never leaves the machine. There’s no terms of service to parse. No data retention policy to audit. The model runs on your hardware, processes your data, and that’s where it stays.
For FIT clients, this opens a new service tier: AI assistance that works inside the client’s own network. No cloud dependency. No external risk.
The Bigger Picture
A year ago, running a capable language model locally required a $1,500+ GPU and significant technical expertise. Today it requires a download and a terminal command.
The tools moved faster than my plans. I had a whole infrastructure project scoped out — GPU upgrades, Kubernetes orchestration, multi-tenant deployment. And the ecosystem just… caught up to me.
That’s the pattern with AI right now. By the time you finish planning, the next version is already easier than what you planned for.
My advice: stop planning. Start installing. You can architect later.
Try It Yourself
- Go to ollama.com
- Install it
- Run
ollama pull qwen2.5:7b - Run
ollama run qwen2.5:7b - Ask it something
Total time: 20 minutes. Total cost: $0.
If you want help integrating local AI into your business workflow, reach out to FIT. We help small businesses and nonprofits set up both cloud and local AI — whichever makes sense for your situation.
Matt Stoltz is the founder of Flower Insider Technologies in southern Minnesota. He runs four AI assistants, three local models, and still thinks the TI-85 was ahead of its time.