How to Run AI Models Locally with Ollama's New Decision Models — A Practical Guide for Small Businesses and Freelancers

By Arya

Ollama's new decision models let you classify, score, and sort data locally — no cloud, no API fees, full privacy. Here's how to actually set them up and use them.

How to Run AI Models Locally with Ollama's New Decision Models — A Practical Guide for Small Businesses and Freelancers

How to Run AI Models Locally with Ollama's New Decision Models — A Practical Guide for Small Businesses and Freelancers

You just got 47 new emails overnight. Fourteen are customer inquiries, nine are vendor invoices, six are support tickets that range from "mildly annoyed" to "about to churn," and the rest are newsletters you'll never read. Sorting through all of that before your first coffee is nobody's idea of productive work.

Now imagine a small AI model — running right on your laptop, not sending a byte of data anywhere — that could classify every one of those messages in seconds. No monthly API bill. No cloud subscription. No wondering whether a customer's private data just got logged on someone else's server.

That's what Ollama's new decision models are built for. Announced in late October 2026, Ollama added decision models purpose-built for classification and scoring, available in four sizes along with macOS speech generation capabilities. These models are designed for one thing: making structured decisions about data. Classification. Scoring. Categorization. Triage. The kind of repetitive judgment calls that eat hours out of your week.

This guide walks you through setting them up, using them for real business tasks, and avoiding the mistakes that trip up most first-timers.

What Are Ollama Decision Models, and Why Should You Care?

Most AI chat models are generalists. You ask them to write a poem, debug code, plan a vacation, draft an email — they try to do it all. That flexibility is powerful, but it comes with trade-offs. General-purpose models are large, slow, and often overkill for simple tasks.

Ollama's decision models take a different approach. They're specialists. Their architecture is optimized for classification and scoring — taking an input (a chunk of text, a customer message, a product description) and returning a structured output (a category, a score, a priority level). Think of them less like a chatbot and more like a very smart sorting machine.

Here's why that matters for small businesses and freelancers:

They're small enough to run on hardware you already own. Decision models come in four sizes, and the smaller variants are designed to run on standard consumer hardware. You don't need a gaming rig or a cloud GPU. Check Ollama's documentation for the specific RAM requirements of each size — they vary, but the smaller models are built with laptops in mind.

They're fast for their task. Because they're not trying to generate creative prose or hold a conversation, they can classify inputs quickly. Run a batch of customer messages through a smaller decision model and you'll have results far faster than manual sorting.

Your data never leaves your machine. This is the big one. If you're handling customer information, medical records, financial data, or anything covered by privacy regulations, running AI locally means you control the entire pipeline. No third-party data processing agreements. No wondering what happens to your prompts after they're sent. For context, Sweden recently blocked ChatGPT and Claude on government workstations, redirecting to Copilot instead over exactly these concerns. Small businesses face the same risks on a smaller scale.

Zero ongoing API costs. Cloud AI pricing adds up fast. If you're making thousands of classification calls per month — scoring leads, sorting tickets, categorizing transactions — the per-token fees can quietly become a real line item. Local models cost you electricity and that's it.

Setting Up Ollama and Pulling Your First Decision Model

If you've never used Ollama before, the setup is simpler than you might expect. Ollama is a lightweight tool that lets you download and run open-source AI models locally. No Docker containers, no Python environments, no command-line wizardry required.

Step 1: Install Ollama

  1. Go to ollama.com and download the installer for your operating system (macOS, Windows, or Linux).
  2. Run the installer. On macOS, it's a standard drag-to-Applications process. On Windows, it's a typical setup wizard.
  3. Open a terminal (Terminal on Mac, Command Prompt or PowerShell on Windows) and type ollama --version. If you see a version number, you're set.

Step 2: Pull a Decision Model

In your terminal, run the pull command for the decision model you want. Check Ollama's model library at ollama.com for the exact model name and available sizes. The command follows the standard Ollama pattern:

ollama pull [decision-model-name]:[size]

Visit the Ollama model library to find the correct model identifier for the decision models — the naming convention and available size tags will be listed there. The four sizes range from smallest (fastest, least RAM) to largest (most accurate, most resource-intensive).

Which size should you pick? Start with one of the mid-range sizes. It's typically the sweet spot for most small business tasks — accurate enough for real work, small enough to run on a standard laptop. If you find it's making too many mistakes on nuanced classifications, step up to the next size. If you're on an older machine with limited memory, drop to the smallest variant.

Step 3: Test It With a Quick Classification

Once the model is downloaded, you can test it immediately:

ollama run [decision-model-name]:[size] "Classify this customer email as one of: complaint, question, praise, spam. Email: 'I've been waiting three weeks for my order and nobody has responded to my last two emails. This is unacceptable.'"

You should get back something clean like: complaint

That's it. You're running a local AI decision model. No account creation, no API key, no billing page.

Five Practical Use Cases That Actually Save Time

Abstract explanations only go so far. Here's where Ollama decision models earn their keep in day-to-day work.

1. Sorting Customer Inquiries by Type and Urgency

If you run a small e-commerce store or service business, you probably get a mix of pre-sale questions, post-sale support requests, complaints, and general inquiries — all landing in the same inbox. Manually reading and routing each one takes time and attention you could spend elsewhere.

Set up a simple classification prompt:

Classify the following customer message into exactly one category:
- billing_question
- product_question
- complaint
- return_request
- praise
- spam

Also assign an urgency score from 1 (low) to 5 (high).

Respond in this exact format:
category: [category]
urgency: [number]

Message: "[paste customer message here]"

Run this against each incoming message. A mid-range decision model handles this kind of structured classification reliably. You can then use the output to route messages — complaints with urgency 4-5 go to you directly, product questions go to your FAQ autoresponder, spam gets archived.

2. Scoring Inbound Leads

Freelancers and small agencies often get more inquiries than they can handle. Not all leads are equal. Some are ready to buy, some are tire-kickers, and some are a poor fit entirely.

Here's a prompt pattern for lead scoring:

You are a lead scoring assistant. Based on the inquiry below, score this lead from 1 to 10 on likelihood to convert to a paying client. Consider: budget indicators, timeline urgency, specificity of request, and fit with a [your service type] business.

Respond in this exact format:
score: [number]
reason: [one sentence explanation]

Inquiry: "[paste lead message here]"

The key here is adapting the prompt to your specific business. Replace [your service type] with what you actually do — web design, bookkeeping, marketing consulting, whatever. The more context you give the model about what a good lead looks like for you, the better its scoring gets.

3. Categorizing Invoices and Expenses

If you're a freelancer doing your own bookkeeping (and let's be honest, most of us are), categorizing expenses is tedious but necessary. Feed invoice descriptions or transaction notes through a classification prompt:

Categorize the following expense into one of these accounting categories:
- office_supplies
- software_subscriptions
- travel
- meals_entertainment
- professional_services
- marketing
- equipment
- other

Respond with only the category name.

Expense description: "Adobe Creative Cloud monthly subscription - $54.99"

This works especially well in batch. If you export your bank transactions as a CSV, you can write a simple script to run each description through the model and get back a categorized list. Even without scripting, manually pasting 50 transactions is faster than thinking through each category yourself.

4. Triaging Support Tickets

For businesses with a support queue, decision models can prioritize tickets based on content analysis. A message mentioning "cancel," "refund," or "switching to a competitor" should probably get attention before a message asking how to reset a password.

You are a support ticket triage system. Analyze the ticket below and provide:
- priority: critical / high / medium / low
- department: billing / technical / account / general
- sentiment: positive / neutral / negative / angry

Respond in exactly that format, nothing else.

Ticket: "[paste ticket content here]"

The structured output format is important here. By constraining the model to specific response patterns, you get consistent, parseable results instead of free-form paragraphs.

5. Content Tagging for Creators

If you publish blog posts, videos, or social media content, tagging and categorizing your archive makes everything more findable. Instead of manually tagging hundreds of old posts, run their titles and descriptions through a classification prompt:

Assign 2-4 relevant tags from this list to the content described below:
[list your standard tags here]

Respond with only the tag names, comma-separated.

Content: "[title and brief description]"

This is the kind of task that's perfect for a decision model — it doesn't need creativity, just consistent judgment applied at scale.

Running Batch Classifications: The Real Productivity Unlock

Classifying one message at a time is useful for understanding how the model works. But the real value comes when you process many inputs at once.

Ollama exposes a local API that runs on your machine. If you're comfortable with basic scripting — or willing to learn — you can write a short script that reads a list of inputs from a file, sends each one to the model, and writes the results to a spreadsheet. Check Ollama's API documentation for the correct endpoint and port number.

Even without scripting, you can use Ollama's command-line interface in a loop. On Mac or Linux:

while IFS= read -r line; do
  echo "$line" | ollama run [decision-model-name]:[size] "Classify this as: complaint, question, praise, spam. Respond with only the category. Message: "
  echo "---"
done < messages.txt

This reads each line from a file called messages.txt, classifies it, and prints the result. It's not elegant, but it works — and it processes locally, privately, for free.

If you're already using AI tools for other parts of your workflow — writing, image creation, brainstorming — and want everything in one place without managing separate local installations, an all-in-one AI tool can complement your local setup for tasks that benefit from cloud-scale models.

What Most People Get Wrong With Local Decision Models

After watching people adopt these tools, a few patterns keep showing up.

Mistake 1: Using Vague Prompts

The single biggest source of bad results. "Classify this email" is too vague. Classify it how? Into what categories? The model needs explicit categories to choose from and a clear output format. Every prompt should specify the exact labels you want and the exact response structure you expect.

Mistake 2: Choosing Too Many Categories

If you give the model 25 possible categories, accuracy drops. Decision models work best with 3-8 clear, distinct categories. If you need more granularity, use a two-stage approach: first classify into broad buckets, then sub-classify within each bucket.

Mistake 3: Starting With the Largest Model

Bigger isn't always better. The largest decision model needs significantly more RAM and runs noticeably slower. For most classification tasks, a mid-range model produces results that are nearly as accurate at much higher speed. Start small, benchmark against your actual data, and only scale up if you're seeing real accuracy problems.

Mistake 4: Expecting Conversational Ability

These are decision models, not chat models. If you ask a decision model to write you a blog post or explain quantum physics, you'll get poor results. They're optimized for structured input → structured output. Use the right tool for the right job. For creative and conversational tasks, a free AI chat tool is a better fit.

Mistake 5: Not Validating Results

Local AI is a tool, not an oracle. When you first deploy a classification workflow, manually check the first 50-100 results. Look for patterns in what the model gets wrong. Often, the fix is simple — a category name is ambiguous, or two categories overlap too much. Adjust your prompt, re-run, and check again. This calibration step is where most people skip ahead and then wonder why their automation produces garbage.

Privacy: Why Running AI Locally Matters More Than You Think

The privacy angle isn't just a nice-to-have — for many businesses, it's the deciding factor.

When you send data to a cloud AI service, you're trusting that provider with your customers' information, your business communications, and your internal documents. Most providers have reasonable privacy policies, but "reasonable" isn't the same as "airtight." And if you're in a regulated industry — healthcare, finance, legal, education — cloud AI can create compliance headaches that aren't worth the convenience.

Running Ollama decision models locally means your data stays on your hardware. Period. There's no network call, no data logging, no third-party processor. For small businesses that handle sensitive customer data, this is a genuine competitive advantage — you can offer AI-powered service without the privacy trade-offs.

If you're evaluating AI tools and privacy matters to you, it's worth reading the privacy policy of any platform you consider. Not all AI services handle data the same way.

Quick Start: What You Can Do in 10 Minutes

If you've read this far and want to try it right now, here's the fastest path:

  1. Install Ollama from ollama.com (2 minutes).
  2. Pull a decision model: check the Ollama model library for the correct name and pick a mid-range size (3-5 minutes depending on your internet).
  3. Classify one real input from your business — a customer email, a support ticket, an expense description — using one of the prompt templates above (1 minute).
  4. Adjust the categories in your prompt to match your actual workflow (2 minutes).
  5. Run 10 more inputs and check the results (2 minutes).

In ten minutes, you'll know whether this model handles your specific use case well enough to build a workflow around it. That's a better investment than reading five more articles about AI productivity.

When Local Models Aren't Enough (And What to Do About It)

Local decision models are excellent for structured classification tasks. But they have limits.

If you need to generate content — write emails, create marketing copy, produce images, compose music — a local decision model won't help. These models are classifiers, not creators.

If you need to handle complex, multi-step reasoning — "read this contract, identify the risky clauses, and draft a response" — you'll want a larger, more capable model than what most laptops can run locally.

For those tasks, the practical approach is a hybrid setup: use local models for classification and sorting (where privacy and cost matter most), and use a cloud-based AI tool for small business for creative and complex work. This way, sensitive data stays local while you still get access to powerful models for everything else.

The key is matching the tool to the task. Not every job needs a cloud supercomputer, and not every job can be handled by a small local model. Knowing which is which is where the real productivity gains come from.

If you want access to every AI model in one subscription — for the creative, generative, and complex tasks that complement your local classification setup — that's worth exploring once your local workflow is running.

Adapting These Workflows for Your Specific Business

The prompt templates above are starting points, not finished products. The real value comes from customizing them to your context.

A few principles for making these prompts your own:

Use your actual category names. Don't use generic labels like "type A" and "type B." Use the terms your team already uses — "warranty claim," "feature request," "billing dispute." The model understands natural language, so use yours.

Add examples to the prompt. If you find the model misclassifying certain inputs, add 2-3 examples directly in the prompt. "For reference: 'My package arrived damaged' = complaint. 'Do you ship to Canada?' = product_question." This technique — called few-shot prompting — dramatically improves accuracy for edge cases.

Test with your messiest data first. Don't test with clean, obvious examples. Test with the ambiguous, poorly-written, typo-filled messages that real customers actually send. If the model handles those, it'll handle everything.

Document what works. When you find a prompt that reliably classifies your data, save it. Put it in a text file, a note, a shared document — wherever your team can find it. Prompt engineering is iterative, and you don't want to reinvent the wheel every time.

For more step-by-step AI workflows like this one, the guides and how-to section covers a range of practical use cases.

The Bigger Picture: AI Decisions Are Moving Local

Ollama's decision models are part of a broader trend. Large enterprises like SAP — which recently reported more than 60,000 internal Joule Work users — are investing heavily in AI-powered decision-making tools. But you don't need to be SAP-sized to benefit.

The tools are getting smaller, faster, and more accessible. A model that would have required a data center five years ago now runs on a MacBook. And the tasks these models handle — sorting, scoring, classifying, triaging — are exactly the tasks that consume disproportionate time in small businesses.

This isn't about replacing human judgment. It's about handling the 80% of decisions that are repetitive and straightforward so you can focus your actual thinking on the 20% that require nuance, creativity, and experience.

Wrapping Up

Ollama's decision models solve a specific, real problem: they let you automate repetitive classification tasks locally, privately, and for free. They won't write your novel or design your logo, but they'll sort your inbox, score your leads, categorize your expenses, and triage your support tickets — all without sending a single piece of data to the cloud.

The setup takes ten minutes. The prompt customization takes another twenty. And the time you save compounds every single day.

Start with one task. The one that annoys you most. Classify it, score it, sort it. See if the results hold up. Then expand from there.

And when you're ready to pair your local classification workflow with powerful AI for writing, images, video, and music, start creating in one place at Gab AI.