Start here

If you are new to running AI on your own computer, this page is for you. It takes about ten minutes, and it assumes no prior knowledge.

A. Run your first local LLM

Running a model on your own machine is mostly a sequence of six decisions. Here is the order that avoids most frustration.

1. 1. Check your GPU and hardware.The memory on your GPU is the main constraint. Knowing how much you have tells you which models are even worth trying.
2. 2. Choose a runtime or application.This is the software that loads and runs the model. Three of them come up constantly — see below.
3. 3. Choose a model and configuration that fits.A model plus a quantization level plus settings is what actually has to fit, not the model name alone.
4. 4. Download and start the model.The first load is usually the slowest. After that, starting again is normally much faster.
5. 5. Try a simple prompt.Ask something small and check the reply appears at a speed you find acceptable before building anything on top of it.
6. 6. Only then consider agents and tools.Agents add moving parts. It is much easier to debug them once a plain model is running reliably.

The three runtimes you will hear about

These are different front doors to the same underlying job. None of them is universally best; the right one depends on how much control you want and how comfortable you are with configuration.

RigFact has not validated every runtime and model combination, and nothing here is a recommendation of one runtime over another. Project documentation for each of them is the right place for installation steps.

B. Understand local AI in plain language

You will meet these words immediately. Here is what each one actually means, before the detail.

Model

The AI itself. A file with the learned weights that turns your text into a reply.

VRAM

The memory on your graphics card that is used while the model is running. It is the resource that most often decides what you can run.

Quantization

A way of making a model smaller so it uses less memory. Different methods and levels can affect quality and speed, so the exact method matters.

Context

How much information the model can work with at one time — your instructions, the conversation so far, and any document you give it. A larger context uses more memory.

Tokens

Small pieces of text the model reads and writes. A word is usually one or a few tokens. Pricing, limits and speed are often described in tokens.

Recipe

The exact model, settings and runtime configuration that RigFact captured or tested. It is the unit of evidence on RigFact — not a suggestion.

Agent

A system where the model does more than answer text: it can decide to take steps toward a task.

Tool calling

A mechanism that lets a model or agent request actions from defined tools or software, instead of only producing text.

Reproducible

Enough technical identity and source information was captured to seriously attempt the same run again.

Will it feel fast?

This is the question almost everyone asks, and it deserves a careful answer. Performance measurements can exist — how many tokens per second a run produced, for example. But a throughput figure on its own does not automatically establish that the experience will feel good to you, because it depends on your task, your patience and what you are comparing against.

RigFact reports when a performance measurement exists, and states plainly when it cannot make a practical claim about how fluid something will feel. There is no approved tokens-per-second threshold that means "usable" or "smooth", and this page will not invent one.

C. From local chat to an agent

"Local AI" covers a wide range of ambition. It helps to see it as a progression, because each step adds real requirements.

1. Local chat

   You send text, the model replies. This is the simplest thing to run locally, and the starting point for almost everyone.

2. Document assistant

   The system may retrieve or hand documents and context to the model, so it can answer about material you provide.

3. Coding assistant

   The model may inspect or change code, depending on the surrounding tooling you use it with.

4. Agent with tools

   The model may be connected to defined tools and workflows, and can decide to act rather than only reply.

5. Email / calendar agent

   This needs considerably more than a model running on your machine — see the note below.

Where to go next