How to run an AI model on your own computer with Ollama
A guide to installing Ollama on Ubuntu and running your first local large language model.
Who it’s for: Beginner to Expert
Time: 15 minutes
What you need#
- 64-bit Linux distribution (Ubuntu 20.04+)
- At least 10GB free disk space
- RAM/VRAM matching your intended modelAI model A program trained on a huge amount of text (and for some, images) that predicts what comes next. It answers by writing one small piece of text after another. size (e.g., 8GB for a 7B model)
- Internet connection for initial download
Steps#
1. Install Ollama#
Run the official install script in your terminalTerminal A window where you type commands instead of clicking. Terminal on macOS and Linux, PowerShell or Terminal on Windows.:
curl -fsSL https://ollama.com/install.sh | sh
curl -fsSL https://ollama.com/install.sh | sh
New to this? How to install the Ollama engine
New to this?: You are running a single command that downloads and sets up the OllamaOllama Free software that downloads AI models and runs them on your own computer, answering on http://localhost:11434. software on your computer. It works like an automated installer that prepares the engine needed to run AI models.
Under the hood How the installation service works
Under the hood: This script installs the Ollama binary and sets it up as a systemd service so it runs in the background on Linux. You can verify the service status using systemctl status ollama.
2. Verify the installation#
Check that the Ollama service is active:
systemctl status ollama
systemctl status ollama
New to this? Checking if Ollama is running
New to this?: This command checks if the Ollama program is currently running in the background of your computer. It is like checking if a light switch is turned on.
Under the hood Verifying the background service
Under the hood: This queries the systemd manager to ensure the Ollama background process is active and healthy.
3. Download and run a model#
Pull a model (like Llama 3.2) and start a chat session:
ollama run llama3.2
ollama run llama3.2
New to this? Downloading and starting a model
New to this?: This command does two things: it downloads the AI model files to your hard drive and then opens a chat window where you can talk to the AI. It is similar to downloading a game and then clicking ‘Play’.
Under the hood How models load into memory
Under the hood: Ollama fetches the model weights (usually 4-bit quantized by default) and loads them into your RAM or VRAM. If no GPU is detected, it falls back to CPU inference.
4. Exit the chat#
To stop talking to the model and return to your terminal, type:
/bye
/bye
New to this? How to close the chat
New to this?: This is a special command used inside the AI chat to tell the program you are finished and want to close the conversation.
Check that it worked#
For specific model tags or advanced APIAPI A door one program offers to others: send a request in an agreed format, get an answer back. configurations, check the official Ollama documentation.
If it doesn’t work#
- Model is too slow Ensure you have enough RAM/VRAM for the model size or try a smaller parameter count (e.g., 1B or 3B instead of 8B).
Going further#
Scaling with Hardware#
Hardware is the main constraint. A 7B model typically needs 8–16GB of RAM/VRAM, while a 70B model requires 48GB or more. If you run out of VRAM, the system may use system RAM (paging), which is significantly slower.
Automation and Scripting#
Instead of interactive chat, you can pipe text directly into a model for automation using: echo "Your prompt here" | ollama run llama3.2.
Researched and drafted with AI from the sources listed here, checked step by step, and reviewed by Rahul Thakar before publishing. Spotted a mistake? Tell us and we'll fix it.