🧠 Offline LLMs in Linux | Ollama on Linux | Easy Setup Guide

📌 What is Ollama?

Ollama is a tool to run LLMs (Large Language Models) locally on your computer with just one command. It’s beginner-friendly, supports offline usage, and works on most modern Linux systems.

---

✅ Features

• Supports models like LLaMA 3, Mistral, Phi-3, Code LLaMA, and more

• CLI-based: clean and fast

• Works on CPU or GPU

• Easy install and model usage

• Free and open-source

---

🖥️ System Requirements

• OS: Linux (Ubuntu, Fedora, Arch, etc.)

• Memory: 8 GB+ RAM recommended

• CPU: Any modern x86_64 processor

• (Optional) GPU: For faster performance (NVIDIA preferred)

---

🛠️ 1. Install Ollama

🔹 Run this command:

curl -fsSL https://ollama.com/install.sh | sh

This:

• Installs the Ollama CLI

• Sets up the system service

• Adds the ollama user and group

  

---

🚀 2. Run a Model

🔹 Example (LLaMA 3):

ollama run llama3

It will:

• Pull the model automatically (first time only)

• Start an interactive chat in your terminal

  

---

🧠 3. Try Other Models

ollama run mistral
ollama run phi3
ollama run codellama
ollama run llama2

🔎 List All Installed Models:

ollama list

❌ Remove a Model:

ollama remove mistral

---

🔧 4. Use as a Local API (Optional)

Start the Ollama server:

ollama serve

Use HTTP API to query models:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3",
  "prompt": "What is the capital of India?"
}'

---

📁 Model Location

Models are stored at:

~/.ollama/models

---

🛑 Uninstall Ollama

sudo systemctl stop ollama
sudo rm -rf /usr/local/bin/ollama /usr/local/lib/ollama ~/.ollama
sudo userdel ollama
sudo groupdel ollama

---

💡 Tips

• Use Q4_K_M or Q8_0 models for better performance

• You can run Ollama completely offline after the model is downloaded

• Combine it with tools like LM Studio, KoboldCpp, or Streamlit apps