How to Run AI Models Locally in 7 Steps

You can run AI models locally on an ordinary laptop with 16 GB of RAM, and you do not need a data center to do it. Tools like Ollama, open-weight models such as Gemma and DeepSeek-R1, and a Docker Compose stack with free SSL let you go from a fresh machine to a private ChatGPT-style server in an afternoon.

What You Need to Run AI Models Locally

To run AI models locally you need three components: a runtime, a model, and enough RAM for the model's parameters. Ollama is the runtime used in this workflow. It is an interface that downloads and executes open-weight models from Meta, Google, DeepSeek and others, and it runs on macOS, Linux and Windows.

The video this guide is based on was published in February 2025 by Erick Wendel, who demonstrated the entire flow on a MacBook M1 Pro with 16 GB of RAM. Nothing in the setup required a GPU server or specialized acceleration, which is the core argument: small and mid-size open models run on ordinary developer hardware.

The models themselves come from a public library. Each listing states its parameter count and memory requirement, which is the single most important number when deciding what fits on your machine.

Install Ollama and Pull Your First Model

Installing Ollama takes one download and no configuration. After installing, you run ollama serve to start the local server, or let Ollama run as a background service if you prefer. The command ollama list shows which models are already downloaded.

From there, ollama run <model-name> either starts a model you already have or downloads it on first use. The transcript shows three practical examples:

• DeepSeek-Coder 1.3B (about 760 MB) responds to prompts like "generate a Node.js server" and is small enough for machines with 4-8 GB of RAM.
• Gemma 2B, a small open model from Google in the Gemini model family, handles natural-language questions and ran comfortably on the 16 GB MacBook.
• Llama 7B variants need roughly 8 GB minimum, so 16 GB is a practical floor once the operating system and other applications take their share.

The developer tip from the video: when unsure whether a model will fit, check the model's stated memory requirement against your free RAM, or simply ask an AI assistant whether your machine can handle it.

Model Size, RAM and the Accuracy Trade-off

Model size is the dividing line between what a laptop handles and what needs a server. Parameter counts scale directly with memory: 1B-2B models run in a few gigabytes, 7B models want about 8 GB, and competitive frontier-class open models are in another category entirely.

The extreme example in the transcript is DeepSeek-R1, the open reasoning model DeepSeek released in January 2025: its full weights are roughly 404 GB, which no consumer laptop can hold. DeepSeek's own model page documents the architecture and sizes across its variants.

The practical consequence of running tiny models is hallucination. The video shows the 1.3B and 2B models producing plausible-looking but wrong code and nonsense answers, and the presenter's own testing concluded that 2B-parameter models and above give usable results. Small models are fine for simple questions, validation tasks and experiments; for accuracy comparable to a hosted chat product, you need bigger weights and more memory.

Calling Local Models Through the API

Ollama exposes an OpenAI-compatible API on a local port by default, so integration with existing services is a matter of pointing the client at http://localhost:11434 (the transcript demonstrates this with a copied code sample), setting the provider to ollama, and passing the model name plus message.

One gotcha appeared on camera: requesting a model that is not installed returns a model-not-found error. The fix is to run the model once first (or pull it), then retry. After that, the same request streamed a Go code example from a 7B model while the presenter was simultaneously recording the video and running other applications, which gives a rough sense of how little headroom a 7B model needs on a 16 GB machine.

For application code, community libraries let JavaScript services talk to the API directly, and the video points to Continue as the next step for wiring these models into an editor as a self-hosted alternative to GitHub Copilot.

Deploying Your Own GPT-Style Server With Docker Compose

The deployment stack is a single Docker Compose file with four pieces: Ollama as the model runtime, a model-loader sidecar that pulls Gemma 2B, DeepSeek-R1 and two code models automatically after Ollama is healthy, Open WebUI as a ChatGPT-style web interface, and Traefik as the entry point that terminates HTTPS.

Traefik reads Docker labels to route traffic: requests to /ollama go to the Ollama service, and requests to the domain root go to Open WebUI. Certificates come from Let's Encrypt using Traefik's built-in ACME support, so the whole stack gets a valid HTTPS certificate with a few annotation lines and no manual cert management. The transcript shows Let's Encrypt validating the domain from outside and issuing the certificate during the first docker compose up.

Configuration lives in a .env file holding the domain, and an upload script copies the project to the server via SCP. The demo authentication in Open WebUI is disabled in the Compose file for convenience; the video explicitly notes you should enable authentication before putting the server in front of real users or company data.

Sizing a Cloud VPS for Heavier Models

The recommended path for competitive accuracy is a cloud server rather than a local machine. The video demonstrates on a Hostinger VPS with 8 GB of RAM and two cores, roughly half the presenter's MacBook, and shows Gemma 2B responding quickly there. With 8 GB the small models work; to run 7B-class and larger models comfortably, the presenter points to 32 GB plans.

Deployment on a clean VPS follows three steps: install Docker, upload the project, run docker compose up. The model downloads happen in the background, and Open WebUI becomes usable as each model finishes loading.

A warning from the transcript applies to any provider: an 8 GB dual-core machine is enough to prove the workflow, but latency and answer quality improve directly with RAM and CPU, so pick the box size based on the largest model you actually plan to serve.

Security and Production Limitations

HTTPS via Let's Encrypt protects the transport, but an Open WebUI deployment with authentication disabled is an open interface to your compute. Before production use, enable login, restrict the API route if it should stay private, and review what data your prompts expose to the model and its logs.

The video's scope ends at a working, TLS-secured single-server deployment. It does not claim compliance certifications, tenant isolation or audit controls, and none should be inferred from running models on your own VPS. Those controls are the responsibility of the surrounding application and organization.

It is also worth being precise about "local": models run on your hardware, but if you deploy on a cloud VPS, inference happens on the provider's machine. True on-device inference only happens with the laptop setup, while the VPS gives you control and capacity at the cost of hardware you rent.

FAQ

• Do I need a GPU to run AI models locally? No. The workflow shown runs on a 16 GB M1 MacBook CPU. Small models from 1B to 2B parameters also run on 4-8 GB machines, with a noticeable drop in answer quality.

• What is the smallest useful model size? DeepSeek-Coder 1.3B (about 760 MB) can generate simple code, but the video's testing found 2B parameters is the practical floor for usable accuracy. Below that, hallucinations are frequent.

• Can I run DeepSeek-R1 fully? Not on consumer hardware. The full model's weights are roughly 404 GB, so you would use its smaller distilled variants, which is what the Docker setup in the video pulls.

• How does the SSL certificate work in the Docker stack? Traefik requests a certificate from Let's Encrypt automatically during startup. It validates that your configured domain is publicly reachable, then issues and renews the certificate without manual steps.

• Is the web interface required? No. Open WebUI is a convenience layer that gives you a chat page for switching models. Everything it does is also available through the Ollama API or CLI, which is how you would integrate models into your own application.

From Video to Article

The core lesson here is that practical AI infrastructure knowledge often lives inside long videos, locked in a format that is hard to search, cite or share with a team. Communities around Brazilian content creators know this well: the audience of Gustavo Dev Doido, students of the Bootcamp do Dev Doido, and members of the Crazystack Typescript community at crazystack.com.br regularly produce walkthroughs and lessons that deserve a written version. If you publish technical videos like this one, Skala Blog turns a YouTube URL into a structured, publishable article: paste the link, get the transcription, review the draft. The deployment knowledge in this guide was exactly the kind of content worth pulling out of a video timeline and into a page a developer can search.

Source video