77. LM Studio

The lesson

LM Studio is a free desktop application for discovering, downloading, and running large language models locally and offline on your own computer. It is aimed at developers, privacy-conscious teams, and anyone who wants to experiment with open-weight models without sending data to cloud APIs.

The main value proposition is local inference. Your model weights, prompts, documents, and responses stay on your machine. Nothing you type into the chat leaves the device. LM Studio combines model discovery, a chat interface, document-based retrieval-augmented generation, and a local API server into one app.

Installing LM Studio on Windows is straightforward. Download the installer from lmstudio.ai/download, run it, and launch the app. Windows x64 or ARM64 is supported, AVX2 is required on x64, and 16 GB of RAM is recommended. The first launch may prompt you to download an inference runtime, which needs internet; everything else works offline once models are present.

You find models in the Discover tab. Search by name or paste a Hugging Face URL. LM Studio supports GGUF models and, on Apple Silicon, MLX models. Quantization is shown in filenames: Q4_K_M is a common 4-bit sweet spot, while Q8_0 gives higher quality at the cost of more RAM and VRAM. To load a model, pick it in the Chat tab model loader and adjust context length and GPU offload if needed.

The Chat tab gives you a ChatGPT-like interface. You can set a system prompt that defines behavior for the current conversation, save system prompts plus parameters as reusable presets, and tune sampling values such as temperature, top-p, top-k, max tokens, and repeat penalty. Lower temperatures produce more deterministic output, which is good for coding and factual tasks.

LM Studio can act as a local OpenAI-compatible API server. From the Developer tab, toggle Start server, or run `lms server start` from the terminal. The default URL is http://localhost:1234, and the OpenAI-compatible base URL is http://localhost:1234/v1. By default no authentication is required; you can enable API tokens in the server settings.

Coding assistants can connect to LM Studio by pointing their base URL at http://localhost:1234/v1 and using any loaded model identifier. Tools such as Claude Code, Codex, Continue.dev, Cursor, Cline, and VS Code extensions all support this pattern. Enable JIT loading in LM Studio so the model loads automatically on first use.

Prompt engineering tips: put role, constraints, and output-format instructions in the system prompt; save reusable presets; use 0.0-0.3 temperature for coding and 0.7-1.0 for brainstorming; match context length to the task to save memory; and use reasoning models for multi-step problems. When chatting with documents, include explicit keywords to improve retrieval.

Performance tips: choose the right quantization, offload layers to GPU, enable Flash Attention if supported, use speculative decoding with a small draft model for a free speed boost, and set idle TTL so unused models are unloaded. If you run out of memory, reduce context length, quantization, or GPU offload. Exposing the server beyond localhost is a security risk, so enable authentication first.

Quick reference: start the server with `lms server start`, list models with `lms ls`, download with `lms get <model>`, load with `lms load <model>`, and stream logs with `lms log stream`. Remember that searching the catalog and downloading models need internet; chatting, document RAG, and running the local server work offline once your models are installed.

Check yourself