# Magnitude

> Run the best local models for your machine

## Overview

Magnitude is an open source inference server for the hardware you already have. It profiles your machine, recommends the right models for it, then downloads, tunes, and runs them. Plug it into the agent you already use.

- **Product**: Local inference server for the hardware you already have
- **License**: Apache 2.0
- **Platforms**: macOS, Linux, and Windows through WSL
- **Hardware**: Apple Silicon, NVIDIA GPUs, AMD GPUs, and CPU-only setups
- **Install**: `npm i -g @magnitudedev/cli`
- **Onboarding**: `magnitude docs onboarding`

## How It Works

1. Profiles your chip, memory, and bandwidth.
2. Estimates fit and tok/s per model, then ranks models by speed, accuracy, intelligence, and memory.
3. Downloads and runs the selected model with a built-in inference server.
4. Tunes speculative decoding and more, all set for your hardware.
5. Connects to a supported agent harness.

## Features

- Models ranked by speed, accuracy, intelligence, and memory
- Hardware profiling to assess fit and estimate tok/s
- Built-in inference server; Ollama and LM Studio are not required
- Automatic speculative decoding and concurrency tuning
- Private, offline model execution
- Compatible GGUF models from Hugging Face
- No token costs, API keys, subscriptions, or rate limits for local inference
- Free and open source under Apache 2.0

## Supported Agent Harnesses

Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. During setup, your agent connects your harness to the model you pick.

## Privacy and Offline Use

Prompts, files, and models stay on your machine. Once a model is downloaded, no internet connection is needed.

## Hardware

There's no fixed minimum. Magnitude profiles your machine and recommends what runs well on it. More memory lets you run larger models.

## Models

Magnitude profiles your chip, memory, and bandwidth, then estimates fit and tok/s for every model in the catalog. It ranks them by speed, accuracy, intelligence, and memory so you can pick.

## FAQ

- **What is Magnitude?** An open source inference server for the hardware you already have. It profiles your machine, recommends the right models for it, then downloads, tunes, and runs them. Plug it into the agent you already use.
- **How does it know what my machine can run?** Magnitude profiles your chip, memory, and bandwidth, then estimates fit and tok/s for every model in the catalog. It ranks them by speed, accuracy, intelligence, and memory so you can pick.
- **What hardware do I need?** There's no fixed minimum. Magnitude profiles your machine and recommends what runs well on it. More memory lets you run larger models.
- **What systems does Magnitude support?** Magnitude runs on macOS, Linux, and Windows through WSL. It supports Apple Silicon, NVIDIA GPUs, AMD GPUs, and CPU-only setups. It profiles your hardware and recommends models that fit.
- **Which harnesses work with it?** Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. During setup, your agent connects your harness to the model you pick.
- **Do I need to manage it after setup?** No. It runs in the background, loads models when your agent needs them, and unloads them when idle or memory gets tight. Your agent can install or switch models through the CLI anytime.
- **Is it private?** Yes. Prompts, files, and models stay on your machine. Once a model is downloaded, no internet connection is needed.

## Getting Started

```sh
npm i -g @magnitudedev/cli && magnitude setup
```

Or send this prompt to your agent:

Set up local models for me with the Magnitude CLI. Install it with `npm i -g @magnitudedev/cli` (or my package manager), then run `magnitude docs onboarding` and follow the instructions.

## Links

- Website: https://magnitude.dev
- Documentation: https://docs.magnitude.dev
- Model documentation: https://docs.magnitude.dev/models
- GitHub: https://github.com/magnitudedev
- Twitter/X: https://x.com/usemagnitude
- Contact: founders@magnitude.dev

## Keywords

local models for Mac, Apple silicon inference, Mac AI models, local inference server, local AI inference, local LLM, hardware profiling, model recommendations, model selection, GGUF, Hugging Face, speculative decoding, offline AI, private AI, open source inference server, agent harness