Magnitude

Run the best local models for your machine

Magnitude profiles your hardware, recommends the right models for it, then downloads, tunes, and runs them. Plug it into the agent you already use.

Why Magnitude?

  • Knows your machine: profiles your hardware to assess fit and estimate tok/s
  • Recommends the best models: ranked by speed, accuracy, intelligence, and memory
  • Tuned end to end: speculative decoding and more, all set for your hardware
  • Easy setup: one command and your agent is running local models
  • Free to run: no token costs, API keys, or rate limits
  • Fully private and offline: models, prompts, and files stay on your machine
  • Models on demand: loaded on request, unloaded when idle or memory fills
  • Open source: Apache 2.0, yours to modify

FAQ

What is Magnitude?
An open source inference server for the hardware you already have. It profiles your machine, recommends the right models for it, then downloads, tunes, and runs them. Plug it into the agent you already use.
How does it know what my machine can run?
Magnitude profiles your chip, memory, and bandwidth, then estimates fit and tok/s for every model in the catalog. It ranks them by speed, accuracy, intelligence, and memory so you can pick.
What hardware do I need?
There's no fixed minimum. Magnitude profiles your machine and recommends what runs well on it. More memory lets you run larger models.
What systems does Magnitude support?
Magnitude runs on macOS, Linux, and Windows through WSL. It supports Apple Silicon, NVIDIA GPUs, AMD GPUs, and CPU-only setups. It profiles your hardware and recommends models that fit.
Which harnesses work with it?
Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. During setup, your agent connects your harness to the model you pick.
Do I need to manage it after setup?
No. It runs in the background, loads models when your agent needs them, and unloads them when idle or memory gets tight. Your agent can install or switch models through the CLI anytime.
Is it private?
Yes. Prompts, files, and models stay on your machine. Once a model is downloaded, no internet connection is needed.

See what your Mac can run
Backed by Y Combinator