Repository avatar
AI Tools
v0.8.0
active

make-localllm-easier

io.github.phonology024/make-localllm-easier

Private or bulk work on the AI model on your own computer: ask, translate, summarize.

Documentation

make-localllm-easier — run the best local LLM your GPU can handle, in one command

PyPI tests License: MIT

Read this in: English | தமிழ்

localllm checks your GPU, picks the most accurate AI model that fits it in your language, downloads it and starts a private chat. Offline and free.

localllm doctor, then a streamed answer at 122 tok/s, recorded on an RX 9070 XT

Recorded from a real run with tools/record_demo.py: the text is the commands' own output and the answer streams at the speed llama-server produced it.

Install

pip install make-localllm-easier
localllm

That's it: the first run shows what your PC can run and asks before it downloads the model. No Python? Download the single localllm file for Windows, macOS or Linux (or the AppImage) from the latest release and run it from a terminal; SHA256SUMS there lets you check it. winget and Homebrew follow (packaging/README.md).

Or ask your coding agent (Claude Code, Cursor, Codex): paste Install localllm: https://raw.githubusercontent.com/phonology024/make-localllm-easier/main/docs/install-for-agents.md. It checks your PC first without changing anything, and asks before each install or download (what it may and may not do).

The commands you'll use

localllm            # pick, download and start the best model, open the chat page
localllm chat       # chat in the terminal, any language
localllm doctor     # what this GPU can run: model sizes, speed, RAM
localllm serve      # just the API: OpenAI, Anthropic, Ollama and Gemini formats at http://127.0.0.1:8080

More: localllm list (every measured model), localllm connect hermes (point an agent or Obsidian at it), localllm eval (score any server in your language). All commands: localllm --help.

Chat with your files. In localllm chat type /attach report.pdf (or a folder); in the --ui page, drop a file on the chat. Each answer comes from the passages that match your question and lists them as sources (file and page), all on this PC. PDF, DOCX, TXT and MD.

Use it from Claude, Cursor or Codex. localllm mcp is an MCP server: the app hands private or bulk work (translate, summarize, ask) to the model on your PC, and the text never leaves it. One line per app in docs/mcp.md.

Why it's different

  • It picks for your GPU. It reads your card and RAM and only offers models that fit and run fast on them.
  • Measured in your language. Models are ranked by real exam scores in Thai, Hindi, Arabic, Japanese and 20+ more languages, not by English benchmarks alone.
  • Uses less RAM. llama.cpp tuned op by op: about 4x less system RAM than its defaults, same speed (below).
  • Private. Your chats never leave your PC, there is no account and no telemetry (Privacy).

Less RAM, same speed

Same 30-turn chat, RX 9070 XT, 32 GB RAM:

modelllama.cpp defaultslocalllmspeed
Qwen3.8-27B Q39.30 GB RAM2.37 GB35.2 tok/s both
gemma-4-26B-A4B QAT9.24 GB RAM2.21 GB~85 tok/s both

Privacy

Chats, files and settings stay on your PC. localllm only goes online to download llama.cpp and models (each checked against a pinned SHA-256) and for update notices ("localllm 0.8 is out", "a better model for Thai now fits your PC"): two small requests a day to the project website, nothing about you is sent, and nothing is installed. Turn them off with localllm updates off or LOCALLLM_NO_UPDATE_CHECK=1. Cloud routing with your own key is off unless you turn it on.

Learn more

Website and leaderboard · FAQ · How it works (doctor output, routing, RAM, where it's tested) · APIs and agents · Benchmarks · Research · Roadmap · Stability · Contributing

Have a GPU we haven't measured? localllm tune, localllm eval --langs en,<yours>, then localllm report opens a prefilled issue (no names, paths or keys) that adds it to the leaderboard.

License

MIT. Models keep their own licenses; benchmark data keeps its own (Apache-2.0).

PYPI
make-localllm-easier
Install Command
pip install make-localllm-easier
Runtime: uvx