llama.cpp
High-performance inference engine written in pure C/C++ with no dependencies. The foundational technology powering most local LLM tools.
Complexity
Expert
2023
Created
C/C++
Language
Very Large
Community
Run LLMs Locally
Discover the best tools for running Large Language Models on your own hardware - from powerful engines to user-friendly applications
Complexity levels:
BeginnerIntermediateAdvancedExpertProduction
All Tools
14 tools available
Ollama
The "Docker for LLMs" - Developer-friendly tool with Docker-like commands for managing and serving models via OpenAI-compatible API.
Complexity
Beginner
2023
Created
Go
Language
Very Large
Community
vLLM
High-throughput, memory-efficient inference engine designed for production serving with PagedAttention algorithm. Best for multi-user scenarios.
Complexity
Expert
2023
Created
Python
Language
Large
Community
LocalAI
Free, open-source OpenAI alternative. Multi-modal API gateway supporting text, image, audio, and video generation through various backends.
Complexity
Advanced
2023
Created
Go
Language
Large
Community
Docker Model Runner
Docker-native way to run AI models. Distributes models as OCI artifacts through Docker Hub, with familiar Docker CLI commands and OpenAI-compatible API.
Complexity
Beginner
2025
Created
Go
Language
Growing
Community
Msty Studio
Privacy-first, feature-rich desktop and web app with advanced workflows, Knowledge Stacks (RAG), split chats, and parallel model comparisons. Supports both local and cloud models.
Complexity
Beginner
2024
Created
JavaScript/TypeScript
Language
Medium
Community
LM Studio
Polished, closed-source desktop app with integrated model browser, chat interface, and one-click local API server. Best UI/UX experience.
Complexity
Beginner
2023
Created
JavaScript/TypeScript
Language
Very Large
Community
GPT4All
Open-source, privacy-focused desktop app with unique LocalDocs RAG feature. Runs entirely offline with no data collection.
Complexity
Beginner
2023
Created
C++/Python
Language
Very Large
Community
Jan
Modern, open-source desktop app emphasizing extensibility and clean UI. 100% offline with plugin architecture.
Complexity
Beginner
2023
Created
TypeScript
Language
Large
Community
Open WebUI
Feature-rich, self-hosted web interface with built-in RAG, multi-user support, and plugin system. Most popular web UI for local LLMs.
Complexity
Intermediate
2023
Created
JavaScript/Svelte
Language
Very Large
Community
Text-Generation-WebUI (Oobabooga)
The "Swiss Army Knife" of local LLM UIs. Massive plugin ecosystem with support for TTS, image generation, and multiple backends.
Complexity
Advanced
2023
Created
Python
Language
Very Large
Community
AnythingLLM
All-in-one RAG application. Makes it incredibly easy to build and manage private knowledge bases with document chat capabilities.
Complexity
Beginner
2023
Created
JavaScript
Language
Large
Community
LibreChat
Polished, open-source ChatGPT clone with enterprise features. Multi-user support with robust authentication methods.
Complexity
Intermediate
2023
Created
TypeScript/React
Language
Large
Community
LobeChat
Modern, open-source AI chat framework with sleek UI, PWA support, and plugin marketplace. Strong mobile experience.
Complexity
Beginner
2023
Created
TypeScript/React
Language
Large
Community