llmware
Framework for building private, local RAG and agent applications with small specialized models on AI PCs, edge and self-hosted setups.
Updated
✨ Try AgentScout's free AI image generator →Good for: building AI agents, working with documents, memory for agents
Use cases
About llmware
llmware is listed in our memory & RAG category and is aimed at builders. Framework for building private, local RAG and agent applications with small specialized models on AI PCs, edge and self-hosted setups. Pricing, as the maker states it: Free and open source (Apache-2.0). Maturity: Mature. We last checked the official link on 2026-10-20.
What is llmware?
llmware is an open-source Python framework for building knowledge-based LLM applications that run locally and privately: on AI PCs and laptops, at the edge or self-hosted, across Windows, Mac and Linux. It supports GGUF, OpenVINO, ONNX Runtime (including Qualcomm QNN), Windows Local Foundry and PyTorch, and picks the right inference technology for the target machine.
It has two parts: a model catalog of 300+ quantized models, including 50+ of llmware's own fine-tuned SLIM, Bling, Dragon and Industry-BERT models for enterprise tasks (cloud models from OpenAI, Anthropic and Google are supported too), and a RAG pipeline that parses, chunks, embeds and indexes documents into libraries you can query with text, semantic, hybrid and metadata filters and prompt with sources.
Sources: the official README and our directory listing, checked 2026-10-20. Official site: llmware-ai.github.io/llmware.
Who is llmware for?
Developers building private RAG and agent apps for enterprise documents who want small models that run on a laptop or on-prem hardware instead of a cloud API.
llmware at a glance
- Open source: Yes (Apache License 2.0)
- GitHub: llmware-ai/llmware · 14,816 stars (as of Oct 21, 2026)
Key features
- Model catalog with 300+ models in optimized, quantized formats
- 50+ fine-tuned small models (SLIM, Bling, Dragon, Industry-BERT) for RAG and automation
- Document parsing, chunking, embedding and knowledge-base libraries
- Text, semantic, hybrid and metadata queries, plus prompt-with-sources
- Runs on CPU, GPU and NPU via GGUF, OpenVINO and ONNX Runtime
- Database options from a laptop to a parallel cluster
llmware FAQ
What is llmware?
llmware is an open-source Python framework for building knowledge-based LLM applications that run locally and privately: on AI PCs and laptops, at the edge or self-hosted, across Windows, Mac and Linux. It supports GGUF, OpenVINO, ONNX Runtime (including Qualcomm QNN), Windows Local Foundry and PyTorch, and picks the right inference technology for the target machine.
Who is llmware for?
Developers building private RAG and agent apps for enterprise documents who want small models that run on a laptop or on-prem hardware instead of a cloud API.
Is llmware free?
llmware's pricing, as the maker states it: Free and open source (Apache-2.0). Check the official site for current plans.
Is llmware open source?
Yes. llmware is released under the Apache License 2.0, according to its GitHub repository.
Does llmware need a GPU or cloud API?
No. The project says virtually all of its examples and models can run on device, on a laptop, and it is optimized for AI PCs; cloud models are optional.
What are alternatives to llmware?
Similar tools include RAGFlow, Agentset, UltraRAG, Honcho, PageIndex and Basic Memory.
llmware ratings and reviews
Be the first to rate llmware: tap 1 to 5 stars below. A written review is optional.
llmware alternatives
Other memory & RAG tools in our directory. See all memory & RAG tools →
RAGFlow
Open-source RAG engine with deep document parsing and agent features for building grounded question-answering over your files.
Agentset
Open-source RAG platform for building agents and apps that answer accurately from your documents.
UltraRAG
Low-code MCP-based framework from OpenBMB for building and debugging complex RAG pipelines with visible reasoning logic.
Honcho
Memory library and service for building stateful agents that remember users.
PageIndex
Document index for reasoning-based, vectorless RAG that lets agents navigate long documents like a table of contents.
Basic Memory
Persistent memory for Claude, Codex and Cursor stored as local Markdown knowledge, available open source or as a hosted cloud workspace.
Popular searches with this tool
🏆 Ranked in: Mem0 alternatives · Headroom alternatives
🔁 Listed in: Headroom alternatives · Mem0 alternatives · PageIndex alternatives
🧭 Task guides: Local and self-hosted tools for agent memory · RAG and document tools for building AI agents · RAG and document tools for agent memory
Badge for makers
Built llmware? Add this badge to your README or site. It links back to this page. About badges
