LocalAI
local AI engine
LocalAI
Read profile Official siteRunning a tool locally can give more control over data, but brings hardware, configuration and maintenance needs. Separate the model, interface and any external services that may be enabled.

These active services are starting points from the directory. Editorial highlighting and score order this selection; they do not replace a trial on your own task.
local AI engine
LocalAI
Read profile Official sitelocal models
Ollama · US
Read profile Official sitemodel serving
vLLM Project
Read profile Official sitelocal models
LM Studio · US
Read profile Official sitelocal model runtime
Docker · US
Read profile Official siteself-hosted language models
Open WebUI · US
Read profile Official sitemodel serving
SGLang
Read profile Official sitelocal models
Jan AI · US
Read profile Official sitelocal model runtime
Apple · US
Read profile Official siteNeeds within the same family require different functions. Start with the expected result to narrow your shortlist.
| Your need | Workflow to examine | Decisive check |
|---|---|---|
| Use a chat interface | The interface and inference engine may run on different machines. | Identify engine address, loaded model and extension connections. |
| Serve a model to several users | Runtime, memory and concurrency determine practical capacity. | Measure waiting and errors under representative load; model file size alone is insufficient. |
| Require offline processing | Obtain models and dependencies first, then check the entire chain disconnected. | Include OCR, embeddings, search and tools; one remote call changes the workflow. |

Try a model without sending a document to a remote service.

Search internal documents with access controls you understand.

Set up a reversible environment for testing several models.

Verify what genuinely works offline.

Measure memory use, speed and quality on your own hardware.

Inspect licences, updates and outbound connections for each component.
Choose criteria before starting. These are practical checks to adapt to your task, not measured performance results for directory services.
| Dimension | How to check | Pitfall |
|---|---|---|
| Memory | Measure actual memory use with your model and context length. | File size is not the entire memory requirement. |
| Responsiveness | Separate loading time, time to first token and total duration. | High throughput after loading can hide a long initial wait. |
| Network | Check connections, extensions and logs in the actual configuration. | A local interface may call a remote service. |
| Execution identity | Record model, revision, quantisation, runtime and chat template; repeat after a controlled restart. | The same model name does not prove identical weights, conversation formatting or settings. |
Separate blocking errors from minor defects. Compare identical inputs, retain failed attempts and include human time. A small trial helps uncover defects; it does not prove general reliability. Repeat when the model, data or configuration changes.
These fictional examples show how to inspect an output. They do not describe observed incidents involving the services on this page.
| Warning sign | Fictional example | How to check |
|---|---|---|
| The model exceeds available memory | Loading works but a longer input crashes the process. | Measure memory at the intended context length with a margin; reduce size or context if needed. |
| Local is confused with offline | A local interface still calls an external service. | Inspect network connections and extensions during a trial without sensitive data. |
| Quantisation weakens a specific task | A smaller variant loses numbers in extraction tasks. | Compare variants on identical documents, including human correction and total time. |
Starting situation. Ask questions about ten internal documents on an isolated workstation. The test must work without a network connection and cite the passages used.

Install model and interface on a test machine. Disconnect it, then check which functions genuinely remain available.

Record memory use, response time, answer quality and passage retrieval on the same questions.

Document versions, licences, updates, backups and how the setup can be exported or replaced.




Not automatically. An interface may enable telemetry, updates or remote APIs; inspect actual connections.
No. Compare speed, memory, task quality and maintenance on your own hardware.
The model generates answers; the interface manages files, users and interactions. Their hosting and licences may differ.