Tool families

Local and open AI: verify the control you really get

Running a tool locally can give more control over data, but brings hardware, configuration and maintenance needs. Separate the model, interface and any external services that may be enabled.

Open desktop computer showing memory and cooling
AI-generated illustration
27 active services in the directory

Services to examine

These active services are starting points from the directory. Editorial highlighting and score order this selection; they do not replace a trial on your own task.

Distinguish workflows before choosing

Needs within the same family require different functions. Start with the expected result to narrow your shortlist.

Need, workflow and evidence to inspect
Your needWorkflow to examineDecisive check
Use a chat interfaceThe interface and inference engine may run on different machines.Identify engine address, loaded model and extension connections.
Serve a model to several usersRuntime, memory and concurrency determine practical capacity.Measure waiting and errors under representative load; model file size alone is insufficient.
Require offline processingObtain models and dependencies first, then check the entire chain disconnected.Include OCR, embeddings, search and tools; one remote call changes the workflow.

When to explore this family

AI-generated illustration

Try a model without sending a document to a remote service.

Laptop and folders on a private desk
AI-generated illustration

Search internal documents with access controls you understand.

AI-generated illustration

Set up a reversible environment for testing several models.

How to compare tools

  1. Creative studio with a camera and drawing tablet
    AI-generated illustration

    Verify what genuinely works offline.

  2. Document research with an atlas and books in a library
    AI-generated illustration

    Measure memory use, speed and quality on your own hardware.

  3. Library with books and a research notebook
    AI-generated illustration

    Inspect licences, updates and outbound connections for each component.

Measure what actually matters

Choose criteria before starting. These are practical checks to adapt to your task, not measured performance results for directory services.

Checks for this family
DimensionHow to checkPitfall
MemoryMeasure actual memory use with your model and context length.File size is not the entire memory requirement.
ResponsivenessSeparate loading time, time to first token and total duration.High throughput after loading can hide a long initial wait.
NetworkCheck connections, extensions and logs in the actual configuration.A local interface may call a remote service.
Execution identityRecord model, revision, quantisation, runtime and chat template; repeat after a controlled restart.The same model name does not prove identical weights, conversation formatting or settings.
How should results be interpreted?

Separate blocking errors from minor defects. Compare identical inputs, retain failed attempts and include human time. A small trial helps uncover defects; it does not prove general reliability. Repeat when the model, data or configuration changes.

Spot a failure before choosing the tool

These fictional examples show how to inspect an output. They do not describe observed incidents involving the services on this page.

Symptom, example and check to perform
Warning signFictional exampleHow to check
The model exceeds available memoryLoading works but a longer input crashes the process.Measure memory at the intended context length with a margin; reduce size or context if needed.
Local is confused with offlineA local interface still calls an external service.Inspect network connections and extensions during a trial without sensitive data.
Quantisation weakens a specific taskA smaller variant loses numbers in extraction tasks.Compare variants on identical documents, including human correction and total time.

Practise with documents and answer keys →

A repeatable trial

Starting situation. Ask questions about ten internal documents on an isolated workstation. The test must work without a network connection and cite the passages used.

  1. AI-generated illustration

    Isolation

    Install model and interface on a test machine. Disconnect it, then check which functions genuinely remain available.

  2. Wooden desk with a laptop and monitor
    AI-generated illustration

    Measurement

    Record memory use, response time, answer quality and passage retrieval on the same questions.

  3. AI-generated illustration

    Maintenance

    Document versions, licences, updates, backups and how the setup can be exported or replaced.

Evidence for a decision

Participants comparing meeting notes around a conference speakerphone and tablet.
AI-generated illustration
Output
An acceptable result on your hardware with controlled network flows.
Sorting documents with folders and an e-reader
AI-generated illustration
Checks
Memory, latency, model licence and remote dependencies.
AI-generated illustration
Evidence
Versions, configuration, memory measurements and disconnected trial.
AI-generated illustration
Failure signal
A local claim when another component transmits the data.
Compare actual cost and keep a decision record →

Related guides

Questions before choosing

Does local mean nothing leaves the machine?

Not automatically. An interface may enable telemetry, updates or remote APIs; inspect actual connections.

Is the largest model always best?

No. Compare speed, memory, task quality and maintenance on your own hardware.

How do model and interface differ?

The model generates answers; the interface manages files, users and interactions. Their hosting and licences may differ.

Explore other tool families