Local AI Inference
Local AI lets you use Decoder's explain and chat features against a model running on your own hardware via Ollama or LM Studio — useful when code cannot leave your environment.
What it is
Routing AI calls to a locally-served OpenAI-compatible endpoint (Ollama, LM Studio) instead of a cloud provider.
Why it's useful
Strongest privacy posture: file content stays in your environment, no third-party billing, works offline.
How Decoder implements it
Configure a local base URL in Settings; Decoder uses the OpenAI-compatible chat completion shape — same prompt pipeline as cloud providers.
When to use it
Code under NDA, regulated environments, air-gapped reviews, exploration without per-token cost.
When NOT to use it
When you need frontier-model reasoning quality and have no local GPU — a small local model will under-perform.
Practical example
Ollama running llama3.1:8b on your laptop. Decoder Explain hits http://localhost:11434/v1 and never touches the cloud.
FAQ
Glossary
- Local inference
- Running model inference on hardware you control, not on a hosted API.
Related
Ollama is a lightweight runtime for serving open-weight LLMs locally. Decoder talks to it through its OpenAI-compatible endpoint.
BYOK means you bring your own AI provider key. Decoder never proxies AI calls through a shared account: your key, your billing, your privacy boundary.
Chat with Your Code turns a repository into a queryable knowledge surface. Ask 'where is auth handled?' or 'what does this script do?' and get answers grounded in your actual files.
Repository Analysis turns a codebase into something you can read, search and interrogate. Upload a ZIP or import a public GitHub project; Decoder indexes structure, runs static checks and gates AI features behind your own key.
OpenRouter is a unified API in front of many model providers. With BYOK you get access to dozens of models in Decoder from a single key.
The EU AI Act (Regulation 2024/1689) sets obligations on providers and deployers of AI. For code analysis, transparency and data control are the levers that matter most.
GDPR governs personal data in the EU. Even code can carry personal data (logs, fixtures, PII in test files). Decoder's defaults minimise exposure.
LM Studio is a desktop app that runs LLMs locally with an OpenAI-compatible API. Decoder targets that endpoint when you choose local inference.