What is an edge LLM in OpenJet?

In OpenJet, an edge LLM is a local model and tool runtime running near the device it supports, rather than a remote API receiving delayed telemetry.

Why use an edge LLM instead of a cloud agent?

Because some edge systems cannot wait for cloud round-trips. If bandwidth is limited or the operating window is short, local inference is the only thing fast enough to matter.

Does OpenJet only work on Jetson?

No. Jetson is a strong fit, but the edge LLM workflow also applies to Linux edge systems where local execution is operationally necessary.

edge llm

Edge LLM for remote systems and short operating windows

This page is for the edge LLM use case where the problem is physics: narrow links, remote systems, and not enough time to wait for the cloud.

OpenJet is an edge LLM interface for operators who need local judgment near the hardware. When logs cannot be shipped fast enough or links are intermittent, it reads local evidence and supports approved action on the device itself.

Install OpenJet Read the docs

Why local wins

Built for intermittent links. When systems are remote and uplinks are weak, OpenJet lets the operator work with local evidence instead of waiting for remote analysis that may arrive too late.
Useful inside short windows. If you only have a brief chance to inspect state and act, local inference can be the difference between operating in-window and missing it entirely.
Runs at the point of operation. OpenJet keeps the model and tool runtime close to the device, which is exactly what many edge LLM deployments need.

Common deployment paths

Use an edge LLM in remote field systems where links are unreliable or expensive.
Inspect telemetry locally when sending raw logs to a hosted model would miss the moment to act.
Run local diagnostics and approved procedures close to the hardware rather than through a remote control plane.

Frequently asked questions

What is an edge LLM in OpenJet?: In OpenJet, an edge LLM is a local model and tool runtime running near the device it supports, rather than a remote API receiving delayed telemetry.
Why use an edge LLM instead of a cloud agent?: Because some edge systems cannot wait for cloud round-trips. If bandwidth is limited or the operating window is short, local inference is the only thing fast enough to matter.
Does OpenJet only work on Jetson?: No. Jetson is a strong fit, but the edge LLM workflow also applies to Linux edge systems where local execution is operationally necessary.

Explore OpenJet

Embed a local model with the Python SDK, run Qwen3.8-27B as a local coding agent, compare local coding-agent alternatives, read the docs, or go back to the homepage.