OpenJetGet Started

Qwen3.8-27B local setup guide

Run Qwen3.8-27B locally as a coding agent

Run the dense Qwen3.8-27B model with OpenJet’s own CUDA engine on an RTX 30, 40 or 50-series GPU with at least 24 GB VRAM. Use OpenJet’s terminal agent or connect your own OpenAI-compatible client.

Hardware-aware setup downloads the model and builds the runtime. The Q4_K_M engine also serves several coding sessions on one card, with a shared context pool and a live monitor for each session. Local mode keeps inference on your machine.

Qwen3.8-27B and Flash-Next are different models

Qwen3.8-27B is the dense model this guide covers. Qwen3.8-Flash-Next is a separate mixture-of-experts model. A token-rate result for one does not establish how the other performs on a coding task.

Strata’s current model guide documents Flash-Next and its variants, DeepSeek V4.1-Flash and Qwen3.6-35B-A3B. Its maintainer described dense Qwen3.8-27B as out of scope. If the dense 27B is the model you want, OpenJet’s specialised CUDA engine serves its supported GGUF quantisations. Both projects support concurrent requests on their respective supported model paths.

Several coding agents on one RTX 3090

With four agents decoding at once on one RTX 3090, the engine produced 424 output tokens/sec in total on short coding prompts and 360 with about 23K tokens of context each (Qwen3.8-27B Q4_K_M, sampled, measured by the engine’s author). Time agents spend on tool calls lowers the total in real use.

Follow the parallel source setup, then start the server with four session slots:

bash
python3 megakernel/server/mk_server.py \
  -m /path/to/Qwen3.8-27B-Q4_K_M.gguf --agents 4 -d 7 --kv q8

Point the agents at http://127.0.0.1:18080/v1. The documented 3090 Each session gets its own slot of the KV cache, as many positions as fit by default. See the engine documentation for the live monitor and limits.

Why use Qwen3.8-27B for local coding?

Qwen3.8-27B is an open-weight dense model aimed at reasoning, coding, long-context work, and tool use. The official model supports a large native context window, but local users should size context to their available memory rather than enabling the maximum by default. For coding-agent work, a useful model is only half the system: it also needs safe repo tools and predictable session management.

  • Private by default. Source code and prompts remain on your workstation.
  • No per-token API bill. Use the compute you already own.
  • Agent-ready. OpenJet adds file search, edits, resumable sessions, and approval-gated shell tools.
  • Flexible deployment. Run on an NVIDIA GPU, a high-memory unified-memory machine, or CPU with partial GPU offload.

Qwen3.8-27B hardware requirements

Memory use changes with the quantization, runtime, context length, and number of GPU layers offloaded. These are planning ranges, not hard minimums. Keep headroom for the KV cache, the operating system, your editor, and the coding-agent process.

Local setupPractical starting pointWhat to expect
Single NVIDIA GPU24 GB VRAM, 32 GB+ system RAMGood fit for a 4-bit quantization with a deliberately sized context.
Unified memory32 GB minimum; 64 GB preferredModel and cache share one pool. More memory gives more context headroom.
CPU plus partial GPU offload32 GB RAM; 64 GB preferredWorks when VRAM is limited, but generation is slower.
Higher-precision or long-context use48 GB+ available memoryUseful when quality or context matters more than minimum footprint.

A 4-bit GGUF is roughly 16–18 GB before cache and runtime overhead. If you hit an out-of-memory error, reduce context length first, then lower the quantization or offload fewer layers. The model’s theoretical maximum context is rarely the right default for a single local workstation.

How to run it as a local coding agent

  1. Install OpenJet. Use Python 3.10 or newer on the machine that will run the model.
  2. Run setup. OpenJet profiles the available CPU, RAM, GPU, and VRAM.
  3. Let setup choose the fit. On a machine with the configured 20 GB memory budget, OpenJet can select the Qwen3.8-27B Q4_K_M MTP profile automatically.
  4. Start inside a repository. The agent can inspect and edit the current project while keeping execution local.
bash
pip install open-jet
openjet setup
cd /path/to/your/repo
openjet

Best settings for coding-agent work

  • Start with 4-bit quantization. It is the most practical balance for fitting a 27B model on a 24 GB GPU.
  • Use a realistic context window. Repository search and context condensing are more memory-efficient than loading an entire codebase into every prompt.
  • Keep shell approval enabled. Let the model propose commands, but inspect state-changing actions before they run.
  • Benchmark your own machine. Prompt processing and generation speed vary substantially by backend, offload, memory bandwidth, and context size.

Qwen3.8-27B local model resources

Qwen3.8-27B gives developers a new 27B-class option for running a capable coding model on hardware they own. OpenJet turns the downloaded model into a working terminal agent instead of leaving users to assemble inference, repository tools, approvals, and session state themselves.

Check the model publisher’s current files, license, and runtime guidance on the Qwen3.8-27B download page.

Qwen3.8-27B local FAQ

Does Strata run the dense Qwen3.8-27B model?
Strata's current model guide lists Qwen3.8-Flash-Next, its variants, DeepSeek V4.1-Flash and Qwen3.6-35B-A3B. Its maintainer has described dense Qwen3.8-27B as out of scope. OpenJet has a specialised CUDA engine for supported Qwen3.8-27B quantisations, including parallel serving for Q4_K_M.
Can several coding agents share one RTX 3090?
Yes. OpenJet's Qwen3.8-27B Q4_K_M engine can serve several concurrent conversations on one GPU. The documented RTX 3090 configuration shares about 132K context positions across sessions. Connect each agent to the server's OpenAI-compatible chat endpoint; each agent still manages its own coding tasks and tools.
Can I run Qwen3.8-27B locally on my own hardware?
Yes. A quantized build can run locally on a capable workstation. A 24 GB GPU is a practical target for a 4-bit quantization, while CPU or unified-memory systems can use system memory at lower generation speeds. Leave additional memory for the context cache and coding-agent tools.
How much VRAM does Qwen3.8-27B need?
The exact requirement depends on quantization, context length, runtime, and GPU offload. A 4-bit model is roughly 16 to 18 GB before context and runtime overhead, so 24 GB VRAM is a sensible starting point. Longer contexts need more memory.
Is Qwen3.8-27B good for a local coding agent?
It is designed for reasoning, coding, and tool use. A useful coding agent also needs a harness for reading files, editing code, searching a repository, running approved commands, and managing context; OpenJet supplies that local tool layer.
How do I use Qwen3.8-27B with a coding agent?
Install a local inference runtime and connect the model to an agent harness with repository tools. OpenJet packages that workflow: hardware-aware setup, file search and edits, resumable sessions, and approval-gated shell commands.

Build your local coding setup

Explore the offline coding agent, compare self-hosted LLM deployment, or read the OpenJet documentation.