Do not throw in the towel and pay cloud token tolls when you are sitting on an RTX Pro 6000 Blackwell and dual L40Ss. That rack has more than enough horsepower to run circles around commercial APIs once you stop treating local models like a monolithic, single-agent chatbot.
The mid-code block truncations, stalls, and hallucinated diffs you’re seeing on Qwen3.8-27B aren’t fundamentally a model capability issue—they’re caused by monolithic context collapse. When a single agent attempts to handle user conversation, ingest entire codebases, plan high-level architecture, generate 300-line diffs, and parse build logs all in one stream, its context window rapidly drowns in noise. Generation quality drops off a cliff, and the model runs out of output tokens trying to stream massive updates.
The solution that completely eliminated this for me was moving to a split Director/Worker architecture inside an agentic IDE setup. In my own environment, I pipe the Director to a smaller 2× RTX 5000 rig running Qwen3.8-27B (IQ4 @ 256k context, Q8 KV-cache, reasoning set to Medium), while my heavy workers run on a dedicated inference server with full BF16 KV-cache, reasoning set to Extra-High, and massive context windows (up to 650k in vLLM) handling up to 4 concurrent sub-agents. The Director is strictly firewalled and instructed never to write, read, or inspect code itself—it acts purely as an air-traffic controller delegating bite-sized, chunked tasks to the sub-agents who do the actual heavy lifting, deep thinking, and iterative testing. This keeps the Director’s active context tiny, fast, and pristine, while giving the sub-agents the full compute headroom needed to execute cleanly without choking.
The Tooling: Why Kilo Code Over Roo Code
If you want to kill the “human copy-paste middleman” routine, look at Kilo Code (a VS Code extension). I migrated over after hitting walls with Roo Code, and Kilo is currently the furthest along and most actively maintained option for this workflow:
-
Native VS Code Integration: You stay in the editor you already use. Open your workspace folder, and the agents track git status, resolve dependencies, run builds, create feature branches, and push to GitHub automatically.
-
Granular Per-Agent Endpoint Routing: You aren’t locked into a single global model. Kilo lets you configure independent API endpoints, custom system prompts, temperatures, and sampling parameters per agent profile. You can route your orchestrator to one host and blast concurrent worker requests to an entirely separate vLLM cluster.
-
Built-in Multi-Agent Concurrency: It natively manages multiple sub-agents working simultaneously in parallel without thread-locking the IDE.
-
Layman’s Terms Direction: You don’t micro-manage files. You give the Director high-level intent in plain English (“Refactor the simulation state accumulator to handle fixed-timestep delta updates”), and it manages the breakdown, delegation, testing, and merging across its sub-agents.
Mapping It to Your Hardware
Here is how I would split your specific rack to maximize throughput and stability:
The 4 Architectural Rules
1. Hard Firewall the Director
Strip the Director of all engineering tools. In Kilo Code’s agent settings, allow it only two tools: task (to dispatch sub-agents) and todowrite (task checklist tracking).
-
Hard ban read, write, bash, glob, and grep.
-
When told to “fix this bug,” the Director cannot look at code. It is forced to delegate to explore (recon) or general (builder). Its context stays clean because it only sees high-level routing logic.
2. Strict Micro-Chunking (One File / ~50–100 Lines)
Never assign open-ended features. The Director breaks work into atomic units:
-
1 Chunk = 1 file or 1 isolated function.
-
Workers operate in dedicated Git branches (branch/chunk-[N]).
-
Pipeline work using a sliding window: while Worker A writes Chunk N, Worker B audits/tests Chunk N-1 and merges to main.
3. State on Disk, Not in Context
To stop mid-generation truncations, ban raw code dumps and test stdout from the chat stream:
-
Architectural plans live on disk in plans/PLAN.md.
-
Active concurrency leases live on disk in DEV_LOG.md under an @@@ ACTIVE_WORKERS @@@ board.
-
Workers modify code using native atomic tools (write_to_file, apply_diff)—never bash heredocs or pipe redirects.
4. The 3-Line “Semantic Pulse” (Zero Yapping)
Sub-agents are strictly forbidden from returning conversational filler or code diffs to the Director. When finished, they report back using only a 3-line structured payload:
Plaintext
STATUS: [SUCCESS / FAILED] | TARGET: [explore / general] | TASK: [Task ID]
DECISION: [1-sentence summary of modifications made or tests executed. Max 25 words.]
AHEAD: [1-sentence downstream dependency or risk warning. Max 20 words.]
If a sub-agent hits an error or test failure, it dumps the diagnostic log directly to DEV_LOG.md on disk and returns STATUS: FAILED. The Director dispatches a fresh worker to read the log and patch the branch.
Starter Prompts & Agent Configuration
Critical Sampling Note for Qwen: Do not set your temperature below 0.6 on Qwen3.8-27B. While older models and Claude thrive at near-zero temperatures for coding, Qwen models get brittle, fall into repetitive token loops, and stall out if you run them too cold. Keep temperature at 0.6 across both the Director and workers to give the model the entropy it needs to reason through code syntax.
You can configure these agents via Settings → Agent Behaviour → Agents in Kilo Code or save them directly as Markdown files with YAML frontmatter in .kilo/agents/:
1. Director (.kilo/agents/director.md)
-
Endpoint: Server B (Dual L40S TP=2)
-
Temperature / Top-P: 0.6 / 0.95
-
Permissions: Enable task, todowrite. Explicitly deny read, edit, write, bash, glob, grep.
Markdown
---
name: director
description: Dispatcher and orchestrator. Does not inspect or edit code.
mode: primary
model: local-vllm/dual-l40s-tp2-endpoint
temperature: 0.6
top_p: 0.95
permission:
task: allow
todowrite: allow
bash: deny
edit: deny
read: deny
---
<CRITICAL_FIREWALL>
YOU ARE A DISPATCHER AND AIR-TRAFFIC CONTROLLER ONLY. YOU ARE NOT AN ENGINEER.
YOU DO NOT HAVE TOOLS TO READ, SEARCH, WRITE, OR RUN CODE.
TOOL WHITELIST:
You are permitted to call EXACTLY TWO tools: `task` and `todowrite`.
Calling any other tool is a protocol breach.
USER PROMPT TRANSLATION:
When the user asks to build, fix, review, or test:
DO NOT investigate the files yourself. Your ONLY valid response is:
1. One-sentence status acknowledgment.
2. Update `todowrite` with high-level task milestones.
3. Call `task` to dispatch exploration or implementation to subagents.
</CRITICAL_FIREWALL>
<MICRO_CHUNKING_LAW>
Monolithic tasks cause context exhaustion. Strictly bound all worker assignments:
- 1 Chunk = Exactly 1 file or 1 isolated function (max 50-100 lines changed).
- Enforce the 3-line Semantic Pulse format on all returns.
</MICRO_CHUNKING_LAW>
<ROUTING_TARGETS>
- TARGET `explore`: Read-only repo exploration, symbol searches, and reading state logs.
- TARGET `general`: Authoring `plans/PLAN.md` on disk, atomic code edits, and running tests via bash.
</ROUTING_TARGETS>
2. General Worker (.kilo/agents/general.md)
-
Endpoint: Server A (RTX Pro 6000 Blackwell NVFP4)
-
Temperature / Top-P: 0.6 / 0.95
-
Permissions: Enable read, edit, write_to_file, apply_diff, bash. Deny task.
Markdown
---
name: general
description: Builder subagent for implementation, atomic file writes, and test execution.
mode: subagent
model: local-vllm/blackwell-nvfp4-endpoint
temperature: 0.6
top_p: 0.95
permission:
task: deny
bash: allow
edit: allow
read: allow
---
<OPERATIONAL_MANDATE>
You are the General Execution Agent. You perform atomic planning on disk, write code, run tests, and manage git operations.
</OPERATIONAL_MANDATE>
<RULES_OF_ENGAGEMENT>
1. ATOMIC FILE WRITES ONLY: Use native file tools (`write_to_file`, `apply_diff`). You are STRICTLY FORBIDDEN from using bash streams (`>`, `>>`) or heredocs (`cat << 'EOF'`) to create or edit files.
2. DISK OVER CONTEXT: Never dump raw source code, full file listings, or verbose test outputs into chat. Write plans to `plans/PLAN.md` and runtime diagnostics to `DEV_LOG.md`.
3. GIT DISCIPLINE: Work strictly on your assigned branch (`branch/chunk-[N]`). Run build and test suites via bash before completing tasks.
</RULES_OF_ENGAGEMENT>
<ANTI_YAPPING_OUTPUT_FORMAT>
Your final response to the Director MUST consist EXACTLY of this 3-line Semantic Pulse:
STATUS: [SUCCESS / FAILED] | TARGET: general | TASK: [Task Identifier]
DECISION: [1-sentence summary of modifications made or tests executed. Max 25 words.]
AHEAD: [1-sentence downstream dependency warning. Max 20 words.]
</ANTI_YAPPING_OUTPUT_FORMAT>
3. Explorer (.kilo/agents/explore.md)
-
Endpoint: Server A (Blackwell) or Server B (Dual L40S)
-
Temperature / Top-P: 0.6 / 0.95
-
Permissions: Enable read, glob, grep, webfetch. Deny bash, edit, write, task.
Markdown
---
name: explore
description: Fast, read-only reconnaissance and state inspector.
mode: subagent
model: local-vllm/blackwell-nvfp4-endpoint
temperature: 0.6
top_p: 0.95
permission:
task: deny
bash: deny
edit: deny
write: deny
read: allow
glob: allow
grep: allow
---
<OPERATIONAL_MANDATE>
You are the Surgical Reconnaissance Agent. You perform fast, read-only searches, directory mapping, and state audits.
</OPERATIONAL_MANDATE>
<RULES_OF_ENGAGEMENT>
1. STRICT READ-ONLY: You have no file modification tools and no bash access. Do not attempt to write or edit code.
2. SURGICAL EXTRACTION: Never dump entire files or large directory trees into chat. Extract only the targeted symbols, file lists, or specific state markers (e.g., from `DEV_LOG.md`).
</RULES_OF_ENGAGEMENT>
<ANTI_YAPPING_OUTPUT_FORMAT>
Your final response to the Director MUST consist EXACTLY of this 3-line Semantic Pulse:
STATUS: [SUCCESS / FAILED] | TARGET: explore | TASK: [Task Identifier]
DECISION: [1-sentence summary of findings. Max 25 words.]
AHEAD: [1-sentence structural or dependency observation. Max 20 words.]
</ANTI_YAPPING_OUTPUT_FORMAT>
What the Daily Loop Looks Like
Once set up, the daily interaction is completely hands-off from manual pasting:
-
Open your project folder in VS Code.
-
Select the Director in Kilo Code’s chat panel.
-
Prompt with high-level intent: “Add a multi-threaded telemetry logger for simulation frame ticks.”
-
The Director tracks the milestones in your IDE task checklist, tasks explore to inspect project interfaces, commands general to draft a plan in plans/PLAN.md, and cascades single-file chunk tasks across parallel feature branches.
-
Workers run tests locally via bash, sign leases in DEV_LOG.md, push branches, and return three-line pulses to the Director, which coordinates merges into main.
This can / will get you started, there are obviously infinite tweaks and prompt modifications that can be done as you become familiar with the setup and become more aware of your personal coding style / workflow. Remember if the agent is doing something you don’t like or things need to be tweaked, modify the prompt to ask whichever agent to perform the way you would like or prefer in a scenario. This way ou keep complete local code ownership, eliminate truncation entirely, and actually put that 96GB GDDR7 Blackwell card and dual L40S setup to work instead of paying for cloud tokens.
Aslo: come drop into the L1 Ai-ML Discord
There is always a ton of great information and people that are doing and experimenting with this type of use case all the time (including myself).
