Building a Local AI Coding Agent Stack with Qwen 3.6 Models and Custom Agent Skills
A practical breakdown demonstrates building a zero-cloud local coding setup using Qwen3.6 dense and MoE models paired with Pi Coding Agent and vLLM. The architecture leverages task-specific agent skills in Elixir, Python, and TypeScript to automate system administration and git commit workflows.
Impact: High
Why it matters
You can run an unconstrained, high-speed coding agent locally by pairing dense LLMs for logic with MoE models for sysadmin speed.
TL;DR
- 01Use dense models like Qwen3.6-27B for code generation and MoE models for fast sysadmin routines.
- 02Switch from Llama.cpp to vLLM when orchestrating multiple simultaneous AI agent prompts.
- 03Externalize prompt rules into script skills (Elixir, Python, TS) to maintain small, clean context windows.
Key facts
- Dense Model Standard
- Qwen3.6-27B for code accuracy
- MoE Speed Model
- Qwen3.6-35B MoE for sysadmin speed
- Inference Engines
- vLLM for concurrent throughput, llama.cpp for instant setup
- Lightweight Fallback
- LFM2.5-8B for low-latency terse execution
Model Selection: Dense vs MoE for Local Coding
In local agentic workflows, model architecture directly impacts output quality. Dense models like Qwen3.6-27B provide higher reasoning precision for multi-file refactoring and complex logic. Conversely, Mixture-of-Experts architectures like Qwen3.6-35B excel at system administration tasks where generation speed is paramount.
Inference Optimization with vLLM and Llama.cpp
For single-user interactive terminal sessions, llama.cpp delivers easy setup out of the box. However, when orchestrating multi-agent pipelines or handling parallel prompt execution, switching to vLLM enables high throughput and request batching. For ultralight tasks, LFM2.5-8B provides terse, low-latency execution.
Modular Agent Skills Architecture
Rather than stuffing all instructions into system prompts, define targeted task skills using executable scripts (.exs in Elixir, Python, or TypeScript). Storing conventional commit rules or NixOS deployment scripts as external skills allows tools like Pi Coding Agent to invoke precise automation logic on demand without ballooning prompt context.
Try it in 2 minutes
python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen3.6-27B --port 8000 --tensor-parallel-size 2bash
✓ When to use
- Building self-hosted coding agent environments without third-party API dependencies or data leakage.
- Automating repetitive devops, commit formatting, and server provisioning via modular script skills.
- Serving concurrent background agent calls using vLLM request batching.
✕ When NOT to use
- Running on hardware with low VRAM and constrained RAM bandwidth without GPU offloading.
- One-off quick chat interactions where setting up inference servers is unnecessary.
What to do today
- Set up vLLM server locally to host Qwen3.6 models with OpenAI-compatible API endpoints.
- Extract system prompt rules like conventional-commit formatting into executable .exs or Python skills.
- Test Pi Coding Agent or Paseo.sh with local model endpoints over Tailscale.
Sources