dasllama Uses Compile-Time AST Macros for 5x Faster CPU Speech and LLM Inference

01Eliminating Kernel Explosion with AST Autotuning

Traditional inference runtimes like llama.cpp rely on thousands of lines of hand-crafted intrinsics maintained across multiple CPU instruction sets. When building real-time voice translation into game engines where the GPU was saturated by rendering frames, developers faced strict CPU latency limits. Integrating massive C++ codebases for small models proved inefficient, prompting the port of Whisper to daslang.

Rather than hand-writing individual compute kernels for every permutation of data type, CPU instruction, and tensor shape, dasllama utilizes compile-time AST macros:

[tune_perm(dot = "vpdpbusd", width = 512, mr = 16, requires = "avx512vnni,avx512bw")]
[tune_perm(dot = "amx_bf16", width = 512, mr = 16, nrsplit = 2, requires = "amx-bf16,avx512bf16,avx512vnni")]
[tune(gen = "dasllama_gemm_gen::k4_tile", fallback = "dot_vpdpbusd_width512_mr16")]
def k4q8_tile_gen(var yp : float?; kqg : uint8 const?; ksg : uint8 const?;
                 xqp : int8 const?; xsp : float const?; xbsp : int const?) {
    kq_tile_ref(4, yp, kqg, ksg, xqp, xsp, xbsp, n, d, g, t0);
}

02Runtime Benchmarks and Hardware Sidecars

Under JIT execution, dasllama races available vectorization widths, loop unroll counts, and hardware intrinsics on the host machine. The optimal configuration is cached in a JSON sidecar file, bypassing profiling on future runs. This autotuning mechanism yields measured speedups over universal hand-tuned engines:

• Whisper Speech-to-Text: 5x faster than baseline whisper.cpp on Apple M1.
• Qwen3.6-35B-A3B Prefill: 2.3x speedup on AMD EPYC and 2.8x on AWS Graviton.
• Unified Codebase: Tokenizers, runtime schedulers, Vulkan SPIR-V shaders, and OpenAI-compatible API servers are written entirely in daslang without C++ FFI bindings.