Qwythos-9B: Community Qwen Merge for Creative Writing and Roleplay

Architectural Merge and Recipe

The creation of Qwythos-9B relies on a passthrough merge (often called a frankenmerge) configured via YAML and hosted on GitHub. Rather than starting from a native 9B base model (since an official Qwen 9B base does not exist), the merge recipe slices and duplicates specific intermediate layers of the Qwen2.5-7B-Instruct architecture. This mathematical expansion increases the context processing depth for stylistic tasks, specifically targeting narrative pacing.

Datasets and Stylistic Fine-Tuning

To achieve its distinct prose, the merged base is aligned with custom datasets styled after Anthropic's Claude and the community-favorite Mythos roleplay models. The resulting model exhibits a higher tolerance for creative length, descriptive adjectives, and complex dialogue formatting compared to vanilla instruction models, which tend to be overly concise or analytical.

Local Execution and GGUF Support

Thanks to the community-provided GGUF quantizations, you can run Qwythos-9B locally. Quantized variants like Q4_K_M or Q5_K_M balance memory footprint and output coherence. Running this 9B model on llama.cpp or Ollama requires approximately 6GB to 8GB of VRAM, making it highly accessible for standard laptop GPUs.