Inference on Singularityhttps://singularity.sg/tags/inference/Recent content in Inference on SingularityHugoen-SG(c) {year} Abhishek DujariSat, 15 Aug 2026 12:10:00 +0800My llama.cpp Command for Qwen3-Coder-Nexthttps://singularity.sg/2026/08/my-llama-cpp-command-for-qwen3-coder-next/Sat, 15 Aug 2026 12:10:00 +0800https://singularity.sg/2026/08/my-llama-cpp-command-for-qwen3-coder-next/<p>I use this command to run Qwen3-Coder-Next locally with <code>llama-server</code>. I am keeping the complete command and an explanation of each option here so I do not have to reconstruct it later.</p> <p>This is the best configuration I have found so far for an RTX 4090 with 24 GB of VRAM and 64 GB of system RAM. My priority is a longer context window rather than maximum inference speed. There is still room to optimize it further, but this setup works well enough for regular use.</p>