Running Qwen Image Edit 2511 4-bit on Amazon EKS

Sat, 15 Aug 2026 12:00:00 +0800

I published Qwen Image Edit 2511 4-bit as a selective NF4 quantization rather than quantizing every transformer layer indiscriminately. Some layers remain at higher precision to preserve output quality. The resulting Diffusers-compatible model can operate below 20 GB of VRAM, including on 16 GB GPUs with CPU offload.

Gary Stafford used the model in his Amazon EKS deployment. His implementation packages the 17 GB model for an NVIDIA L40S, distributes it from S3 to node-local EBS, loads it into a FastAPI inference service, and reports model and GPU readiness through health checks. The accompanying deployment repository includes the containers, Kubernetes manifests, caching DaemonSet, API, and end-to-end inference tests.

Hugging-Face on Singularity

Running Qwen Image Edit 2511 4-bit on Amazon EKS