Loading post…
Hugging FaceYiYi Xu, Daniel Gu, Sayak Paul, Alvaro Somoza, Dhruv Nair, Aritra Roy Gosthipaty, Linoy Tsaban, Apolinário from multimodal AI artTutorial
Diffusers welcomes FLUX-2
Summary
Diffusers introduces support for FLUX-2 models through the Flux2Pipeline and Flux2Transformer2DModel classes. Running the 4-bit quantized checkpoint diffusers/FLUX.2-dev-bnb-4bit requires loading Mistral3ForConditionalGeneration as the text encoder and the transformer in bfloat16 precision with CPU offloading enabled. During image generation with a prompt and 50 inference steps, the pipeline encodes text embeddings through the Mistral 3 model before executing diffusion operations. However, executing the pipeline on a GPU with 14.56 GiB capacity triggers a CUDA OutOfMemoryError during 4-bit dequantization inside bitsandbytes matrix multiplication operations. Additionally, the execution logs warn that Flax classes are deprecated in Diffusers and will be removed in version 1.0.0.
Context
Running 4-bit quantized FLUX-2 text-to-image generation using Diffusers and Transformers on GPU hardware with CPU model offloading.
Approach / What changed
Loading Flux2Transformer2DModel and Mistral3ForConditionalGeneration from diffusers/FLUX.2-dev-bnb-4bit with bfloat16 precision, configuring Flux2Pipeline with CPU offload, and generating an image via prompt encoding and diffusion steps.
Takeaways
FLUX.2 pipeline integration in Diffusers pairs Flux2Transformer2DModel with Mistral3ForConditionalGeneration as the text encoder.
Inference on a 14.56 GiB GPU with 4-bit bitsandbytes quantization and CPU offloading encountered a CUDA OutOfMemoryError during linear dequantization.
Flax classes are deprecated across Diffusers and are scheduled to be removed in version 1.0.0.
Source and freshness
- Source
- huggingface.co
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.