b12x Linear and MoE Backends¶

b12x provides optional CUDA kernels for NVIDIA SM120 and SM121 GPUs. Install the dependency with:

uv pip install "vllm[b12x]"

b12x linear kernels participate in automatic selection after established optimized backends and before emulation. Select the linear and MoE backends explicitly with:

vllm serve <model> \
    --linear-backend b12x \
    --moe-backend b12x

Only pass --moe-backend b12x for a compatible packed-weight MoE model. The linear and MoE backends can be selected independently.

b12x uses MXFP8 activations by default for MXFP4 MoE and the checkpoint's activation format for NVFP4 MoE. MXFP4 falls back to BF16 when its A8 path does not support the model configuration. Set VLLM_B12X_MOE_FP4_FORCE_A16=1 to force BF16 activations for either FP4 weight format.

Supported Configurations¶

Backend	Supported configurations
Linear	Per-tensor FP8, 128x128 block FP8, MXFP8, NVFP4 (including W4A16), MXFP4, IQ2_XS, IQ2_XXS, and Q8_0
MoE	Tensor-parallel MXFP4 weights with BF16 or MXFP8 activations; NVFP4 weights with BF16, NVFP4, or MXFP8 activations; IQ2_XS, IQ2_XXS, and Q8_0 weights with BF16 activations
Embedding	Packed Q8_0 lookup with BF16 output

The b12x MoE backend does not support expert parallelism, expert maps, EXL3, or NF3.

ModelOpt block codecs¶

ModelOpt mixed-precision checkpoints select block codecs through the quantization_config.quantized_layers entries in config.json. Each entry declares quant_algo, group_size, block_payload_bytes, and packing="ggml":

`quant_algo`	`group_size`	`block_payload_bytes`
`IQ2_XS`	256	74
`IQ2_XXS`	256	66
`Q8_0`	32	34

Serialized matrices have uint8 shape [N, K/group_size, block_payload_bytes]. Dense preparation rearranges the encoded blocks without expanding the matrix to BF16. Routed experts use bias-free SiLU or non-gated ReLU²; tensor-parallel shards must preserve whole blocks and satisfy the kernel's alignment constraints. Only Q8_0 supports packed embedding lookup.

Quantized normalization weights and other packed vectors are not supported. Store normalization weights as ordinary floating-point tensors and omit their entries from quantization_config.quantized_layers.

Dense, embedding, and expert matrices remain packed. Their execution uses BF16 activations without activation quantization. Prepare the kernels and workspaces before CUDA graph capture.