[Roadmap] Blackwell MXFP8 and NVFP4 RL training

[zianglih]

@HumansAnd

Miles

• MXFP8 & NVFP4

  • [feat] Expand mxfp8 quantization to support fine-grained layer precision config #614
  • Bump Megatron version to inlcude --te-precision-config-file #567
  • [Draft] [WIP] Support --fp8-param-gather for mxfp8 #919

• MXFP8

  • Implement Blackwell MXFP8 recipe #512
  • Support DeepSeek v3.2 #963

• NVFP4

  • Implement nvfp4 #546
  • Add nvfp4 quantizer files #907
  • Use TE NVFP4 reference quantizer and add bitwise exact tests #1054

SGLang

• MXFP8 & NVFP4
  • [FlashInfer v0.6.6][RL] Support fp8-last-n-bf16 RL for flashinfer_trtllm_routed moe backend sgl-project/sglang#20214
• MXFP8
  • Add mxfp8 support for online quantization, Triton dense linear, and CUTLASS MoE sgl-project/sglang#17449
  • [RL] Support per-layer mixed FP8/BF16 serving for FP8 checkpoints sgl-project/sglang#18742
  • Expand deep_gemm entrypoint to support more FP8 recipes. sgl-project/sglang#17294
  • [FlashInfer v0.6.4] [RL] Integrate FlashInfer mxfp8 gemm, MoE, and routed MoE sgl-project/sglang#19537
  • [RL] Support mxfp8 DeepSeek V3 sgl-project/sglang#21280
  • [FlashInver v0.6.7] Integrate flashinfer_trtllm mxfp8 gemm sgl-project/sglang#21576
  • [RL] Fix weight update for mxfp8 flashinfer_cutlass gemm backend sgl-project/sglang#22484
• NVFP4
  • [RL] Add an nvfp4 online input scale mode sgl-project/sglang#18012
  • Fix nvfp4 weight update sgl-project/sglang#18085
  • [RL] Refactor NVFP4 shuffling/swizzling to in-place replacement sgl-project/sglang#22204
  • [RL] Support FlashInfer per-token NVFP4 MoE sgl-project/sglang#22918

TransformerEngine

• MXFP8 & NVFP4
  • Add NVTE_BACKWARD_OVERRIDE=high_precision|dequantized NVIDIA/TransformerEngine#2644
  • Optimizations for MXFP8/NVFP4 dequantize kernels NVIDIA/TransformerEngine#2865
• NVFP4
  • Implement per-token NVFP4 fprop recipe NVIDIA/TransformerEngine#2931
  • CuTe DSL kernels in cuDNN frontend

FlashInfer

• MXFP8
  • Implement cutlass_fused_moe mxfp8 flashinfer-ai/flashinfer#2581
• NVFP4
  • [feat] Trtllm-gen Per-token Nvfp4 MoE flashinfer-ai/flashinfer#3027

[hanxdmech-ship-it]

Great work! We're also working on something similar and would love to collaborate and exchange ideas. Looking forward to potential synergies between our projects.