Question about training data and handling unavailable rule-based rewards

[zzzqzhou]

Hi, thanks for releasing this work. I have a question about the exact training data and reward computation used in your work.

Could you clarify which prompt datasets and splits were used for training? In particular:

• Did you directly follow the DiffusionNFT setup, using Pick-a-Pic for the model-based rewards and the FlowGRPO training sets for GenEval and OCR?
• How were prompts from Pick-a-Pic, GenEval, and OCR mixed during simultaneous multi-reward training?
• What sampling ratio or batch composition was used for the different prompt sources?

I am also unsure how rule-based rewards are handled when they are not applicable to a sample. For example, a normal Pick-a-Pic prompt may not contain target text for computing the OCR reward, and it may not have the structured annotations required by GenEval.

In these cases, do you:

• mask out the unavailable reward and exclude the sample from that reward-specific advantage and gradient;
• assign a constant or zero reward;
• compute OCR or GenEval only on their corresponding prompt subsets; or
• use another strategy?

It would also be helpful to know whether each reward-specific advantage is normalized only over samples for which that reward is valid. These details seem important for reproducing the specialist-sample setup and the per-reward gradient computation.

Thanks!