Question about the 60k / 10k FFHQ split mentioned in README

[Laura-musiol]

Hi, thanks for releasing the code and benchmarks.

While reading the README, I noticed the statement that for FFHQ the first 60k images are used for training and the remaining 10k for testing.

In the paper, you also mention that the experiments use samples from
https://github.com/layer6ai-labs/dgm-eval

However, looking through that repository and the linked generative model repositories, it seems that the models there were trained on the full 70k FFHQ dataset, rather than using a fixed 60k / 10k split.

Because of this, I'm unsure how the data was actually prepared for the experiments.

Could you clarify:

• Were the generative models retrained using the fixed 60k / 10k FFHQ split mentioned in the README?
• Or were pretrained models from the dgm-eval repository used, which appear to be trained on the full 70k dataset, and the 60k / 10k split was applied only during evaluation?

Thanks in advance for the clarification!

[marcojira]

Thank you for your question!

• We use samples from https://github.com/layer6ai-labs/dgm-eval, there was no retraining.
• The 60k/10k split is used during evaluation to get the different reference sets for FLD.

Unfortunately as you've noted most models did not use a train/test split when working on FFHQ so the overfitting/memorization results aren't particularly meaningful for this dataset (this is why we don't analyze most overfitted faces for example). FLD in this case is more evaluating the quality/diversity of the generated samples.

[Laura-musiol]

Thanks for the quick response! That clarifies exactly what I was asking about.