Well, the ideal would probably be to train a discriminator based on human ratings of generated outputs.
Take generation 0 (G0), produce output which is accepted or rejected based on humans, train a discriminator to predict those ratings off output, and then use the combined accepted outputs from humans and trained discriminator to train G1.
Repeat again for G1, G2, G3, etc.
My guess would be that the end result would continue to get better and better rather than worse.
The problem is if the diffusion model can’t properly reject weird hands or pupils, those magnify in subsequent rounds.
But there’s likely adaptive and maladaptive tendencies in the diffusion model, and adding a halfway decent filter between human selection and synthetic selection of outputs separate from the diffusion model itself would effectively curb the magnification here.