Seeing other people lose it, one after the other at unpredictable times, is what would really screw up the ones who still had it.
Humans are really bad at dealing with expected events which come at unpredictable times, both bad things and good things - the former can induce constant fear and anxiety and is apparently used in torture, whilst the later (for example the receiving of “premiums” at random intervals) can induce gambling addiction.
(I wasn’t clear before that it’s after a random period different per individual. Sorry to any would be torturer).