You’re not lying. Looking for the hidden quests without knowing if you had already done them and just sort of blindly farming for a bit took by far the most time.
The last one I was missing in the end was the crumpled note in Caldeum Prison which has a low drop rate to boot.