Absolutely, I think there should be certain objective things that have to happen before “fearing for your life” is a valid defence.
Someone breaking your window after trying to enter forcefully through your door is where I start thinking it’s okay to use a deadly weapon to defend yourself.
Someone knocking on your door (regardless of the time of day) is not a reasonable situation to fear for your life, at least to the extent where you attack the person.