No, aspartame does not give you a headache. It’s probably the caffeine that’s present in many sodas, such as coke. There is no known biological mechanism for aspartame to give you a headache. It’s just confirmation bias.
The main harm of sugary drinks are the incredibly harmful effects from huge insulin spikes and damaging your liver. Teeth is pretty low on the list.
Diet soda drinks are still harmful to your teeth because of their high acidity. Not as harmful as sugary drinks, but still. It’s the only proven major health concern of diet sodas.