There is further conversation with another user below this, but that isn't the case in some places, namely Virginia.
Additionally, statutory rape isn't just bound to age of consent. There are myriad other statutes it can apply through, namely family, age difference, and positions of power over minors above the age of consent.