But JoJo got good long before halfway through. For me it starts with Battle Tendency but for many it's good from Stardust Crusaders on. That leaves many more seasons and even more manga.
My advice for anyone getting into JoJo would probably be start with Stardust Crusaders, they explain why Dio is a big deal in the narrative and the mask isn't relevant anymore so none of it really matters.
You are right that the first season is a total drag and very skippable though.