Just call it a stadium. The names will vary with being called a field, park, or stadium, but if you say stadium everyone will know what you mean.
E.g. Turn left by (the stadium/Wrigley Field/Fenway Park/Dodger Stadium)
ETA: Field is generally used for small ones, like you would see in a public park. So if you go to city park to play football/soccer/baseball/whatever, you would call it the $SPORT field if it is on grass. If it is on a hard surface like basketball or tennis, you would say court. Stadium is for large structures with several thousand+ of spectators, and again the proper names are inconsistent, but stadium as a general term works.
Hopefully this helps. American English is all sorts of ambiguous and inconsistent. One positive is it is nice not having gendered nouns and it is forgiving enough that even with grammar errors, a native speaker will know what you mean, like the plural of cow is sometimes cattle when you used as an adjective “cattle farm”. Nobody would be confused if you said “cows farm”, but along with the inevitable accent, it would be a tell that you aren’t a native speaker