I’ve seen variations of this idea discussed in a few places, and there is a bunch of research happening around embodiment reinforcement training. A few prominent examples here
What you’re describing is very similar to stuff people have done with genetic algorithms where the agents evolve within the environment, the last link above focuses on that approach. I definitely think this is a really promising approach because we don’t have to figure out the algorithm that produces intelligence that way, but can wait for one to evolve instead.
And yeah, it’s going to be really exciting to plug AI models into different kinds of sensory data, it doesn’t even have to be physical world data. You could plug it into any stream of data that has temporal patterns in it, for example weather data, economic activity, or whatever and the AI will end up building a predictive model of it. I really think this is going to be the way forward for making actual AGI.