Agreed, the charges idea was a great suggestion by Toaster. It should help prevent the “but I was only wildshaped for 20 seconds” type of complaints.
Either way time is weird in D&D.
Assume that you are involved in a “long” combat at 10 rounds. Wildshape lasts hours, so yes I wildshape into a dire wolf and finish the battle in 1 minute of in game time; but being a wolf is not that useful out of combat…so I “waste” 59 minutes of wildshape time at Lvl 2 and 119 minutes at Lvl 4 etc…you could use some of the time saying that I was scouting the area around the battle for a few minutes…but still, the point stands.