This is how most magic works in Unknown Armies, by the way. That game is fantastic and just drips with flavor and insight (2nd edition anyway. I haven’t read 3rd)
If your book mage wants to charge up, he has to go find some rare valuable books and add them to his collection. If the chaos mage wants a charge, she has to start taking some risks. You can do some crazy things with a major charge, but getting one is probably work. Or win Russian roulette if you’re a chaos mage.
In practice it can be difficult with a large group where everyone has their own obsession. It provides good down time options and plot hooks, though.