I work on networking for distributed rendering for a major cloud provider- very familiar with gpu architecture and use-cases :)
Saying they do more math is a bit tricky. The CPU does crazy types of very complicated math and accomplishes tasks we still have a hard time offloading to GPUs.
I agree with the rest of your statement as a good explanation for why GPUs can do faster and more efficient batch processing of the workloads that can be fit to the SIMD set up we use for most modern GPUs (ignoring general purpose gpu and fancier compute options)