It’s hard to know for sure, but I’m guessing if you’re forcing hevc_nvenc in ffmpeg then handbrake is using libx265. Nvidia’s encoder rips through files, but that speed comes at the cost of bigger file sizes. libx265 uses the cpu so it’ll be much slower, but you’ll probably see files 30% smaller. As for audio, you could always try -c:a copy as an option for ffmpeg. I don’t actually use handbrake myself, so I don’t know how to directly copy streams.
As a side note, if you don’t need to worry about licensing issues mp4 will compress much better than mkv.