the benefit of sampling above 20khz is that you can even out the signal over a period of time which will make it more accurate for frequencies up to 20khz. you will get a noisy signal but all the noise is in frequencies you can't hear.
you also need to consider how the voltage is generated. in general there are limits regarding how quickly can voltage surge. e.g. you can't reproduce a square wave properly in most cases after amplification. in the end this makes dsd much less relevant.
you also need to consider that the reproduction is not perfect and neither is the recording. e.g. a square wave will not be captured properly
edit: I forgot to mention that the slew rate limit has a parallel on the speaker/headphone membrane but it's much worse than the amp since it's a physical object with momentum.