Did racist use the "biological advantage" argument when Black athletes started competing alongside white athletes?

Given that racists and slavers used the “natural physical strength” of black people to justify putting them on hard labor and some medics still think that blacks has higher resistance to pain, I wonder if when black athletes started to join mixed race sport teams, some racist would have used the same “biological...