Im not opposed to immigration, but it is simply a fact of reality that we are doing it wrong, and the result is a re emerging right.
As for the reduction in labor force that would occur without a constant influx of desperate unskilled workers: the basic system of supply and demand would then cause wages to skyrocket, transferring wealth to the lower and middle class on a large scale in the short term, and spur technological advancements in automation to compensate, proper governmental regulation provided to prevent outsourcing.
This in turn will require a shift to full UBI and automation taxes to keep society functional, finally putting us on a track to a post-scarcity future.
In short, constant, unrestricted immigration does both the refugees facing growing resentment and the marginalized native lower class a disservice by helping the ruling class to keep their labor suppressed.