At this point at least, LLMs require vast amounts of GPU time, and the GPUs used likely need to be fairly close coupled to run at decent speeds, and as such one couldn’t spread itself over the vast network of consumer computers. At least not in a way that would still allow it to learn.
Any near-term AGI or similar would be based on a similar approach, making it fairly “safe” in that respect at least. Only a million other disruptive ways it can affect society…
In the event of such a worm/virus, we forget actually do have a very effective nuclear option: switch off the computers and re-image them. Painful as it might be, we could even shut off or partition the global IP network temporarily if faced with such an existential threat.
We don’t live in a sci-fi novel after all, and this distributed AI wouldn’t be able to hide in a single machine, plotting against us. It would only be able to “think” as a giant networked cluster, something easy to detect and disrupt.