My understanding is that polyethylene is relatively stable and inert. That’s why it’s commonly used in food packaging, medical implants, and other applications where you really do not want things leaching out. So it’s probably not a bad choice as a filler material.
I would be more worried about the really cheap plastics used in low-end consumer products. Plastics used in electronics also often have fire retardants to meet fire safety regulations, and those can definitely leach out over time. The soft grip materials on things like toothbrushes and screwdrivers are gross, too; sometimes that stuff degrades into sticky goo and starts releasing oily compounds.