As a poly and bisexual trans person, I really don't believe that poly people are or should be part of the LGBT community. The vast majority of poly people are straight and cisgender and are either unaffected by or apathetic towards the attacks on the rights of LGBT people.
I think that including other groups like poly people and furries (which you didn't mention, but I have seen others advocate for) into the larger community "umbrella" isn't appropriate and, in my opinion, distracts from the real violence and oppression that many of us still face. People who are in sexual minority groups absolutely deserve support and respect but I don't think that the best way to do that is to call them LGBT.