We don't know the impact. We do know the impact of natural crossing. That's the point.
And other points: GMO foods include, for example, "roundup ready" so roundup and other herbicides won't kill them. But the herbicides still harm ground, water, beneficial insects, domestic animals, and BTW people. So instead of finding a long term solution, managed pest control, it's a short term fix that in the long run makes things worse; we are already seeing "super weeds" that evolved resistance to roundup, just as widespread use of DDT favored evolution of DDT-resistant insects.
Next point: GMO seeds are pricing out small farmers, especially in the developing world. For example, in Iraq the "redevelopment" funds from the U.S. specifically prohibit farmers from saving seed for the next year's planting, forcing them to purchase expensive GMO seeds from major corporations.
Next point: They are reducing biodiversity. That means an infection, insect invasion, disease, wreaks havoc because all the plants (or animals) are identical. In normal farming, some plants or animals get wiped out but others survive. Species require biodiversity to maintain themselves.
Next point: No one is even proposing banning these items, just labeling them. If corporations have free speech, don't the rest of us?
