Don't see it as impossible at all.  You are connecting two things that aren't connected.  And, you are saying on one hand it's impossible to separate the money in politics from big govt, and on the other hand the money is so tiny comparatively that it's inconsequential.  Those two hands don't jive. 
But, let's take a look for a moment at what would happen if national elections (to start) were not donor driven.  The elected politicians would still spend the money, they would still (even more so) try and appease the electorate, they simply wouldn't need to make stupid concessions to get the required money to run the campaign.  Some stupid concessions would still need to be made to sway blocks of voters, but that Sheldon Allison or George Soros vote would be weighted the same, or at least closer to that of the average voter.  Perhaps Sheldon and George would still have a modicum more sway than I do, mostly because they employee many people and so a politician might see them as valuable from that perspective, but not just because they have deep pockets to fund fund fund. 
Small government politicians would find it easier to remain small government politicians, big government politicians would find it just as easy to be truer to themselves.  But they could actually appropriate those funds where they felt it was most worthy, not into the pockets of those who had helped them get elected.  Or at least, they could try and appropriate the funds in accordance with the wishes of the electorate and not in accordance of the pockets of the biggest donors.  The sway of the NRA or the Teachers Union would still exist because they are huge organizations representing large blocks of voters, but that sway would be somewhat reduced because their power would end at the size of the voter block in not at the size of their pocketbook.
You tend to say that the 2-3B dollars is a pittance compared to the trillions the govt spends, so it doesn't matter, but the big problem is that it's those groups that spend that 2-3B are able to get preferential treatment in earning money off the Govt spending those trillions.