My view on it (part of my response to someone who PM'd me on the subject, I don't feel like re-composing it):
In real life, outside-the-Internet society, jerks and annoyances aren't jailed or exiled, they're shunned and ignored (by those who consider them jerks and annoyances). I don't see why it should be different in this online society. If he broke rules, sure, I'd agree with banning him. Being annoying to some/many isn't enough.
For example, PapaG was annoying to many (and another person widely accused of being a troll) and he was (within the rules of the community) shunned and/or criticized. He realized he wasn't much liked and it stopped being worth it for him to come to the site and create fights. I actually wish it hadn't come to that, but I think that's a lot better than him simply being banned by mod action.
I prefer to err on the side of less bannings than more. So unless a person is a spammer, a rules-breaker or some other egregious offender, I don't agree with a ban.
