I don't think calling more fouls is "letting it go".
Both calls looked correct to me--I don't think that it's legal to inhibit movement with your body like that away from the ball, although I'm no expert on NBA rules. And Bayless clearly jumped into the defender on his jumpshot.