The whole term "cancel culture" was about celebrities--specifically, people with power like Harvey Weinstein, Bill Cosby, etc. That was during the #metoo movement. That's when conservatives began complaining about how people were being "cancelled" and that even their prior work was not being distributed.
Then they began applying it to "regular people" being fired for doing stupid things. The things you claim aren't really cancel culture. So no, I think they were upset by it applying to celebrities and then began applying it to any conservative, famous or not, who caught consequences.
Being blackballed when he was still an effective quarterback in the prime of this career obviously counts as being cancelled. On ability alone, he was better than plenty of quarterbacks in the league at the time, including some starters.
What you're really saying is, "Shouldn't we be concerned when liberals cancel conservatives, even if conservatives have been doing this for decades?" And that was my original point.