Your mistake is thinking that "sports" and "athletics" are one in the same. I don't believe that's true. Some things are sports, some are athletics, and some are both.
Sports are competition-based physical activities, and outcomes are determined by objective criteria, such as who scores more points, who finishes first or who finishes in the least number of attempts. Baseball and golf are sports, but not athletics. Anything that John Daly or John Kruk can carve out a career in obviously does not require athleticism. Let's put them in 'A' in the above diagram.
Athletics are activities that requires physical abilities and attributes above and beyond what sports demand. Some activities, such as figure skating or diving have competitions, but they are based on subjective criteria (scoring by judges) These are in 'B'
Some activities fall under both categories. Football, soccer, hockey, basketball, track & field are all sports, and they all require athleticism. They fall under 'A & B'.
Because people misuse the terms or use them interchangeably there are always going to be arguments of this nature. It's far too clumsy to rename such an award the "AP Male Athlete and/or Sportsman of the Year", and even then, some people will bitch about it.
The thing that I find funny about the way this year's voting went down - if you're gonna vote for Jimmie Johnson on the male side, why not vote for the jockey instead of the horse who came in second on the female side? Or, vice versa - if you're gonna vote for the horse, why not vote for Jimmie Johnson's car instead of the man who drove it?
cha-CHING