better sports predictor: computer or games?

[Don]

I see a lot of sites these day that try to predict the outcome of some game by running a simulation, and for a while I thought you might as well ask NBA Live 2012 what the answer is, but on second thought, it might actually be worse.

As far as I can tell, games try to break down the players into stats, i.e. Peyton Manning has an accuracy of 99 because it looks like he's the most accurate guy in the NFL, and the game models 99 accuracy as such and such. On the other hand, it looks like most computer models that aren't games try to model a character as say '105 Quarterback Rating' or '35 PER' and then try to model the characters solely based on that.

But stuff like QB rating or PER rating is observed after the fact the game is played. That is, you're saying I see this guy plays well and according to my formula that translates to a power level of X. If you go this way then the winner is usually the team with the highest combined power level. If you attempt to model sports like a game down into individual stats, it sounds at least possible that maybe the highest combined stats doesn't always win since maybe you've high accuracy but they got a good defense or whatever.

Besides, game makers generally have a significant incentive to model the game correctly, since nobody likes to play a sports game that doesn't remotely look like the real thing. On the other hand most computer model seems to be developed by just a random guy on the Internet. I'm not saying more is always better but unless someone has some fundamental insight to model sports, I'd take quantity here because more people = more thorough usually.

I'm not saying models are useless, but right now I see no reason to believe it'd be better than running the same prediction in Madden/NBA Live 2012. In fact, I'm more inclined to trust the result from a game since it's actually attempting to model some of the physical aspect of the game, as opposed to models that simply take a player and equate them to some power level.

[Flip]

I guess you are trying to figure out whether actual stats are better against a rating of those stats? If a QB in real life is very good, Madden gives them a high rating, but thats not the same as plugging in yards, completion %, scrambles, etc into a huge computer. Other factors such as how well one team plays at home or against another particular team is probably not calculated into the Madden formula either...

[Don]

Well those computer simluations that claim to do 5000 repetitions or whatever are presumably not trying to simulate a game down to the physical level, i.e. if LeBron has a strength of X and a speed of Y what he do in all those different games. Otherwise that'd seem to be way too time consuming. It's more like LeBron has a basketball power level of X against his enemy who has a power level of Y so he's expected to score 28.5 points and grab 7.6 rebounds.

But the biggest problem I have is say you run one of those simulations with say 2001 Lakers against say the Bulls in their prime. You'll notice Shaq only score roughly his average presumably because his offensive level gets neutralized by the Bull's defense level. However, I don't care what is Michael Jordan/Dennis Rodman/Scottie Pippen's defensive power level. None of these guys can possibly guard Shaq just because of the size/strength difference. Sure, you can say Michael Jordan is a better player overall than Shaq on offense, but Shaq's physical abilities makes him virtually unstoppable in a matchup like this. This doesn't mean the Lakers will always win but you should not get a result of Shaq scoring roughly like his average (arguably below, because Bulls are a strong defense team).

Lebron Jame and Kobe Bryant might have similar basketball stats, but their style of playing is very different. Kobe can be sometimes by stopped by an elite defender since his game isn't predicated on him being stronger/faster than anyone else. On the other hand LeBron's game basically looks like something you might expect from a manga when he drives and 3 people just bounce away from him, because he's just that much stronger/faster than anyone else. I find it hard to believe this doesn't account for a difference especially if you've some hypothetical fight across time that features the best players ever, but these simluations always show that there's no real difference between Michael Jordan, Kobe Bryant, LeBron James, or Shaq beyond what their basketball contribution would indicate. If you're playing some kind of all-time all-star game, I'd definitely think you'll rely on mismatches, i.e. nobody could guard Shaq or LeBron but people have some success guarding Jordan or Kobe. The Lakers, even during their dynasty years, always had a weakness in PG since Derek Fisher just isn't very good. If you watch the games you can tell most teams attack the PG position since Lakers are strong everywhere else. But I never see things like that from a simulation, i.e. whatever PG plays the Lakers never get some insane stats beyond the fact that Derek Fisher is probably a below-average defender, even though most PGs put up MVP like numbers because that's the only spot that the Lakers are particularly weak at especially in the Shaq-Kobe years. You'd be crazy to try to attack Shaq or Kobe!