What If Sports

After this article in the Sun Times (part of a series of stories for the 25th anniversary of Flyin Illini) I saw this on twitter:

The big thing for me was not the comparison between the teams but the prospect of this website that simulated games between whatever teams you want.

Instantly, I doubted it. I mean, anybody can "run a simulation" and spit you out a score. It doesn't really mean it's accurate. And there's no way to test it anyways.

I started thinking about the Chiefs vs Broncos this year. They played twice and the Broncos won twice. If you run the simulation 20 times and it says that the Broncos would win 15 out of 20 or 18 out of 20, you don't really have any way to believe it either way.

And then I realized. There is a way to test their simulation accuracy. Not between historical teams or in football. But in baseball and basketball, teams play the same teams a lot in the same season. If I compared the simulations to actual results. If they're close enough, I can trust the site.

Calibration Test #1: MLB

I picked Boston vs Tampa Bay in 2013 because they played 23 times including the postseason.
Boston won 15 for a winning percentage of .652.

I picked this first and now am going to simulate the 20 games, 10 home for each team. I'm picking 20 as hopefully a large enough sample size to get a decent idea. 10,000 would be better. But I don't have all day and 20 is better than simulating three times.

Simulation Results: Boston won 13 out of 20 simulations for a percentage of .650. Fucking nailed it. I would have considered 11 to 15 (.550 to .750) a success. I am totally surprised.

Calibration Test #2: NBA

I picked Chicago vs Indiana in 2011 because they played 9 times including the postseason and Chicago won 7 for a winning percentage of .778.

Let's see how the simulation stacks up with 20 tries.

Chicago won 16 out of 20 simulations for a percentage of .800. Again, spot on. I'm super impressed. These were not cherry picked simulations. I picked the teams and then ran the simulations. I'm in.

- - -

Comparison #1: 1989 Illini vs 2005 Illini

Might as well start here. 20 simulations...

2005 Illini win 14...70% of the time.

Comparison #2: 2013 Chiefs vs 2013 Broncos

Chiefs couldn't win one in two tries. They can at least get one right?

Broncos win 17...85% of the time. They won 10 of 10 at home, 7 of 10 on the road.

I think that seems fair. The Broncos were always going to win that home game. But at Arrowhead, the Chiefs take 1 out of 3.

Comparison #3: Super Bowl 48: Seahawks vs Broncos

This was a beat down, but how many times would the Seahawks really win? KJ Wright said they'd win 90 out of 100. After 20 simulations in the Meadowlands...

Each team won 10 simulations.

Comparison #4: Wild-Card Playoffs: Chiefs at Colts

That game. How many times would the Chiefs have won at Lucas Oil Stadium?

10. Even giving the Colts all 20 home games, each team wins 10. I'm okay with this.

Comparison #5: 2005 National Championship: Illini vs Tarheels

This is the big one. The one we should have won. Right?

North Carolina wins 15 out of 20. 75%.

This is the first one that shocks me. Oh well, what can you do?