This is a continuation of yesterday's interview with GT Industrial Engineer Professor Sokol. Part I explained Professor Sokol's relationship to Georgia Tech and briefly described the LRMC system he has developed to predict the results of the NCAA basketball tournament. Today's questions dive deeper into Professor Sokol's system, its relationship with the NCAA Tournament selection committee, and Sokol's opinions on the use of ranking systems in other sports:
FTRS: An ESPN.com article has your colleague, George Nemhauser stating that the NCAA is "stuck on the RPI." What is wrong with the RPI ranking system?
Sokol: RPI's biggest problem is that it ignores the difference between close games and blowouts, and that difference really tells a lot about what a game means.
We actually met with the NCAA folks to talk about this sort of thing, and they feel that incorporating margin of victory in any way (even a vague one) would encourage running-up the score, and also could raise the stakes for potential point-shavers/gamblers.
We understand their concerns (and with 340+ teams of unpaid players to police, it's much more serious than, say, at the professional level where there are only 10% as many players, and they're paid enough by their teams that the payoff from point shaving is less attractive). So, as a compromise, we now provide the NCAA with a version of LRMC that doesn't take margin of victory into account. It's certainly not as accurate as our regular LRMC, but it's still more accurate than RPI, so it's now one of many pieces of input that the NCAA gives their selection committee.
FTRS: Besides basketball, do you follow any other sports? Have you created any prediction or ranking systems for other sports like you have for NCAA basketball?
Sokol: I'm a big baseball fan, and also follow pro and college football. I'll watch other major sports (NBA, NHL) occasionally but I don't keep up with them much, and I'll jump on the Team USA bandwagon for the Olympics, World Cup, etc. for just about any sport.
For most pro sports, an LRMC-type system isn't so useful because teams play such a high fraction (often 100%) of the other teams before the playoffs. College sports are where it's most needed, because teams play such a small fraction (less than 10%) of the other teams, so the answer to "how will X fare against Y" is less obvious. (For example, NBA.com asked us to try applying LRMC to the NBA, and we discovered it's not significantly better than just looking at teams' win/loss records.)
We've tried LRMC on college football (using regular-season results to predict bowl results), but there's an important difference that makes it less successful: motivation. In the NCAA Tournament, everyone has exactly the same thing at stake: a spot in the next round. In bowl games, some teams are excited to be there (and will play better) while other teams are in a "minor bowl" compared to their usual expectation, and don't care so much.
We've actually discovered the same thing in college basketball at the conference tournament stage of the season -- many of the major-conference teams know they're in the NCAA tournament win or lose at the conference level, so they play with less motivation. As a result, these games are less predictable than NCAA tournament games, but not in ways that LRMC can handle since it often differs by coach. For example, Mike Krzyzewski says he still motivates his team to win the conference tournament, while Lute Olson used to say the best outcome for him was to lose in the first round so the team could rest up for the NCAA tournament.
FTRS: Do you think basketball should use a rating system similar to the Bowl Championship Series to determine seedings and berths in the tournament over an actual selection committee?
Sokol: That's a tough question, for several reasons.
First, there are some things LRMC doesn't include that make a difference for selection/seeding. Injuries/suspensions are the biggest factor, so a human would need to make some adjustment.
For the last few years, the selection committee has done a very good job -- the seeds have held up well. But since committee members rotate on and off, that might not be true all the time, so I'd be in favor of some sort of ranking system (with adjustment for injury).
Second, there are a couple of philosophical questions that would have to be resolved:
(1) Who should get at-large bids?
Personally, I think it should be the best teams, regardless of anything else.
But other people think it should be a reward for playing tough out-of-conference schedules, even if another team has a harder schedule overall because it's in a tougher conference. My opinion is that strength-of-schedule is a tool for helping to determine who's best, rather than an end in itself -- but from a dollars-and-cents point of view, I can understand the incentive to encourage tough out-of-conference schedules; each out-of-conference matchup between two good teams is an extra game that'll get good TV ratings, so the networks will pay more for TV contracts.
And still other people think there should be some sort of "affirmative action" for small conferences, "to be fair" to those teams. My opinion is that it's not being fair; it's giving them something they don't deserve at the expense of being fair to major-conference teams that are better.
(2) What's the point of the NCAA Tournament?
Personally, I think it's to figure out which team is playing best at the end of the season, and declare them the national champion.
But there's some good evidence that the tournament's huge popularity is not because it picks a champion, but because there are lots of upsets and Cinderellas. In that case, it might be more popular to include more smaller teams and worse teams, since odds are that one or two of them will win a game or two and keep all the Cinderella-lovers happy. [Disclosure: my wife thinks I'm nuts because in a game where I don't have a rooting interest, I'll usually root for the better team because I like to see teams get what they "deserve". She says (correctly) that I'm weird, since most people like to root for the underdog.]
You mentioned the BCS in your question, and it seems to me that the BCS has struck a compromise on these issues. The main purpose of the BCS is to determine a national champion by having the two best teams play each other, and (as a side issue) to give 8 other teams (and their conferences) significant exposure (and money) in a big game.
This year especially, I think the BCS did a nice job of that. They correctly sorted out the two best teams from the undefeated set (does anyone really think TCU and Boise State were better than Texas and Alabama? The contrary arguments all were about "fairness" and "giving them a chance"), but also gave the smaller-conference teams a chance at exposure and money.
(Editors Note: Professor Sokol's opinions, as he told us, do not reflect the opinions of his colleagues, Georgia Tech, nor the NCAA.)
The short answer is that yes -- although the selection committee gets a lot of quantitative information and has done a nice job lately, I'd like to see more quantitative structure imposed in cases where the committee members rotate off and the incoming members are less [experienced]...
Part III will be posted tomorrow morning. Professor Sokol talks about predicting Cinderellas and the accuracy of initial and current models.
Note: Dr. Sokol will be at the Lindbergh Taco Mac in person to discuss his system and its methodologies on Saturday, February 13th at 7:15 pm right before the Wake Forest basketball game. This could be an excellent opportunity for you to learn more about the LRMC system on a deeper level.
Click here for an LRMC Summary written by Dr. Sokol himself
Click here for the LRMC's 2009 NCAAT projections.
Click here for the LRMC's current 2010 basketball rankings.
Once again, Part 1 can be viewed here.