September 23, 2010
If you were asked to rate the CFL teams after 11 weeks, how would you go about it? There's the basic idea of going by wins, which indicates that Calgary's really good, Montreal, Saskatchewan, Hamilton and Toronto are in the next pack, and Winnipeg, B.C. and Edmonton are at the bottom. That doesn't seem too unreasonable, but wins alone aren't necessarily the most convincing metric. You could incorporate points for and against, which helps, but doesn't necessarily tell the whole story either. You could always make an attempt to rank teams based on your own impressions of them, but that brings your own biases into it; even if you don't support a specific team, you may be inclined to reward performances that impress you more than the specific stats deserve. Of course, you can always just argue that your own team is the best.
None of these approaches are necessarily wrong, but they each have their own implicit oversights. To get a better picture of where teams really are, it's worth looking at as many different angles as possible. One angle that isn't commonly considered in the CFL is the use of statistical rankings systems like the Ratings Percentage Index and the Simple Rating System. Rob Pettapiece, who co-edits The CIS Blog with me, has a math degree from the University of Waterloo and has worked for Statistics Canada. He has run RPI and SRS calculations for CIS sports for years, and recently did the same for the CFL this season through this past weekend's games, coming up with some interesting results in the process.
Calgary tops the list in both RPI and SRS, which isn't surprising considering their 9-2 record. They also have an incredible +151 point differential (they've scored 396 points while allowing just 245), which demonstrates their dominance in more than just these statistical models. However, the two systems have ideas about the league's other teams that don't necessarily match up with the standings.
Perhaps most notable are the systems' takes on the Southern Ontario teams., RPI puts Hamilton (yes, Hamilton!) in a tie for second with Montreal, with Saskatchewan just behind. RPI has Toronto fifth, just behind Saskatchewan, while SRS puts the 6-5 Argonauts in seventh, ahead of only the lowly Eskimos and behind the 3-8 Blue Bombers and Lions. SRS loves Calgary and Montreal, but thinks Saskatchewan is a long ways below them, and has Winnipeg and Hamilton pegged as almost exactly average.
The complete results and my interview with Pettapiece about how exactly he puts these statistics together are after the jump.
Here's the RPI sheet:
And the SRS one:
Now, time for the breakdown of how these work. The systems focus on different aspects of a team's record, which Pettapiece said makes them complementary.
"I like having two systems that do different things," he said. "One (RPI) is more concerned with if you're at home, the other (SRS) is more concerned with margin of victory."
As Pettapiece explained it, RPI (most famously used by NCAA basketball) is predominantly used to adjust teams' records thanks to home/away splits and strength of schedule. The first thing he does is to create an adjusted winning percentage for each team (column D). Each game still needs to count as both a win and a loss, but this RPI doesn't treat home and road losses equally the way a conventional standings table does. It gives a team 0.6 of a win for winning at home and 1.4 wins for winning on the road. Losing on the road only counts as 0.6 of a loss, while losing at home is worth 1.4 losses. Pettapiece said this reflects the importance of home-field advantage.
"It either amplifies the game if you do what's unexpected or diminishes it if you do what's expected," he said.
A recency factor is then applied, which gives more recent games greater weight. This factor is calculated with a standard exponential decay equation of the sort used in calculating half-lives of radioactive isotopes or compound interest. The half-life in this case is 90 days. Pre-season games are also considered, but their impact is further halved.
For example, the preseason game between Hamilton and Toronto on June 13 carries a weight of approximately 0.233 (all figures rounded to three decimal places) according to this formula. Toronto hosted the game and won, so they would receive 0.233*0.6, or 0.140 of a win. Hamilton would receive 0.140 of a
loss. The Tiger-Cats' first regular-season game, which they lost 49-29 on the road in Winnipeg, counts as 0.540*0.6, or 0.324 of a loss. Their most recent game, a 35-31 win on the road against B.C., counts as 0.985*1.4, or 1.379 of a win. Their adjusted winning percentage (0.53) is obtained by dividing their adjusted wins total by the total of their adjusted wins and losses. This makes up 25 per cent of the final RPI number.
Once adjusted win percentages are calculated for all teams, Pettapiece figures out their opponents' average adjusted winning percentage. This is an ongoing calculation, so games played after the teams met still matter, but the opponent's record against the team he's calculating opponent winning percentage for is removed. Thus, Winnipeg's winning percentage is the same number each time they play Hamilton, and it's simply their season-long adjusted winning percentage with results from games against the Tiger-Cats removed. All of Hamilton's opponents' winning percentages would then be tallied and divided by the number of games they've played to provide the opponents' winning percentage number.
That works out to 0.553 for Hamilton, the highest number of any CFL team, and that's a huge reason they're tied for second in the rankings, as opponents' adjusted winning percentage counts for 50 per cent of the final RPI number. This is also why Calgary and Montreal aren't as dominant in RPI as you might expect, as they have the lowest opponents' adjusted winning percentage. In real terms, this means the RPI is telling us that (by its terms) Hamilton has faced a more difficult schedule than any other CFL team, while the top teams have played the easiest opponents (logical considering that they don't have to play themselves).
The last RPI component is opponents' opponents' winning percentage, which is calculated in the same way but just goes a step further. For example, if Hamilton is playing Winnipeg, you'd do the opponents' winning percentage calculation (season-long) from the Blue Bombers' perspective. This tells you about the strength of Winnipeg's wins; have they beaten good teams, or just the bottom-dwellers? This is then repeated for all of Hamilton's opponents and averaged out to give 0.488, which counts for the final 25 per cent of the Tiger-Cats' RPI number.
SRS is too complicated to explain in full here, but it's a ranking system that's been around for decades, and the math behind it is outlined in full at Pro Football Reference. Basically, it considers teams' point margins and the point margins of their opponents. Unlike RPI, it doesn't care about whether wins are at home or on the road. One of the interesting elements with it is that because it's on a point scale, unlike RPI, you can use it as a predictor similar to a gambling point spread. For example, for this week's neutral-site Touchdown Atlantic game in Moncton between Edmonton and Toronto, you'd simply look at the difference between Edmonton's -6.02 rating from Toronto's -2.15 rating, which would suggest that the Argonauts would win by four points.
If you want to take home-field advantage into account, Pettapiece said a simple calculation (all points scored by home teams this year - all points scored by road teams) leads him to believe that home field is worth about a touchdown in any given game. Thus, SRS would predict that Saturday's game in Calgary between the Stampeders (+4.95) and the B.C. Lions (-0.91) would be a 13-point Calgary victory. That's even more of a beatdown than the Vegas line of Calgary-10.5 is predicting, which does not bode well for B.C.
One other note on the SRS system Pettapiece uses is that it caps blowout wins at 25 points; thus, a team that wins by 30 would only receive credit for a 25-point win, and the defeated team would only be recorded as losing by 25. The idea there is to prevent one awful loss from utterly destroying a team's SRS rating (or conversely, inflating another team's).
"You don't want to reward teams for running up the score against the Eskimos," Pettapiece said.
Pettapiece said that means the spread between the best and worst teams isn't as high as it might be otherwise, however.
"I think Edmonton might be more than six points worse than average," he said.
What does this all mean? Well, if you compare these systems' results to the league standings, both suggest that the Tiger-Cats could be better than many imagine. RPI loves them for winning unexpected games on the road (like last week's one in B.C.) and playing a tough schedule. SRS isn't as high on them, but it still has them a respectable fourth overall. Both systems also suggest that the Argonauts may be worse than their record would suggest, thanks to narrow victories, big losses and struggles on the road; the SRS conclusion that Toronto's the second-worst team in the league would come as a surprise to many.
In the end, both of these systems are just another way to look at the league. They're not necessarily completely accurate in their team rankings, but they provide us with some interesting information on strengths of schedule, margins of victory and which teams might be better or worse than straight wins and losses select. Having more information is always a good thing in my books, and these rankings represent a significant leap forward on that front.