My Thoughts‎ > ‎

Measuring Defensive Performance

posted Oct 16, 2017, 11:54 AM by Robert Vollman
In 2002, Iain Fyffe posited the idea that you could crudely estimate a player's defensive performance by whatever portion of his ice time couldn't be explained by his scoring.

To take a practical example, consider the defensemen on last year's Stanley Cup Champion, Pittsburgh Penguins. Justin Schultz outscored Brian Dumoulin 51-15 in the regular season, and yet in the playoffs when the games matter most, Dumoulin led the team's blue line with an average of 19:03 minutes per night at 5-on-5, while Schultz was mere seconds out of last, with 16:16. Clearly, Dumoulin's defensive contributions were considerable to have earned that much ice time with such modest scoring, and Schultz's must have been very slight.

Yesterday, I decided to play with Fyffe's idea. Here's what I did:
1. I grabbed all the relevant 5-on-5 data from XtraHockeyStats (in minutes!), from 2008-09 to 2016-17.
2. I calculated each player's points per 60 minutes, and their ice time per game.

Now, I did not just compare these, because a player's scoring rate and average ice time is highly contextual. To truly test Fyffe's idea, a player's scoring rate isn't what's important -- it's his scoring rate relative to the team's other options. For example, scoring a lot of points would get a centre a lot more ice time on the Arizona Coyotes than it would on the Pittsburgh Penguins. So I need to keep going.

3. For each team and for each season, I added up points, ice time, and games played, by position (forward or defense).
4. That allowed me to calculate the average points per 60 minutes, and the average ice time per game, by position, for each team and season.
5. Then, I divided a player's actual points per 60 minutes by his team's average points per 60 minutes at his position (NOT counting his own, of course). 
6. I repeated step 5, but for ice time per game.

So, at this point, we know how many points per game a player scored relative to the team's other options at that position. If a player averaged 1.6 points per 60 minutes, and the team's remaining forwards averaged 1.3, then his scoring rate is 1.23 higher. That means that his ice time should also be 1.23 higher than the team average, all things being equal. If it is higher, then he may be very good defensively, and if it is lower, then he may be weak defensively.

7. I summed this information up, and selected everybody who had played at least 200 games over this time span. I have always found that single-season sample sizes are far too small for virtually any statistical purpose, and I doubt this would be an exception.
8. I put it on the following chart so I could see it visually, and confirm that there was a relationship between a player's points per game and average ice time. As one increases, so does the other, exactly as one would expect.

9. I calculated the correlation between points per 60 minutes and ice time per game, and it's 0.48. Crudely, that means that 70% of a player's ice time can be explained by his scoring rate.
10. Using the formula identified by the trend line, I calculated what someone's relative ice time per game should be, based on his relative scoring per 60 minutes.

At last, I sorted them by the widest difference, looking to identify which players had the most extra ice time, and which players had the least. Let's look at each group.

Players With Extra Ice Time

In theory, those with a lot more ice time than expected were bringing extra contributions to the table. Primarily that would be defensive contributions, but it could also be grit, leadership, and who knows what else.

On defense, the greatest outlier was Greg Zanon, a former NHL defensemen for four different teams. Go look up his data, and you'll see why he's here. He took only 333 shots and scored just 62 points in 493 career games, and yet he averaged 19:51 minutes per game. His scoring rate was less than half (48%) the team's other defensemen, and yet his ice time was 11% higher. So, it stands to reason that he was playing well defensively to get all that ice time.

Other defensemen high on the list include Jonas Brodin, Ryan Suter, Jay Bouwmeester, Mattias Ohlund, Drew Doughty, Eric Brewer, Brenden Dillon, Robyn Regehr, Braydon Coburn, Andrew MacDonald, Marc Staal, Alexei Emelin, and Francois Beauchemin.

Most of these defensemen can be safely classified as strong defensive defensemen. Others simply bring to light the question of coaching error. A player's ice time is a function of many things: his offense, his defense, his grit/leadership/intangibles, the team's other options, and the coach's assessment. In some cases, we have to wonder about the coach's assessment -- but I'll leave that as an exercise for the reader. 

Among forwards, Ilya Kovalchuk surprisingly led the list. His scoring was 33% higher than expected, but his ice time was 40% higher. That may not seem like a big deal, but look at the trend line. Getting 40% more ice time corresponds to less than 20% more scoring, not 33%. Plus, it's hard to get much more than 40% extra ice time, no matter how much you score.

Kovalchuk's high result may also be the function of playing for some relatively talent-thin teams that really didn't have any option other than just to play Kovalchuk all the time. In essence his scoring rate was so high that no amount of ice time was too high. Besides, the entirety of what Kovalchuk brought to the table might be understated, especially given how dramatically his teams plunged down the standings when he left town.

Other forwards on the list include Sami Pahlsson, Ryan Kesler, Dany Heatley, Jordan Staal, Ryan Nugent-Hopkins, Olli Jokinen, Ryan O'Reilly, Boone Jenner, Chad Larose, Jarome Iginla, Paul Stastny, Mikko Koivu, Ryan Callahan and Ryan Zetterberg. Again, I think it's plain to see which of these are strong defensive players, and which ones merely played for teams that were seriously lacking in other forward options. 

Players With Limited Ice Time

On the flip side, some players get far less ice time than their scoring justifies, because they are serious defensive liabilities. One obvious group of examples are enforcers, which dominate the bottom of the list; Paul Bissonnette, Cam Janssen, Darcy Hordichuk, George Parros, Jody Shelley, and so on and so forth.

Skipping such players, and we see the lowest-ranked defenseman is Marc-Andre Bergeron, which is absolutely no surprise. He was an excellent scorer, but was used as merely a depth option because he wasn't trusted defensively at all. Cody Franson was the second-lowest, which isn't a surprise given that massive disconnect between his above-average stats, and the fact that he is used as a third-pairing guy, and often finds it hard to find a free agent contract at all. Obviously, he is seen as a considerable liability in one respect or another.

Other defensemen on the list include Kurtis Foster, Justin Falk, Dougie Hamilton, John Scott, Matt Gilroy, Ryan Wilson, Brent Burns, Tyson Barrie, Yannick Weber, Steve Montador, Nate Prosser, Torey Krug, Matt Dumba, and so on. Perhaps you view some of these as coaching errors, but their presence on this list is consistent with most of their reputations.

Up front it's mostly just tough guys, but there's also Tyler Toffoli, Sven Baertschi, Dale Weise, Erik Haula, Kyle Chipchura, Patrik Elias, Ryan Spooner, Tanner Pearson, Derek MacKenzie, and others. 


I also wanted to look for players who were good scorers and great defensively, like Duncan Keith. Because Fyffe's theory doesn't really help us identify the defensive skill of those who have great offensive skill. I mean, Keith's ice time can be justified either way. As it stands, Keith's scoring rate was 41.5% higher than the team's other defenseman, and his ice time was 23.1% higher. Which is actually just a slight bit below the trend line. He's mixed up with several strong two-way players. To use another example, Pavel Datsyuk's scoring was 37.4% higher, and his ice time was 25.9% higher.

I'll be exploring this more in the future, but I thought that I'd share my preliminary assessments.