The more I work with microstats and tracking the formation habits of teams, the more I get into discussions with people who say “there’s no way teams actually do that” or something very similar. Many of the people telling me this are very smart hockey people from all areas of the sport, not just the stats circles I run in and it makes me increasingly wonder about the processes currently used in hockey. It makes me question whether I am working off of the same information as the people who are closer to the game.
Many teams are very secretive about the processes they use to evaluate their players and prepare for facing their opponents. Scouting talent is often a complete mystery as well. The secrets are all to be expected because teams do not want to divulge what they view to be a tactical advantage to their opponents. Keeping other teams in the dark about process also has side effects. It also keeps analysts and fans in the dark. Unless you can get a former coach to spill the beans, those on the outside are left to wonder what the coaching staff and/or analytics department are really doing on a day to day basis.
As part of the stats community, I know that we all tend to get excited when we hear a team talk about using analytics. After hearing tell of ridiculously ineffective stats systems sold to teams, we’ve moved on to wondering what “analytics” teams are really employing. This goes far beyond Corsi and Fenwick of course and to the very heart of the “Summer of Analytics”. I’ve seen a lot of debate about the hiring that teams have done recently. Some have called them PR moves designed to make fans happy or to make the team look progressive. Others have said the teams are just shutting down their most vocal critics. Some believe the teams have hired “stats guys” because they actually know what they are doing and can help the team.
Whatever your view of the hiring, they were not simply about teams hiring someone who knows how to read a usage chart. There are so many possibilities for statistical analysis that the sky really is the limit. The critical aspect of all of it is whether the coaches, executives and scouts use the information they are given. The hiring we’ve seen has ranged from coders to video analysts to statisticians to what I like to call “translators”. I think of translators as people who can take the high end statistical models and information and apply them to the team. The practical application is really the whole point of this stuff in the first place.
There’s a lot of skepticism out there about just how much teams actually do in this regard. We constantly hammer home that Corsi and Fenwick are not the end all be all to analyzing hockey and yet whenever someone points out that a team uses analytics the immediate response is “well then why did they use {Player X} in all the games last season” or “sure they use this stuff, yeah right” because the usage doesn’t fit perfectly within what we think a player’s CF% is telling us. I admit that sometimes this frustrates me. Part of that is probably my optimistic view of how analytics are evolving in hockey causing me to resist the thought that we are much earlier in the evolutionary process than I would like to think. The other part of it is my thought that if we want to analyze how a team is playing, we have to do much more than just look at the basic “advanced” stats.
We’ve all seen the behind the scenes videos showing coaches pouring over game film of their opponents to prepare for games. I know what I would be looking for and I have always assumed that an NHL coach would be looking for the same things and far more given his knowledge of the sport. Perhaps that is naïve of me, but how else would a coach be where he is at this level of professional hockey if he wasn’t knowledgeable? I’m not saying there aren’t people out there who may have more knowledge of different aspects of analyzing hockey because there are, but I have a hard time believing there are coaches with a complete lack of understanding of the fundamentals. When I see interesting or unique deployment of players, I think “this coach obviously uses some sort of analytics to keep track of how this is working for his team”. This isn’t to say that some coaches are not unique or more talented than others, but that some things just hint at analytics usage more than others.
Recently I was opining that the Blackhawks were struggling against another team’s forecheck and suggested that their breakouts playbook may need to be spiced up a bit to help the players adapt more efficiently in those situations. I also said that if the team had to resort to chipping the puck off the boards in behind the coverage, they needed to improve their puck retrieval skills. In one of the next few games, one of the Blackhawks announcers was talking about Chicago being one of the best teams in the league at retrieving pucks that have been dumped into the zone. In my experience tracking the team, they have not been all that good in those retrieval situations compared to other teams so I thought that perhaps I was missing something. How can someone make a statement like that without having evidence to suggest it in the first place? Someone had to have told the announcer that Chicago was a good retrieval team because the only other explanations are that he was giving his opinion based upon the eye test or he was just making it up. I know this was probably just an offhand comment from the announcer. This certainly would not be the first time that the lawyer in me has led me to over analyze something small, but little things like that are really all we have to give us clues about what teams are really doing.
This also illustrates one of the most important points of analytics in hockey: context. If someone did tell the announcer that Chicago was a good retrieval team, then they must be tracking that kind of thing right? If someone within the Chicago organization is tracking zone entries, what context are they giving it? Perhaps Chicago has gotten better at retrieving dump ins this season compared to the data they tracked last season and that was the basis for the statement. In order to say they are one of the best teams in the league though, you have to have access to the league’s numbers. The NHL does not track zone entries or make that data available in RTSS feeds. That means if you want to put any context to what the team is tracking, you have to be tracking the same thing for every other team in the league. Context is crucial to successfully using these types of analytics.
Even more recently, I mentioned something on Twitter about a defenseman being targeted on an opponent’s zone entries. I got skeptical replies that essentially said that NHL teams don’t really do that. All of the data I have seen and tracked myself suggests that they do. As the conversation went on and expanded, I saw explanations about gap control and positioning, but the times that I have really noticed targeting were on breakouts originating deep in the defensive zone. These are designed plays that are put in motion long before the forward carrying the puck into the offensive zone has a chance to read the gap. I completely understand positioning and gap control dictating the entry on a motion regroup in the neutral zone and for last second passes or chips just prior to the zone entry, but it’s harder to believe that this is the thinking on breakouts starting behind the net. Again, these are inferences I have drawn from watching the play, reviewing and analyzing data, and tracking games. Inference is not perfect of course and limited by my knowledge and experience so I could be completely wrong, but it sure doesn’t feel like it.
Much of the analytics work being done is looking for patterns in the data or in response to seeing a pattern in the play and researching it. This is all bred from not really knowing what is going on behind the scenes with each and every team. When I see patterns in the data, I tend to think it is due to something the teams are intentionally doing or something that is characteristic of the team’s systems, players and overall process (for better or for worse). Maybe that’s the wrong way to go about it, but the alternative would necessitate me believing that NHL teams leave a large part of their success or failure up to chance. I’m not sure I’m willing to do that. I’m an outsider so I can’t be sure, but it just does not make sense that a billion (?) dollar or so group of businesses would leave their future up to chance.
So what does this leave us with?
Maybe we are victims of our own bravado. Perhaps, because we are, understandably, left to guess at what teams are really doing, our bravado leads us to believe that teams are not at the same level of sophistication as us analytically speaking. Is it unreasonable to think that teams don’t have an understanding of so many of the statistical devices that can be used to evaluate the play of the team and its players? It just seems so simple that a coach would instruct his players to target the weaker player in a defensive pairing on zone entries, so it makes it hard for me to believe that they are not doing that. If it is just bravado leading us to doubt these efforts as intentional or characteristic, then we are vastly underestimating the work being done by teams.
If we are not actually blinded by our bravado, if the skeptical views are based in fact, which may well be the case, what does that mean? The only thing it could mean is that teams are vastly underestimating the power of the tools available to them. If teams are not scouting their opponents in advance, particularly those within their division, then they are vastly underestimating what analytics can do for them. If teams are not checking their internal statistics in a league wide context, then they are vastly underestimating what analytics can do for them. If teams think the best use of a coach’s time is to breakdown the last game of his team’s next opponent as a method of advanced scouting, then they are vastly underestimating what analytics can do for them.
Analytics should turn a critical eye on not just the team using them, but all of the other teams in the league as well. They should be employed to exploit an opponent’s weaknesses, plan ways to combat an opponent’s strengths and properly use the team’s personnel to achieve these ends.
I’m not sure which view on this issue is really the truth and perhaps I’ll never know. It’s difficult for me to grasp why teams would leave all of the tools they could be using to gain a competitive advantage on the table. Maybe it is a combination of both views. Maybe teams are using some analytics, but have rudimentary processes in place that limit their ability to use this data effectively. By and large, the understanding of analytics by fans works off of Corsi and Fenwick because those are the easiest to learn. Not every fan who enjoys analytics has the time to become versed on the minutia, so it’s important to remember that what I am talking about here goes far beyond just shot attempts. There is a wealth of information to collect that can be used to gain an advantage on your opponents.
To think that all of it is there for the taking and teams are leaving it untouched would mean that hockey is still in the dark ages. Why they would not want to speed up the process, free up some of their coaches’ time, scout their opponents in advance, put the metrics they are using in a league wide context and much more is beyond my comprehension. The implementation of a system for doing all of this and more would not be too difficult. What many people are capable of understanding and doing with statistics, video review and tracking games from their living rooms is kind of amazing particularly when you consider that those of us who do this are working with little to no budget. The possibilities for teams, with all of their resources, to make analytics work for them are endless. If NHL and AHL teams are not at least doing what many people are doing from their couches, they are doing themselves and their fans a disservice.
