The topic of data recording error with regards to Shots On Goal has made for interesting discussion lately. JLikens, in a
post at Objective NHL a few months ago, noted a seemingly significant discrepancy in total shots (i.e. shots on goal taken by both teams) in the New Jersey Devils’ home games relative to their road games. Tom Awad over at Puck Prospectus recently wrote an
article in which he had similar findings. Vic Ferrari, here at IOF,
surmised that official scorers around the league seem to have different definitions of what constitutes a shot
on goal versus a shot
at goal, but that a difference in home shots at goal versus road shots at goal may be due to an actual strategic
effect.
I decided to track shots for a couple of recent Devils games. Far more sampling needs to be done, but I was encouraged to post my findings of even the small sample, so that’s what I’ll do.
11/25/09, Ottawa at New Jersey
MY COUNTS:
Devils Shots: 28 Shots on Goal, 21 Attempts/Blocked, 9 Missed Shots = 58 total shots directed
Sens Shots: 29 SOG, 12 A/B, 10 MS = 51 total shots directed
NHL OFFICIAL COUNTS;
Devils Shots: 21 SOG, 11 A/B, 6 MS = 38 total shots directed
Sens Shots: 25 SOG, 5 A/B, 6 MS = 36 total shots directed
You can see that my totals are pretty far off from the official ones. It could have been a fluke, or I could have just been too generous with what dictates a “shot.” I was fairly meticulous though, liberally pausing and rewinding my dvr to double check anything questionable. (But I did watch the game at 10pm on Thanksgiving Day after drinking Bloody Marys since noon lol.)
I decided to do a road game.
11/27/09, New Jersey at Boston
MY COUNTS:
Devils shots: 38 SOG, 11 A/B, 12 MS = 61 shots directed
Bruins shots: 35 SOG, 17 A/B, 16 MS = 68 shots directed
NHL OFFICIAL COUNTS:
Devils shots: 37 SOG, 7 A/B, 8 MS = 52 shots directed
Bruins shots: 33 SOG, 17 A/B, 8 MS = 58 shots directed
For the game in Boston I was over by ~10 directed shots compared to ~20 directed shots in New Jersey. Again, it could have been a fluke, or I could have been too generous in my recording. Or, frankly, there could be wide discrepancies throughout the NHL on how shots are actually being counted, as well as how they should be counted.
I spoke to a well-known hockey blogger who recently discussed shot counting bias with a fellow who once worked for a minor league hockey team. Apparently the shot counter there was an old-timer who had been doing it for a while and was pretty “set in his ways.” And those ways were…well…debatable.
For the Devils’ next game, 11/28/09 at home against the Islanders, I decided to watch the TV broadcast while simultaneously checking out the NHL Play By Play feed on my laptop. In particular, every time I saw what I thought looked like a shot attempt, I checked to see if and how it was scored officially.
At times I wondered if I was watching the same game as the NHL dude.
Check out this clip from 5:51 remaining in the third period…
[link]Now, um, have I gone batshit crazy, or is that basically the definition of a “shot”, both
at goal, and in this case,
on goal? Yet, in this game it was officially classified as neither! The Play By Play feed goes from “Faceoff, 6:15” to “Stoppage of play, 5:51” with no record of anything in between. I also double checked the box score, and Trent Hunter has a big old zero in the shots column for the third period.
Here’s another not-significant-enough-event-to-be-officially-classified-as-anything, from a few seconds later.
[link]That one’s more debatable. But I showed it to you on purpose for a point I’ll make in a minute. For now I’ll just say this: the puck was directed at the net, had some probability greater than zero of going into the net, and would have been counted as a shot on goal had it gone in. Same shot – sometimes counted as a shot, sometimes counted as nothing.
I could comb through the whole game for examples to post, some more glaring than others, but you get the point.
A few days later, I did the same thing of watching the TV broadcast and NHL Play By Play feed simultaneously for a Vancouver Canucks home game against San Jose. In this case, I was actually pleasantly surprised that almost every event I perceived as a shot directed at net was in fact recorded as some sort of shot attempt (i.e. SOG, A/B, MS).
I’ll wrap this up now with a few editorial comments.
1) I found it quite valuable to partake in the exercise of recording shots at net. I highly recommend it. You get a real sense of how the game works, and you realize just how important it is for a team to be able to even attempt shots.
2) I think shot attempts are maybe the most valuable metric we currently have. There is so much parity in the quality of shooters, goaltenders, and team defenses throughout the league that just attempting a shot and giving your team a chance to have the puck go in is huge.
3) On the latter note – chance. It’s a key word in understanding the importance of shot attempts. Randomness (i.e. luck, i.e. “chance”) is absolutely rampant with regards to goal scoring. We’re talking about 2 or 3 goals being scored on 50 or 60 shot attempts. What do you think constitutes the difference (or most of the difference) between them? Imo it ain’t Ovechkin getting an uncontested breakaway versus Hextall letting in a softy from center ice. Most shot attempts that result in goals, compared to shot attempts that don’t, do so because of the sheer physical randomness subjected upon a small disc of vulcanized rubber being hurled through the atmosphere at a high velocity with several obstacles in its way.
4) Extreme recording error for data like “shot attempts” is pretty pathetic for an enterprise like the National Hockey League. Many of these recorded statistics are used to hand out awards, dole out salaries, and rewrite history books. For fuck’s sake, get a panel of college interns in every city to record data individually, just so you have a few sources to compare/average. The kids would probably work for pizza and beer. :)
5) Consistent and reliable shot attempt data is the most fundamental starting point for analysis imo. It’s more fundamental than scoring chances (and less prone to human bias and inconsistency); though perhaps less fundamental than puck location. While a certain amount of speculation about a player’s intent is unavoidable and necessary, I think it’s better to err on the side of generosity when it comes to recording an event as a shot attempt. I.e., If it’s not an obvious pass, and it’s directed at net, and it has a reasonable probability > 0 of being a goal, it should be recorded as a shot attempt of some kind.
6) Given consistent data, I’m inclined to think all individual and on-ice Sh% and Sv% equations should have shot attempts as the denominator. Before you scoff at the notion of a goalie getting credit for a blocked shot, realize that it’s mostly a matter of semantics. Goalies get penalized all the time for letting in goals on events that often wouldn’t have been recorded as shots in the first place had they not gone in. Having a denominator with significantly less bias is more important (and more insightful) than any other factor. If shooters, goalies, or defenses have the ability to control the number of goals they score or let up in lieu of quantity of shot attempts, it will show up, so long as we have consistent data.
