Tuesday, December 01, 2009

A Brief Look Into the New Jersey Devils’ Official Shot Totals

The topic of data recording error with regards to Shots On Goal has made for interesting discussion lately. JLikens, in a post at Objective NHL a few months ago, noted a seemingly significant discrepancy in total shots (i.e. shots on goal taken by both teams) in the New Jersey Devils’ home games relative to their road games. Tom Awad over at Puck Prospectus recently wrote an article in which he had similar findings. Vic Ferrari, here at IOF, surmised that official scorers around the league seem to have different definitions of what constitutes a shot on goal versus a shot at goal, but that a difference in home shots at goal versus road shots at goal may be due to an actual strategic effect.

I decided to track shots for a couple of recent Devils games. Far more sampling needs to be done, but I was encouraged to post my findings of even the small sample, so that’s what I’ll do.


11/25/09, Ottawa at New Jersey

MY COUNTS:

Devils Shots: 28 Shots on Goal, 21 Attempts/Blocked, 9 Missed Shots = 58 total shots directed

Sens Shots: 29 SOG, 12 A/B, 10 MS = 51 total shots directed


NHL OFFICIAL COUNTS;

Devils Shots: 21 SOG, 11 A/B, 6 MS = 38 total shots directed

Sens Shots: 25 SOG, 5 A/B, 6 MS = 36 total shots directed


You can see that my totals are pretty far off from the official ones. It could have been a fluke, or I could have just been too generous with what dictates a “shot.” I was fairly meticulous though, liberally pausing and rewinding my dvr to double check anything questionable. (But I did watch the game at 10pm on Thanksgiving Day after drinking Bloody Marys since noon lol.)

I decided to do a road game.


11/27/09, New Jersey at Boston

MY COUNTS:

Devils shots: 38 SOG, 11 A/B, 12 MS = 61 shots directed

Bruins shots: 35 SOG, 17 A/B, 16 MS = 68 shots directed


NHL OFFICIAL COUNTS:

Devils shots: 37 SOG, 7 A/B, 8 MS = 52 shots directed

Bruins shots: 33 SOG, 17 A/B, 8 MS = 58 shots directed


For the game in Boston I was over by ~10 directed shots compared to ~20 directed shots in New Jersey. Again, it could have been a fluke, or I could have been too generous in my recording. Or, frankly, there could be wide discrepancies throughout the NHL on how shots are actually being counted, as well as how they should be counted.

I spoke to a well-known hockey blogger who recently discussed shot counting bias with a fellow who once worked for a minor league hockey team. Apparently the shot counter there was an old-timer who had been doing it for a while and was pretty “set in his ways.” And those ways were…well…debatable.

For the Devils’ next game, 11/28/09 at home against the Islanders, I decided to watch the TV broadcast while simultaneously checking out the NHL Play By Play feed on my laptop. In particular, every time I saw what I thought looked like a shot attempt, I checked to see if and how it was scored officially.

At times I wondered if I was watching the same game as the NHL dude.

Check out this clip from 5:51 remaining in the third period…

[link]

Now, um, have I gone batshit crazy, or is that basically the definition of a “shot”, both at goal, and in this case, on goal? Yet, in this game it was officially classified as neither! The Play By Play feed goes from “Faceoff, 6:15” to “Stoppage of play, 5:51” with no record of anything in between. I also double checked the box score, and Trent Hunter has a big old zero in the shots column for the third period.

Here’s another not-significant-enough-event-to-be-officially-classified-as-anything, from a few seconds later.

[link]

That one’s more debatable. But I showed it to you on purpose for a point I’ll make in a minute. For now I’ll just say this: the puck was directed at the net, had some probability greater than zero of going into the net, and would have been counted as a shot on goal had it gone in. Same shot – sometimes counted as a shot, sometimes counted as nothing.

I could comb through the whole game for examples to post, some more glaring than others, but you get the point.

A few days later, I did the same thing of watching the TV broadcast and NHL Play By Play feed simultaneously for a Vancouver Canucks home game against San Jose. In this case, I was actually pleasantly surprised that almost every event I perceived as a shot directed at net was in fact recorded as some sort of shot attempt (i.e. SOG, A/B, MS).

I’ll wrap this up now with a few editorial comments.

1) I found it quite valuable to partake in the exercise of recording shots at net. I highly recommend it. You get a real sense of how the game works, and you realize just how important it is for a team to be able to even attempt shots.

2) I think shot attempts are maybe the most valuable metric we currently have. There is so much parity in the quality of shooters, goaltenders, and team defenses throughout the league that just attempting a shot and giving your team a chance to have the puck go in is huge.

3) On the latter note – chance. It’s a key word in understanding the importance of shot attempts. Randomness (i.e. luck, i.e. “chance”) is absolutely rampant with regards to goal scoring. We’re talking about 2 or 3 goals being scored on 50 or 60 shot attempts. What do you think constitutes the difference (or most of the difference) between them? Imo it ain’t Ovechkin getting an uncontested breakaway versus Hextall letting in a softy from center ice. Most shot attempts that result in goals, compared to shot attempts that don’t, do so because of the sheer physical randomness subjected upon a small disc of vulcanized rubber being hurled through the atmosphere at a high velocity with several obstacles in its way.

4) Extreme recording error for data like “shot attempts” is pretty pathetic for an enterprise like the National Hockey League. Many of these recorded statistics are used to hand out awards, dole out salaries, and rewrite history books. For fuck’s sake, get a panel of college interns in every city to record data individually, just so you have a few sources to compare/average. The kids would probably work for pizza and beer. :)

5) Consistent and reliable shot attempt data is the most fundamental starting point for analysis imo. It’s more fundamental than scoring chances (and less prone to human bias and inconsistency); though perhaps less fundamental than puck location. While a certain amount of speculation about a player’s intent is unavoidable and necessary, I think it’s better to err on the side of generosity when it comes to recording an event as a shot attempt. I.e., If it’s not an obvious pass, and it’s directed at net, and it has a reasonable probability > 0 of being a goal, it should be recorded as a shot attempt of some kind.

6) Given consistent data, I’m inclined to think all individual and on-ice Sh% and Sv% equations should have shot attempts as the denominator. Before you scoff at the notion of a goalie getting credit for a blocked shot, realize that it’s mostly a matter of semantics. Goalies get penalized all the time for letting in goals on events that often wouldn’t have been recorded as shots in the first place had they not gone in. Having a denominator with significantly less bias is more important (and more insightful) than any other factor. If shooters, goalies, or defenses have the ability to control the number of goals they score or let up in lieu of quantity of shot attempts, it will show up, so long as we have consistent data.

28 Comments:

Blogger Hawerchuk said...

I hope this doesn't seem like blindly trusting regression, but:

Goal+SOG: R^2 = 0.29 with Win %
Goal+SOG+Miss: R^2 = 0.35
Goal+SOG+Miss+Block: R^2 = 0.29

We knew blocks were a mess (home/road bias, rink bias) so we expected that. But it would seem that the NHL does a better job of tracking shots that go in the general direction of the net than it does at tracking shots that were stopped and would have gone in otherwise.

Incidentally, how are crossbars and posts not shots on goal?

12/01/2009 4:53 pm  
Blogger Wexler said...

"Incidentally, how are crossbars and posts not shots on goal?"

I think the general conception of a shot on goal is that it had some chance of becoming a goal, and only a save by the goaltender prevents it from doing so. The idea then- perhaps flawed- is that this gives you some useful measurements (a goalie's ability to prevent goals, a shooter's ability to convert shots to goals, a team's ability to create chances, etc).

If a player hits iron and the bounces out, the thinking is that the shooter gave his team less of a chance to score a goal, the goalie did less to prevent a goal, etc.

12/01/2009 5:28 pm  
Blogger Wexler said...

Really enjoyable article.

12/01/2009 5:32 pm  
Blogger Sunny Mehta said...

Hawerchuk,

Hm, interesting. Maybe until we get more accurate data we should use SOG + MS.

When I was tracking shots, a lot of times it was legitimately hard to tell whether a shot was missed or blocked (i.e. deflected wide because it hit something/someone). I figured the important thing would be to record it as some kind of shot attempt. Apparently the Devils' shot recorder (amongst others) doesn't feel the same way though lol.

12/01/2009 5:32 pm  
Blogger Sunny Mehta said...

Wexler,

Thanks. fym.

12/01/2009 5:33 pm  
Blogger Doogie2K said...

My conception has always been that a goalpost is a missed shot. It's as close to a SOG that one can get, but in the end, it's the same as a shot off the back boards, with perhaps slightly more chance of bouncing off the goalie or a defender and in.

12/01/2009 5:46 pm  
Blogger Matt said...

Goal+SOG: R^2 = 0.29 with Win %
Goal+SOG+Miss: R^2 = 0.35
Goal+SOG+Miss+Block: R^2 = 0.29


I WAS RIIIIGGGGHTTT!!!!! TAKE THAT, RIVERSQ!!!!!

OK, I'm good now.

12/01/2009 9:40 pm  
Blogger Ron said...

A post/crossbar is not a shot on goal. Ask any goalie, they'll tell you.

When I talked with the ice officials as a game here in Minnesota (where they've gotten grief over this) they said that the NHL gave them explicit instructions on what to count. He told me that if it was a soft shot or dump they would not count it.

Me, I thought was if the puck would have gone in the net without the goaltender's effort that makes a save. But I was told I was wrong... <grin>

12/01/2009 10:13 pm  
Blogger Sunny Mehta said...

Matt,

The thing is, I'd guess the reason that's true is mostly because of recording bias. If we had good, consistent data, I would bet against blocked shots being a persistent and significant "skill", and I'd bet that total shot attempts would have the highest correlation to goal scoring.

12/02/2009 12:59 am  
Blogger Sunny Mehta said...

Guys,

re: whether a goalpost should be counted as a shot on goal

If someone vehemently wanted to take the stance of, "Look, it missed the net, the goalie didn't have to make a save - it should not get counted as a shot on goal.", I think I'd probably respond with 1) "You're right." 2) "Who cares?"

If it's not already obvious, I think it really doesn't matter what you call it.

20 years ago people probably argued whether a particular baseball play should be ruled a hit or an error. And I'm sure someone vehemently said, "Look, it wasn't the pitcher's fault, it was an error - the fielder should have had it, and the pitcher shouldn't get (dis)credited with a 'hit against'." And maybe they're right from a certain semantical standpoint.

But then people realized that it doesn't make a fkn difference what you call it because the bottom line is that pitchers by and large don't have any control of whether balls in play go for hits or not, so if you were evaluating the pitcher you'd be much better off knowing that the ball was "a line drive travelling at x mph", etc.

12/02/2009 1:35 am  
Blogger Sunny Mehta said...

By the same token...

Ron,

He told me that if it was a soft shot or dump they would not count it.



See I think it's a more egregious mistake to show absolutely no record of an event that was even a possible/questionable "shot" than to mis-label something as a "missed shot" or "blocked shot" that maybe should have been a "shot on goal."

12/02/2009 1:56 am  
Blogger Vic Ferrari said...

Just terrific stuff, Sunny.

If you look at the season so far, and the spread of team shooting percentages when the score is tied (or any time when playing to the score) ... I suspect that it's a tighter grouping that chance would expect, and by a wide margin. And using shots-AT-net (corsi) ... probably spot on.

I hoped that this was either an extraordinary coincidence or there was inconsistent shot counting in the NHL, from venue to venue. The alternative (that the universe is folding in upon itself) was unpleasant. :)

The PK and playing-to-the-score at EV is when the relationship between scoring chances and corsi starts to shift, esp with regards to the blocked shots. And in the case of the former, it depends a lot on the team. Or at least it seems to, it's still early days with the scoring chance project.

12/02/2009 5:39 am  
Blogger Sunny Mehta said...

Thanks, Vic.

re: your last paragraph

Good points. I was actually going to mention the fact that, the most likely situation where we'd see the relationship affected between shot attempts (particularly blocks) and scoring chances is on the PK.

However, I will also say that, when I was recording shots, I tried to pay real close attention to when there was a PP (to look for what we're talking about), and, I don't know man, I guess I'm still pretty dubious that there's a huge difference between 5-on-5 and 5-on-4 wrt the value of shot attempts. I certainly could be wrong, and perhaps you've actually looked at the distributions, but I think if I were betting on the outcomes of PP's, I'd still look first and foremost at shot attempts.

12/02/2009 10:29 am  
Blogger nightfly said...

I'm amazed the league doesn't count a soft shot as a shot. I'm even inclined to count the dump-ins as shots if they're on net. I mean, if a dump-in goes into the net (like Rob Davison's a few years ago against Vesa Toskala), does it suddenly not count as a goal?

I have a bit of a bias on this topic since I am a goaltender and have been an official statistician and scorekeeper in local roller and dek hockey leagues. Of course, often working alone I had no chance at keeping time played or +/- or such - but I could record the basic stuff and I made sure I was as rigorous and consistent as I could be on that. If the keeper had to stop it, it was considered a shot.

12/02/2009 11:06 am  
Blogger Vic Ferrari said...

Yeah, I always thought they recorded everything on net as a shot. In any case clearly there is some inconsistency.

Lat year, in the playoff series between NYR and WSH (about game 4, I think) the Caps in general, and Ovechkin in particular, had thrown a lot of rubber at Lundqvist. The TV crew assembled a clip of all of Ovechkin's shots on goal to that point. I assumed that they were 'official' shots, but I may be wrong.

The first thing that struck me was how, for a guy who takes a tonne of shots, OV didn't waste many. All but one was from inside the dots, by angle. The popular narrative was that they were 'keeping him to the outside'. Damn, I dunno. I think that in most parallel universes he had a bunch of goals in that series to that point.

The lone exception, though, was one where he basically turned and swept the puck out of trouble and towards the NYR end of the rink. It happened to be on goal.

Didn't take long to find, this must be it, from game 1 in DC.
http://www.nhl.com/scores/htmlreports/20082009/PL030121.HTM
Item 138. A 107 ft shot. I don't even think he intended to put it on net.

Plus, it's not that uncommon to see shots on goal listed at a distance of 80 or 90 feet (from the backboards) at NHL.com. Some the idea of a 'soft shot' isn't the same for everybody.

I wonder if just weeding out all shots/shot-attempts from beyond the blueline would even out some of the recording bias?

12/02/2009 12:11 pm  
Blogger Vic Ferrari said...

Hrmm.

Same game, item 149 shows Fleshmann's goal being a tip-in from 136 feet.

12/02/2009 12:12 pm  
Blogger Vic Ferrari said...

You may be right, Sunny. More likely, it's probably close enough that using shots-AT-net is better because it reduces some of the scorer bias.

Last I checked for the Oilers, a few games ago ... PK shots-ON-net against meshed better with the on-ice scoring chance numbers for the PKers. Eerily so for the forwards. Really small sample, though. Personally, I'm not confident one way or the other at this point.

I agree with all of your verbage above, btw. Well said. Watch a few games through this lens and then listen to the narrative from the HNIC commentators ... they're all going to seem like delusional poets.

At even strength, territorial advantage is almost everything so long as the score is close and/or the end of the game is a long ways away. It's that and odd man rushes, and most of those fall from the sky.

Even when score effects are in play ... the scoring chances swing in the direction of the shots-at-net, just not quite as dramatically.

12/02/2009 12:23 pm  
Blogger Olivier said...

Vic: Interesting point about Ovechkin.

My experience of Ovechkin this year and last year is quite the opposit: he takes a lot of his shots from between the dots, but high, often just outside the faceoffs zone. Dude has a heavy, precise, quickly released shot and, like Guerrero in his heydays, uses those freakish abilities to turn bad percentage shots into honest percentage ones. So a lot of those 500+ shots are long bombs of the kind he's the only guy (with maybe Kovalchuk & Heatley) who's actually able to convert them in goals.

So there is a bunch of shots he takes on a given night that I'd say, at face value, nope that's not a chance, too far away. And the goalie stops it cleanly and that's it. But you can see he was trying to unload it from the defensmen shoelaces, or to catch the goaltender flat footed, and even if he doesn't , some of his linemates are down low looking for rebounds. I don't record them as SC, except if he actually convert them, and it's a hill I'll die on ;).

But then again, maybe I caught him in bad days... Also, he stills ends up getting 3-5 prime grade chances a game. Add that to those long bombs, and it's quite obvious how he ends up potting 60+ in a given year.

12/02/2009 3:14 pm  
Blogger Hawerchuk said...

Vic,

Here's what the different sources have to say for that Ovechkin shot:

NHL.com - Ovechkin 107 ft
ESPN.com PBP - Ovechkin 82 ft
ESPN.com gamecast - Ovechkin 108 ft
ESPN.com x-y - Ovechkin (102 from net, -32)
CBS.com - the scorer missed it

Fleischmann's shot is at (35,4) on ESPN.

12/02/2009 7:51 pm  
Blogger Triumph said...

great stuff, sunny. the devils' scorer is probably the same guy who's been doing it since 82, so maybe a new guy comes in soon.

i guess it's like the scoring chances project over at mc79hockey - you want something done in the world of nhl stats, you gotta do it yourself.

12/03/2009 9:27 am  
Blogger nightfly said...

Vic, Olivier - excellent point on Ovechkin. For me, it's not necessarily that he can take shots and score consistently from poor areas of the ice because of his skill level (though he obviously can); it's that he can shoot *when* few others can, as long as he's in a good spot to shoot from, and make it a tough shot to stop. IOW he gets off stuff in traffic, or when his body position and stick position would preclude a shot attempt from almost anyone else.

The drop-dead-sick sliding-on-his-back goal against Phoenix is an example that I don't discount because he was "lucky" to actually hit the net. Of course he was, but who else would have even tried to sweep the thing on net? He had the ability to try. It was lucky that it went in, maybe, but he would never had gotten lucky if he hadn't had the incredible stick and body control - and more importantly, the willpower and trust in his skill - to try.

But if you please, there's a better, less "lucky" example - I remember last season, one game against the Islanders, that he won in overtime with a wrister (maybe 35 feet) from dead-on that tucked under the crossbar. From the way he was skating and the puck position, there was no hint that a credible attempt could be made - suddenly the puck is rocketing on net as if linear-inducted. MacDonald couldn't stop it, but Dryden and Hasek and the rest would probably have been beaten as well. It was simply dumbfounding how he even got it off, much less got into it as if it were a one-timer. It was preposterous, like a video game glitch where the shot heads on net before the shot animation has rendered.

Gretzky was like this, only with him it was his freakish mental edge and precision; the Greg Maddux to Ovie's Pedro Martinez, if you will. He reada goalies and defenders so well that his shots and passes always seemed to catch everyone leaning or shifting or about to expose a hole, through which he would sneak the puck.

12/03/2009 12:56 pm  
Blogger Vic Ferrari said...

On Ovechkin, the thing I can't get over is how often he gets his shots off from inside the faceoff dots, by angle.

Oiler fans grimace every time Moreau takes a shot from the "Ethan Zone", about three feet outside the dot on the left side. He'll probably score once or twice from there is year, when a goalie loses his angle or just whiffs on one. But you just can't see any net from there, it's all goalie. He'll ring another 30 or 40 shots wide of the far post and back out of the zone.

Back to Ovechkin, if he's pushed too far wide he rarely takes the shot, he carries it behind the net and battles, attempts to cut to the middle or passes. NHL defenders just can't seem to keep him to the outside when he's coming down the wing though. I suppose if you try to hard and play him too tight ... you could look like a fool if he cuts to the slot and beats you one on one. No NHL defenceman wants to be a YouTube assclown.

He reminds me of Shanahan in his prime, he could score from further out as well. Really helps a line I think, especially a PP unit. With the new sticks as well ... most of his shot attempts, while not glorious chances, if he makes the perfect shot no goalie will stop it unless they guess.

I think he'll break 92 goals one of these years. He'll need the cookie to crumble his way, good health, and some shoddy goaltending in the SE division would help. I suspect he'll do it, though.

12/03/2009 2:29 pm  
Blogger Sunny Mehta said...

Guys,

Warning: the following data is likely rife with randomness due to small sample size and possible selection bias issues.

Nevertheless I thought it was interesting...


Shots in 1st half of the season compared to Shots in 2nd half of the season (measured in Pearson correlation for 08/09 season, only at even strength w/score tied):

Shots on goal +/- at home
r= .60 (again, that's a correlation of first half to second half)

Shots on goal +/- on the road
r= .21


Missed Shots included:

+/- at home
r= .63

+/- on the road
r= .35


Missed Shots and Blocked Shots included:

+/- at home
r= .65

+/- on the road
r= .59

12/03/2009 4:47 pm  
Blogger Vic Ferrari said...

Damn, brother, you are on fire. And as you know, your warning mellows the correlations.

Colour me convinced about your opinion in the original post. We could use "close scoreand at EV", by any sensible measure, instead of "EV tied". Then we could shuffle the home games and road games randomly, pick the first twenty of each, and repeat you exercise a multitude of times. And we've explained the Jlikens anamoly. Damn.

The real beauty of this, is that the value has not been slightly misunderstood. Google if you don't believe me, this article is popular, and the value is almost universally vastly misunderstood. It's the perfect storm. You're like slipper with math, Sunny.

12/03/2009 6:29 pm  
Blogger Sunny Mehta said...

Thanks, Vic! It means a ton coming from you, but honestly I'm just mostly running with the torch you lit.

Great idea with the "close score" random selection thing. Unfortunately I don't yet have the programming chops that you and Gabe do, so I had to use the data I already had. It would be great to look into though.

I'm also now kind of curious what individual ES Sh% and Sv% numbers look like if we use, say, all shot attempts post-lockout as the denominator, and we view the percentages by home and road separately.

12/03/2009 7:03 pm  
Blogger Sunny Mehta said...

Also, dudes,

I just chopped up the 08-09 ES Tied data and had a look at a bunch of interesting splits.

Usual caveats apply but...

The New Jersey Devils posted a 6.5% Shot Attempts Shooting Percentage (i.e. Goals divided by Total Shot Attempts) at home which was good for... you guessed it: First place - ahead of the Boston Bruins, a full shooting percentage point higher than the Pens and Wings (5.3%), almost two full shooting percentage points higher than the Chicago Blackhawks (4.6%), and more than 50 percent better than the average team (4.3%) shot at home!

And as you also probably guessed, these same Devils on the road... 3.8%, good for 21st place.

Either it was just total randomness, or, the Devils have an uncanny ability in their home rink to summon the inner auras of the 1984 Edmonton Oilers (perhaps through some mind-altering chemical in the sludgey Newark water?), or maybe, just maybe, the data recording in Jersey is totally fucked.

12/04/2009 4:14 pm  
Blogger Sunny Mehta said...

here are what the distributions look like...

Home Sh% histogram

------------

Road Sh% histogram

12/04/2009 4:47 pm  
Blogger dawgbone said...

Ever taken a look at things like the giveaway/takeaway stat that the NHL tracks?

The Oilers had almost as many giveaways at home (696) as the Coyotes had home and road giveaways and takeaways combined (788).

I mean, you could take that at face value, but when you compare their road giveaway/takeaway numbers:

Oilers 342 give / 220 take

Yotes 296 give / 227 take

There's not a lot of difference there... it's all in the difference between the Oilers counter and the Yotes counter.

In terms of what defines a "shot on goal"... I thought that is pretty obvious. Any shot that would count as a goal if the goaltender was not there. It's odd that you don't count a soft dump in from centre as a shot if the goaltender stops it... but if it bounces in it is. Not sure I buy the explaination behind that.

12/09/2009 9:58 am  

Post a Comment

<< Home