Friday, September 04, 2009

Positional Shot Quality

[Note: I had my curiosity piqued the other day, and I ended up writing this article. Since I don't have a blog, I thought I'd post it here. Thanks to Vic and crew. Hopefully I'll be able to contribute more in the future. - Sunny Mehta]


Shot quality at even strength has been a point of debate around these parts lately. Some folks have built shot quality models. Others have poked holes into those models’ reliability and significance. Some people have linked specific team defensive tactics with suppressing opposing shot quality. Others have questioned the evidence for its existence on a league-wide scale.

Rather than focus on what we don’t know, I’m going to focus on what we do know. Let’s start with something most everyone agrees on: A good amount of evidence exists that offensive players can, in fact, display the ability to take higher quality shots than other offensive players. I.e., shooting percentages for certain players are persistently higher than can reasonably be explained by chance (see: Kovalchuk, Ilya).

Further, we observe a large difference in shooting percentages between forwards and defensemen. It makes intuitive sense that shots taken by forwards are more likely to go in than shots taken by defensemen. The numbers back that up. We don’t need any fancy models: forwards shot 9.6 percent at even strength last season, and defensemen shot 4.1 percent. And we see similar splits every year. [Note: the data in this article is taken from Behind The Net, which unfortunately includes empty net goals, as well as certain inconsistencies with regards to players traded mid-season. My numbers would be slightly more accurate with those two issues resolved, but the conclusions would be the same.]

So, before we look at a team’s overall shooting percentage and attempt to debate how much of it was luck versus skill (or randomness versus shot quality), we can at least first look at the composition of their shots with regards to forwards and defensemen. After all, that is one area of “shot quality” we know exists and can therefore confidently adjust for.

The following chart shows the results of my inquiry:

(2008-09 Even Strength)


Let me explain the last column on the right. It is each team’s Expected Shooting Percentage based on the ratio of shots taken by forwards relative to defensemen. (Each team’s forwards and defensemen were assigned league average shooting percentage, 9.6 and 4.1, respectively.)

A few observations:

1) Some teams, even after adjusting for forward/defensemen shot ratio, over- and under-performed their expectation by a significant amount. That could be due to either randomness or the fact that their individual forwards and defensemen are offensively better or worse than average.

2) A more precise way to model this would be to weight every team’s shots by the historical shooting percentage of the player who took each shot, and then see how far teams over- and under-performed their expectation. That’d be a hell of a task.

3) I’d be curious how persistent the ability to have forwards take more shots relative to defensemen is at the team level. If someone wants to run these numbers for previous seasons for comparison sake, or run some sort of split-reliability test, that’d be great.

4) If every team took the same number of shots, the scoring impact due to F/D shot ratio from the 30th team to the 1st team is 13 goals. Doesn’t sound like a lot, but on the other hand, I guess ~2 wins isn’t scoff-worthy.

5) Detroit led the league in total shots. No surprise there. But they also led the league in shots by forwards. That may surprise people. The Wings are often, and incorrectly, accused of padding their shot totals by taking a ton of shots from the point.

6) Pittsburgh was second in shots by forwards. Considering they have a few forwards with some pretty serious talent, don’t be surprised if the Pens are near the top of the shooting percentage list again next season. (And considering Pittsburgh increased their overall shot totals and decreased their shots against once Bylsma took over, don’t be surprised if they are near the top of the standings either.) With a whopping 181 goals, Pittsburgh’s forwards led the league by a fairly wide margin. (They had more than double the goals of the Islanders’ forwards. LOL.)

7) The fact that defensemen have such persistently lower shooting percentages than forwards certainly begs the interesting inquisition into the effect of shot distance on shot quality. Are defensemen scoring at a lower rate than forwards primarily because they are shooting from farther out or because they have less offensive talent?


Viewing teams’ shots and shooting percentages separated out by forwards and defensemen has value. The two positions are inherently different, and observing them separately adds nuance and gives a more descriptive view of what is happening on the ice. And as far as measures of shot quality go, this one is much less prone to recording bias than others.

Viewing a breakdown like this for teams' shots against would probably also be insightful.

17 Comments:

Blogger JLikens said...

Nice work.

I ran the numbers for 0708 using the data from behindthenet. While some of the data appears to be missing -- there were some 4300 EV goals scored in 2007-08, but only some 4000 according to the data contained on the shot distance page -- F shots / D shots at the team level appears to be somewhat reliable.

The correlation between team F/D ratio in 0708 and team F/D ration in 08/09 is about 0.45.

Perhaps counter-intuitively, F / D ratio -- and therefore expected shooting percentage -- appears to be uncorrelated with actual shooting percentage (r=0.12 for 0809, r= - 0.07 for 0708).

9/04/2009 11:22 pm  
Blogger JLikens said...

Oh, and just for the purposes of clarity, all the numbers I used the above post are for 5-on-5 only.

9/04/2009 11:24 pm  
Blogger Sunny Mehta said...

J,

re: missing data

the BTN "Individual Shots" page defaults to 10 min/60 TOI. manually change the number in the URL to "0" and you'll get all the shots.

9/04/2009 11:54 pm  
Blogger JLikens said...

Ah, my mistake then.

I'll re-run the numbers with the entire data set.

9/06/2009 11:19 pm  
Blogger Kent W. said...

Great stuff, Sunny. I was wondering when you'd start writing for hockey (aside from in comment sections, of course).

9/07/2009 3:43 pm  
Blogger Sunny Mehta said...

Thanks, Kent!

JLikens, cool, post 'em when ya got 'em.

9/07/2009 4:02 pm  
Blogger JLikens said...

Finally got around to correcting the data.

Here's what I get for the relevant correlations.

r [F/D ratio 0708 & F/D ratio 0809] = 0.51

r [F/D ratio 0708 & EV shooting percentage 0708] = 0.06

r [F/D ratio 0809 & EV shooting percentage 0809] = 0.12

So while the teams that had a higher proportion of forwards taking shots in 0708 also tended to have the same profile in the following year, in neither year was there any relationship between that and actual shooting percentage. (Although in both cases the correlations were in the anticipated direction).

9/10/2009 11:19 pm  
Blogger Sunny Mehta said...

Why do you think that is? Simply because, of all things, it's randomness that still has the biggest say with regards to shooting percentage?

I'd love to see this data for Shots Against. If a team can persistently force a certain F/D ratio from their opponent, it seems like that might have an impact on their opponent's S%.

But then again, if in the end randomness has such an overwhelming effect on all the "quality" stuff, it seems that teams' best bet is to focus on "quantity".

9/11/2009 1:07 am  
Blogger JLikens said...

"Why do you think that is? Simply because, of all things, it's randomness that still has the biggest say with regards to shooting percentage?"

I suspect that's the main reason.

Also, the team-to-team spread in expected shooting percentage (as calculated from F/D ratio) is quite limited -- well under 1%.

To me, that necessarily precludes F/D ratio from accounting for any more than a small fraction of the overall variance.

9/11/2009 1:24 pm  
Blogger The Falconer said...

"Are defensemen scoring at a lower rate than forwards primarily because they are shooting from farther out or because they have less offensive talent?"

Observation: Ray Bourque certainly seemed pretty skilled in the All Star shot accuracy exercise.

I know that one data point does not a pattern make, but how could distance not matter?

I play hockey, it is much easier to score from the slot than it is from the blueline in my league. Hasn't that always been true? Are we trying too hard to re-invent the wheel perhaps?

9/12/2009 9:09 pm  
Blogger Hawerchuk said...

Sunny,

Cool idea. I got the following #s for 5v5 initial shots 2001-09:

F - 8.58% (262701 shots)
D - 4.01% (89499 shots)

If I take the forward shooting percentages in each grid location and multiple it by the number of shot opportunities, I get 4.59%. So forwards seem to have 14% better shot quality than defensemen...(It's possible I have a few players misclassified.)

9/13/2009 1:03 am  
Blogger Sunny Mehta said...

Falconer,

Definitely my intuition says the same thing as you. I think most people's probably would.

I also think, with a lot of the shot quality stuff, the question of "does it exist?" is not as important as the question of "how much does it matter?" Know what I mean?

Also, with the insane amount of randomness that soaks NHL results/stats, particularly over the course of one single season, erring on the side of caution about what we "know" to be true seems reasonable to me. (But yes, getting carried away with that notion can start to be unreasonable. At a certain point we all "accept" certain assumptions about truth.)

If we accept the notion that shots from farther out are less likely to go in than shots from closer in (which is entirely reasonable), where does that leave us? Do we have the ability to use concrete information to build a model with good predictive value?

9/13/2009 9:09 am  
Blogger Sunny Mehta said...

Hawerchuk,

Thanks! Couple questions...

1) What do you mean by "initial shots"? Do you just mean all shots at 5v5, or did you do something like strip rebounds out based on the PbP data?

2) I don't completely understand your last paragraph. Can you maybe explain in a different way exactly what you did?

9/13/2009 9:14 am  
Blogger Hawerchuk said...

1) Initial shots has rebounds stripped out.

2) We have:

P_shooting_def(x,y) - probability of a goal being scored by a defenseman at (x,y)

Loc_shooting_def(x,y) - count of shots+goals by defensemen at (x,y)

P_shooting_fwd(x,y) - probability of a goal being scored by a forward at (x,y)

sum(Goals_def) = sum(P_shooting_def*Loc_shooting_def) = 3588 (or 4.01%)

sum(Goals_fwd_def) = sum(P_shooting_fwd*Loc_shooting_def) = 4104 (or 4.59%)

In other words, if you take the shooting percentage by location for forwards and assume that they got the same set of shots as defensemen did, they would score 14% more often. (All other things being equal...)

Does that make sense? I should write a blog post with the charts.

9/13/2009 11:21 am  
Blogger Sunny Mehta said...

Got it, thanks Gabe.

9/13/2009 8:30 pm  
Blogger Sunny Mehta said...

all, fyi gabe explains his comment in more detail (and with those great charts) here:

http://www.puckprospectus.com/article.php?articleid=273


and Tom Tango also comments about it on his blog here:

http://www.insidethebook.com/ee/index.php/site/article/what_if_forwards_were_limited_to_their_shot_quality_like_defensemen_are/

9/18/2009 1:43 pm  
Blogger oilswell said...

Nice. Tom's explanation of the 14% doesn't seem too plausible except for shot selection, but I agree with his conclusions.

9/20/2009 11:56 pm  

Post a Comment

<< Home