Tuesday, May 05, 2009

Scoring Chances: Part III of Many. Scoring Against Minnesota.

For years the Wild goalies have been stopping pucks at a better rate than average. A lot of the credit for this has been given to Jacques Lemaire and his systems play. I suspect that there is something to that, just not as much as most people. Sure, the Wild PK, and especially the fact that they take very few penalties, helps the goalies overall save percentage numbers. They rarely have to kill a 5v3 penalty either.

But at even strength hockey, I think that the goalies deserve mor of the props. Fernandez and Roloson have continued to be very good at stopping pucks at evens since leaving the Twin Cities, and I suspect that Backstrom would also.

On the season as a whole, the Oilers averaged 35.6 scoring chances for every 100 pucks that they shot with malice in the general direction of the other team's goalie. They averaged 65.7 scoring chances for every shot that was on goal (though a great many scoring chances were not recorded as shots on goal).

So if Minnesota really are giving up fewer scoring chances, relative to the number of shots they allow, we should expect the Oilers to have done far worse at generating scoring chances at EV when they played The Wild. Here is how it shook out:

vs League vs Wild
35.6 35.9
65.7 64.9

Pretty damn similar, no? Yeah, I know it's just six games and it might be coincidence. Still, methinks Backstrom deserves more props than he gets from most hockey fans around the league. And it is unimaginable that the Wild don't track scoring chances themselves, so I expect they knew what they had in Backstrom when they signed him to the long term deal this winter.

The Wild are an interesting team. They have been living off of good goaltending and excellent special teams for a while now, though. If they can land the Sedins this summer, that should send a ripple down the lineup, and they should be very strong in 09/10. They are a lot more fun to watch than they used to be, I think the reputation is larger than the facts in this case. The series against COL last spring was some of the best hockey of the playoffs to my mind.


Blogger JLikens said...

I've often grappled with this very question myself, having come to the conclusion that the truth is in the middle -- while the Wild are in fact a good defensive team that tends to limit quality shots against, it also seems that they've benefited from having above average netminding as well (Backstrom, Fernandez, Roloson, and so forth).

I think that the evidence tends to implicate the system as the main factor, though.

For example, Backstrom had an EV save percentage of 0.923, which translates to a shot quality neutral EV save percentage of 0.917. That's pretty close to the league average.

And while his EV save percentage was 0.925 in 2007-08, the Wild's SQA index at EV was 0.89, yielding a EV SQN SV% of 0.916. Again, that's fairly middling.

In any event, it'll be interesting to see what happens to Backstrom's numbers when/if he leaves Minnesota.

5/05/2009 9:47 pm  
Blogger sunnymehta.com said...

I too have discussed this a few times, once on Mirtle's blog and once on Contrarian Goaltender's blog. That discussion was pretty good, if you wanna check it out...


5/06/2009 12:29 am  
Blogger Black Dog said...

That's terrific stuff Vic

5/06/2009 8:04 am  
Blogger Vic Ferrari said...


I don't put too much stock in shot quality metrics at all.

Looking at your stuff, I think kryzwicki ? , Ryder's associate, had he only numbers that really repeated much or correlated to team save%.

When you look at his equation, though, his allowance for the same shot coming on the PP having a higher chance of going in ... in a way that's an indictment of the mehodology itself, though it's surely practical.

I think you could just add the total 5v4PP minutes played in the league plus 1.5 (or so) x the 4v3 time plus 2 (or so) x the 5v3 time, divide by 30 teams and that's the average PP 'time' in the league.

Then calc the same for each team (in terms of 'time' they spent on the PK), subtract the league average, and voila, you'd probably have something awfully close in terms of repeatability and correlation to team save%.

But we would have solved nothing.

If we add in something like tip-ins (total, and number that went in, and/or on goal relative to went wide). And you'd do better yet in terms of correlation to team save%. But it would come at the expense of repeatability, because that's mostly just luck.

You ar going to post something on EVsave% distribution, no? Maybe if you use 1/4s of the season you'd get a larger sample and a smoother line. Plotted against random chance it will be outside the curve noticably (because Luongo is better than Khabibulin, Kiprusoff, etc.) And it will probably be skewed to the left with a thick tail to the right. That's how most ability curves shake out at high levels of sport, anyways.

Your 'random chance' curve above is just dice rollers. load the dice based on historic EVsave% of the goalies and almost all that discrepancy will disappear. The curves are nearly on top of each other. The wee space left between them (the random chance with weighted dice will be slightly broader I suspect) represents the sum total of injury effects, team effects, and goalies in decline or on the rise. And it ain't much real estate for all those theories to live on.

5/06/2009 10:21 am  
Blogger Vic Ferrari said...

Just to add, the main point of the post above is to show that style and results are different things.

Lemaire teams in N.J had terrific outshooting numbers. Probably because he had better players. While many were rationalizing The Wild's poor outshooting numbers in 07/08 as a style issue, The Wild themselves stated that it was a problem, and were looking for additions that would increase the time the spent in the offensive zone. They even signed the AHL shots leader (Kolanos) to a deal.

And if Gaborik had stayed healthy they probably would have improved in this regard. But he didn't.

If I'd asked this question of Oiler fans, even sharp ones, everyone would have guessed far lower numbers. I think that 99% would have expected the Oilers to have had fewer scoring chances relative to the number of shots on, or at, goal in Minny games. But that just isn't the case.

Hockey pundits never referred to Tampa, DAL or N.J as 'puck possession' teams in 03/04, but all three outshot at a hell of a clip at evens, and had way more faceoffs in the offensive zone than their own. They were all terrific puck possession teams by any sensible measure. Just not by perception. (the Oilers were pretty damn good too, btw, as were ANA, terrible poercentages there though ... Burke inherited a much better team than most realize).

So while the flow in Wild games has, historically, left much to be desired, I doubt it was having much of an impact on scoring chance quality. The shooters on the other team are the ones that make the decision when to shoot or pass or rag the puck.

5/06/2009 10:37 am  
Blogger Vic Ferrari said...

"I doubt it was having much of an impact on scoring chance quality"
should have this added to it:
" or the number of scoring chances in he game, relative to the pace of he game as measured by the total number of shots directed at net."


Also, and 'shot quality' measure should line up with Dennis' scoring chances for the Oilers from last season. I severely doubt that there will be any significant relationship there. I have no plans analyze the SQ metrics, because it strikes me as a waste of time, I just don't see the validity in them. I do, however, look forward to reading someone else's cold analysis of the matter.

5/06/2009 10:49 am  
Blogger Statman said...

What's wrong with SQ / SQNSV%? Isn't it an improvement over straight SV%?

5/06/2009 12:46 pm  
Blogger The Contrarian Goaltender said...

Teams shoot from farther away against Minnesota than they do against average teams at even strength. Or, more accurately, the shots that end up on net against Minnesota originate from farther away than they do for average teams 5 on 5. That's what the NHL play-by-play data has said for years, and that is what the shot quality metrics are capturing. Is that entirely scorer bias? If not, then how is that not significant? Longer shots are less likely to go in, are they not?

I think you've demonstrated from Dennis' numbers and just in general that the NHL team with the possession advantage will eventually get the puck into dangerous areas and eventually create scoring chances. The question is whether scoring chances are created equal. I'm still not convinced that they are in a place like Minnesota. They're probably coming from farther out, and including fewer really good chances like rebounds, odd-man rushes, etc. I've seen numerical evidence that Minnesota allows fewer rebounds than average, for example.

5/06/2009 3:11 pm  
Blogger Vic Ferrari said...


There is a post on here earlier linking to the Wild management blog, and they talk about playing a trapezoid in their own end. Bank Shot clarifies iirc.

Dot-to-goalpost is the age old rule, but they seem determined to take away the low shots from bad angles as well, the shots from outside the dots by angle, but close in. Of course for everything you take away you give something back, your players can't be everywhere. They'll end up with more shots from the high middle. Shooters decide when to shoot, not goalies.

And playing passively in your own end (box and a man in, hard 4 + 1, or apparently "layered own zone defense" according to both Crawford and Andy Murray in broadcsting stints ... this game needs some consistent naming rules :D ). That slows the game down a bunch. (SEE Anaheim in the next one vs DET, at all times the Getzlaf line isn't on the ice, THAT is a boring team)

Dennis said at LT's that the scoring chances were going to be negative for Marchant vs Datsyuk, but that there weren't many in total. That's why. And you can isolate that using shots metrics just using the play-by-play sheet. As it's just one game.

Almost all baseball fans believe clutchness is a huge element of the game, almost all hockey fans that team play affecting shot quality is a huge element of the game. But nobody has ever been able to prove either to be true, so they probably aren't.

Go back in time 20 years and tell 1000 baseball fans that OBP is a better predictor of future results that clutch performance ... get more than 1% buy-in and you should be in politics. But screw politics, you've got a time machine for crissakes, the world is your oyster. :)

Regier says that games with fewer total shots directed at net have fewer scoring chances and are just generally less entertaining.

Hopefully someone like Scott, Tyler, Dennis, etc do a poll of the oilogospher readers of "funnest games to watch last season" and "most boring games last year". Then as well as checking the BUF theory mathematically (20 most total corsi vs scoring chance total in the same games, and the same for the bottom 20), we'd have some suggestion of how 'flow', or entertainment value, is reflected in scoring chances and corsi.

There is another post here that scraped all of the data from (the admittedly very dodgy)

5/06/2009 3:34 pm  
Blogger sunnymehta.com said...


"Almost all baseball fans believe clutchness is a huge element of the game, almost all hockey fans that team play affecting shot quality is a huge element of the game. But nobody has ever been able to prove either to be true, so they probably aren't. Go back in time 20 years and tell 1000 baseball fans that OBP is a better predictor of future results that clutch performance"

Bleh, that analogy is a little convoluted imo.

Define "proof". IMO no commenter here believes in shot quality BECAUSE OMG IT HAS TO EXIST, but simply because empirical analysis has shown correlations between shot distance and goal probability.

Define "huge element". IMO no commenter here believes shot quality is a better predictor of future results than other better metrics. But if shot quality makes even a 10 percent difference (Alan Ryder's ballpark IIRC), and if significant outliers can exist, it seems foolish to ignore it.

A better baseball analogy IMO would be BABIP. Until recent years people probably overvalued the effect that pitchers had on balls in play. But then many statheads went way to the other side, that pitchers have NO control of it, and that all non-strikeout pitchers were lucky. But it's hard to ignore the Tom Glavines of the world. Absence of evidence is not evidence of absence. Eventually data comes out that shows pitchers can control GB/FB/LD profile to a good degree, which in turn affects BABIP. Now more great stuff is coming out about how pitchers miss with different pitches within the strike zone. It all sheds more light, and we all learn more.

So to recap, here's what we know.

1) Minnesota has given up quite a few shots at Even Strength the past couple of seasons.

2) Relative to that, they haven't given up a lot of goals.

3) Their average recorded distance of shot allowed is further than average.

4)Correlations exist between recorded shot distance and goal probability.

5) Minnesota's management feels Backstrom plays a significant enough role in their low GAA such that he is worth $6M/yr.

I personally am confident in saying that I have no idea exactly how much MIN's ES Sv% is the result of Backstrom versus shot quality.

5/06/2009 5:41 pm  
Blogger Vic Ferrari said...

"I personally am confident in saying that I have no idea exactly how much MIN's ES Sv% is the result of Backstrom versus shot quality."

Neither am I. I suspect there is something to it, just precious little.

I mean .010 is a lot on terms of sustainable save% at evens, but that's just one extra shot on goal going in out of 100 ... you're not going to see that well with the naked eye, if at all. We;re left to math tools to tell us if i really exists.

And the only reliable shot distance data comes from the NHL, and that is just one dimension, distance from the backboards. Take away the middle high (Lidstrom's Uncannyland) and you leave the low close angles (Mario Country). You give to get.

If you don't see a shift in the general population, then it's small. And you're left to build the a picture out of dots. If the picture you build out of dots should result in big spreads in the results of the general population, but doesn't, then it's probably wrong.

Jlikens' look at the shot quality stuff has genuine merit, but the fact remains, I could and a "x3" into the equations of the stuff JLikens was looking at, making shot quality seem trebly important, and it wouldn't change the results of his assessment one iota.

Nobody has ever addressed the magnitude in a meaningful way, not that I've read. Where is the model?

The baseball sabrmetrics stuff you reference is unfamiliar to me, though almost certainly simply and tragically misleading, if history repeats itself, at least. Do you wager on baseball, sunny?

5/06/2009 6:06 pm  
Blogger Vic Ferrari said...

Just googled BABIP. I suspect that the distribution of talent for pitchers has no tail to the left, or very little, a slight skew to the left (median a shade less than the average) and a tail to the right.

Has anyone figured that out yet, the behaviour of the population? Knowing a bit about these frequentist thinkers and their proclivity for elegant 18th century pure mathematical solutions, and their equal disdain for nature, I'd doubt it.

5/06/2009 7:07 pm  
Blogger JLikens said...


I tend to agree with you in that, for the majority of NHL teams, shot quality isn't very meaningful at all -- most teams cluster around the league average and there isn't really much variance.

However, as Sunny mentioned, I think the metric has some value in terms of the teams at the margins, like the Wild -- teams that have year after year been among the league leaders in shot quality.

Even then, the effect isn't too significant -- you mentioned a possible effect of 0.01, which I think is reasonable.

And you're right in that a lot of the team-to-team variance in shot quality can be explained through special teams (that is, a team's tendency to take/draw penalties).

But even if we confine the analysis to even strength play -- note that the numbers I used in my original post are for EV only -- it still appears that at least some of the Wild's enhanced team save percentage can be attributed to the team rather than the goaltender.

I'm not saying that this is necessarily discernible to the naked eye -- it may or may not be. But there does seem to be some effect.

5/06/2009 9:42 pm  
Blogger sunnymehta.com said...


I lol'd at this:

"Knowing a bit about these frequentist thinkers and their proclivity for elegant 18th century pure mathematical solutions, and their equal disdain for nature, I'd doubt it."

You have a way with words. I really enjoy your writing.

I'm not sure about population graphs for BABIP. The information surely exists, but google isn't helping me find anything. I loosely know Nate Silver (of Baseball Prospectus) from poker, maybe I'll email him.

And no, I don't bet on baseball. Know a bit about the relevant metrics and all, but the market isn't so great. I'm sure in the pre-sabr days there were a ton of inefficiencies that aren't exploitable now. (Hockey betting is where the money's at these days!) :)

Also, I thought you'd find this link interesting...


5/07/2009 2:45 am  
Blogger Vic Ferrari said...

I think the market for baseball looks good, I was going to take a run at it this summer, in the unlikely event that time permits.

Generally lower holds in MLB on the games lines, so there should be more opportunity.

5/07/2009 8:32 am  
Blogger Vic Ferrari said...


Well if we're talking about a top-five team in shot quality being expected to get one extra point in the standings due to this aspect of their game, and a bottom five team losing one point in the satndings over the course of the season ... then it still seems exaggerated to me intuitively, but it's reached the point that it's not worth arguing about.

But, Minny at the extreme edge, flattering their goalie by .075 in save%, that seems huge to me, that would be an additional five points in the standings because of shot quality against. Implying they could get very nearly the same results with any of dozens of goalies available this summer.

And Minny themselves, knowing their own scoring chance numbers and systems, having watched and rewatched all of their games, with a respected goalie coach ... they remain oblivious to this phenomenon? Seems like a stretch to me.

And the only publicly available scoring chance data tells us that there is nothing in it for this extreme team.

I dunno, the practical arrows are pointing the opposite direction of the pure math arrows, at least by my eye.

5/07/2009 8:48 am  
Blogger Vic Ferrari said...


Thanks for the link to the sisu article.

It's important to know how these shot quality metrics are built. Essentially they are constructed using series of linear regressions. Using any and all data to find the manipulation of numbers that results in the highest relationship of 'shot quality' to 'shooting percentage', bith for and against.

And of course the pearson correlation is essentially an element of linear regression. So if we use pearson correlation to assess it, we get some sort of small positive correlation because we must. That's the way it was built.

And these correlations won't change if I publish my own shots quality data by stealing their equations and multiplying by .1, I would have made shot quality one tenth as important, and every test conducted on previous measures would have the same veracity (aleit poor) with my data.

If the magnitudes are correct, then we could build a table, or a chart, of the shot-for quality distribution in the league. Run a simulation, and get a spread of results, in terms of team shooting percentages, that is way, way wider than the one that JLikens was kind enough to show us on his blog.

Why? Because the magnitudes are too high and because the series of regressions captured a lot of luck. That's why the repeatability (odd games vs even games) for EV shots-against data will be poor, but slightly positive, and the predictive value nil.

5/07/2009 9:09 am  
Blogger The Contrarian Goaltender said...

"And Minny themselves, knowing their own scoring chance numbers and systems, having watched and rewatched all of their games, with a respected goalie coach ... they remain oblivious to this phenomenon? Seems like a stretch to me."

Is Minnesota's front office also aware that they have a 24 year old backup goalie with a .930 career even-strength save percentage on 1,207 SA, as well as a .924 career save percentage in 3 AHL seasons? From the binomial formula, that's a .06 probability Harding puts up those results or better in the NHL if he is an average goalie at EV and there are no Minnesota team effects.

I think Backstrom is an above average goalie, and the Wild were just making the safe choice. Either that or they expect him to improve, but he's already 31 so I doubt that is going to happen. Even if shot quality effects are minimal 5 on 5 I'm not sure I see the value in Backstrom at $6M over Harding at whatever he'll be making. And if playing goal really is much easier in Minnesota, then I really don't see it. So I'm not sure I buy the "Minnesota's front office wouldn't get it wrong" argument.

"It's important to know how these shot quality metrics are built. Essentially they are constructed using series of linear regressions. Using any and all data to find the manipulation of numbers that results in the highest relationship of 'shot quality' to 'shooting percentage', bith for and against."

That's true for Ryder and Kryzwicki, but Gabe Desjardins has one that is, as far as I can tell, based entirely on shot location. He figured out the league average shooting percentage for shots from each spot on the ice using both X (from back boards) and Y (from side boards) distances. He also treats 5 on 5 and 4 on 5 separately. For what it's worth his results suggest that Niklas Backstrom faced the easiest EV shots of any goalie in the league over the last 2 seasons.

5/07/2009 1:54 pm  
Blogger Vic Ferrari said...

The cbssportsline stuff that Gabe uses is pretty wild though. As I commented on Gabes blog, just watch the Oiler ones go up as a game is on. They'll go back and rationalize them when ther is a break to make the times and number of shots match NHL.com. It's hilarious.

The guy who records them, here at least, must be drinking heavily. I think I could do better just by listening to the radio.

I wrote a script to peel all that CBSsportsline stuff off of there about two years ago. I realized that all of the ice shot location, goal shot location, and type of shot were wrong much of the time. And I mean not even close.

I used it in aggregate (all nhl games in all buildings) to make a plot of 'where goals are scored from'. It's in a blog post here somewhere in the archives. Hopefully the randomness washed out given 30 different scorers, or some guys had tendencies that are the opposite of others, cancelling them out.

5/07/2009 2:23 pm  
Blogger Vic Ferrari said...

BTW: Those ar damn impressive numbers for Harding. I can see why MC was pimping for an RFA offer sheet there.

5/07/2009 2:26 pm  
Blogger sunnymehta.com said...

Question for the panel...

Do we have any evidence of shooting percentages being affected in situations where one team is leading by, say one goal, late in the game? I remember Tyler posting that chart of shot ratios being affected by score, but I can't remember if any definitive conclusion was drawn about percentages.

It seems like a decent place to start, no? Because even if we throw out the distance-related shot quality metrics, if we see shooting percentages drop and save percentages increase across the board when a team is protecting a lead, wouldn't that be at least some indicator that team defense is able to affect save percentage?

(Although, as I finished writing that sentence, it occurred to me that it could be another chicken-or-the-egg thing because even if we saw the percentages affected, one could make the argument that it's the offensive players choosing to shoot from anywhere.)

5/07/2009 6:48 pm  
Blogger Vic Ferrari said...

People look too much at playing to the score imo. Over one game it can have an effect for sure, but if we're talking forest and not trees ... then it mostly washes away as the same team is going to be leading and trailing at different times in the season.

I think a lot of these things that seem to affect save% in isolation cancel each other out. I mean when Luongo was in FLA he put up huge EVsave%'s.

And one group of fans thought he was overrated because he faced a lot of shots, and goalies who face a lot of shots should stop a higher percentage. That it was harder to be standing around for 10 minutes without a shot, like the goalies in DET or N.J, and then have to face a short barrage of shots.

And the other group of fans thought that he would be even better behind N.Js defence, he wouldn't have to face as any rebounds and their D never got beat one on one.

Now it's possible that neither were right, but it's also possible that both were right in some measure, and the phenomenons cancelled each other out.

I dunno, just spitballing.

And as CG points out today, just because it's not there in any significant way in the general population, doesn't mean that it doesn't exist for some teams in some way. But it makes it less likely, and very difficult to say with any confidence.

I'd be interested in hearing other folks' take on it.

5/07/2009 9:11 pm  
Blogger mc79hockey said...

The guy who records them, here at least, must be drinking heavily. I think I could do better just by listening to the radio.Even with Rod?

Great series of posts Vic. I'm still digesting them all. I clearly picked the wrong week to go on vacation and not really look at the internets.

5/10/2009 11:42 pm  
Blogger Hawerchuk said...

Great stuff here, Vic, as usual. I hadn't checked your site in a while so there's tons to digest.

On the shot quality stuff: I don't use CBS sports line for shot data (I took one stab at it and gave up) but I do use the ESPN location data. From what I can tell, it's part of Stats Inc's pay feed, which is not part of the NHL's feed due to some falling out they had. I ran a huge QA on the ESPN data to make sure shot time and distance matched up with the NHL feed and it correlated extremely well.

I'd be happy to take a run at comparing Dennis' data to the shot quality data - is there somewhere I can get the data in on spreadsheet or should I just screen-scrape MC's site?

5/21/2009 6:59 am  
Blogger Jim Philips said...

I think that it is great that they are doing great against goals. It is something beneficial for me and the pph free demo

5/30/2013 11:56 pm  

Post a Comment

<< Home