Thursday, May 13, 2010

Forest v. Trees

I thought I would take a quick look at the effect of defensemen on save percentage, relative to their teammates. This is just for 5v5 in the NHL during the past two seasons.

To do this I used Desjardins' terrific behindthenet statistics site. I arbitrarily set the cutoff at 30 games played, and grabbed the 08/09 and 09/10 data.

From there I took the on-ice save percentage and subtracted the off-ice save percentage. The off-ice save percentage applies only to the games in which the player was on the roster. If you're doing this yourself, note that the shots on Gabe's site are actually saved shots, you'll need to add goals against to get total shots. Also empty net goals are included in the data, which is unfortunate, and is going to make the offensive defensemen on bad teams seem a touch worse.

By way of example, in 08/09 Chris Pronger had a 5v5 on-ice save percentage of .915. When he was in the game, but not on the ice, the opponents scored at a 2.20 goals per 60 clip, and the ducks made saves at a 26.5 saves per 60 clip. So 26.5/(26.5+2.20) = .923. So, in 08/09 Chris Pronger had a 5v5 off-ice save percentage of .923. His net score was -.008, meaning that the Ducks goalies stopped pucks at a .008 better clip when he was on the bench at 5v5, rather than on the ice. We'll call that his 5v5 save percentage score.

This season Pronger was +.008.

The same exercise was repeated for all players.

Real Effects:

If there is a real ability of defensemen to affect shot quality, it should repeat from season to season in the general population. So below is a scatter plot of 08/09 vs 09/10 save percentage scores for all defensemen who played at least 30 games in each season. Click to enlarge.

The guy at the top right is Mark Fistric, the player to the far middle right is Brett Festerling, and Jack Johnson is the player the furthest to the bottom left. These aren't outliers in the true sense of the word, the universe requires that some guys have this level of good and bad fortune. In fact if you remove them, the correlation from season to season becomes negative, and the bunching of the dots becomes far more tightly grouped than we would expect by random chance.

In fact, the bunching is tighter than we would expect by random chance already. The reason is NOT survival bias (a phenomenon that corrupts many MLB stats beyond recognition) coaches and GMs in this league seem to have this stuff figured out. You could make a case that Alzner would have made the Caps this year if he'd been luckier in 08/09, and that Hedican probably had another year or two left in him. But that's just two guys and both are at the edges of their careers, it's a non-issue on the whole.

There is some censorship bias though. The group is bunched a bit too closely together because good defenders tend to play more against good forwards, so their 5v5 save percentage score suffers a smidgen. And if a guy is playing tough minutes and getting shelled in terms of save percentage score, coaches tend to give them a break from the tough gig, probably just to make sure that they don't start losing confidence. So the correlation of Corsi QualComp to save percentage score is only r=-.01. But this defender was playing tough minutes in the first place because he's a good player, the coaches know that, so next year he'll be back to facing tougher comp. So the correlation of 08/09 Corsi QualComp to 09/10 5v5 save percentage score is a touch stronger, r=-.09.

In terms of old school QualCOMP, which is a tad self referring. postdictive r=-.05, predictive r=-.10

The same phenomenon occurs from 07/08 to 08/09, though not quite as strongly.

Bottom Line:

The ability of defensemen to affect shot quality against does exist in the population, but it is so small that we will never be able to sensibly apply it to any player in particular. And a paradox is created, the type of defensemen who are helping the goalie save percentage a bit (presumably because they make fewer mistakes of the spectacularly bad variety) are, as a group, seeing slightly worse save percentages behind them, because they are the guys the coaches are leaning on to play tougher opposition. And the guys who have talent but are guilty of the occasional egregious error ... as a group, they do a whisker better than average by 5v5 save percentage score. This is presumably because their coaches have the good sense not to play them much against Malkin, Kovalchuk and Heatley types.

Trailing Thoughts:

Even though it will not be popular with fans, I think the right guys for a team to target are defenders who have been getting some bad bounces in recent seasons. They should come cheaper in trade than their true value dictates. They are:

1. YouTube clowns. Souray was the poster child for the phenomenon. Plus he had a brutal save percentage score his last year in Montreal, but had a decent level of competition (Carbonneau wasn't fooled) and decent Fenwick numbers (and therefore almost certainly decent on-ice scoring chance numbers). That save% score bounced back in a ridiculous way for his first two years as an Oiler, and plummeted this season, creating the illusion that Charlie Huddy was a supergenius. The dice have no memory, after all.

2. Guys with back to back seasons of poor 5v5 save percentage scores, three in a row is even better. NHL teams aren't swayed much at all by these (oddly enough, it is the implicit core focus of fandom) but no matter how square your head is ... two or three seasons in a row of bad luck and the mind searches for reasons. Just watch a craps table for a couple of hours if you don't believe me, and that's with dice. So it's bound to affect at least some GMs. The universe requires it happens to some defenders, so I think these guys could come cheaper than their true market value. Zbynek Michalek fits the bill, Colin White, Bieksa and Jack Johnson maybe. Niedermayer isn't going anywhere, unfortunately, but he's been rolling snake-eyes over the last couple, creating the illusion of a sharp decline in his play.

3. Scoring chance % may not be everything for defenders, but it's almost everything. And it repeats really well just as a raw number. If you apply LupulSmid to account for quality of D partner, then the season to season correlation goes through the roof, r=.85 for the Oilers D, and that's without accounting for the rest of the contextual issues (quality of competition, how often they start in their own end of the rink, etc.). So it's time to move a guy like Smid, before the bottom falls out of his save% score. Because it will eventually.

30 Comments:

Blogger R O said...

2. Guys with back to back seasons of poor 5v5 save percentage scores, three in a row is even better. NHL teams aren't swayed much at all by these (oddly enough, it is the core focus of fandom) but no matter how square your head is ... two or three seasons in a row of bad luck and the mind searches for reasons. That's bound to happen to some GMs. The universe requires it happens to some defenders, so I think these guys could come cheaper than their true market value. Zbynek Michalek fits the bill, Colin White, Bieksa and Jack Johnson maybe. Niedermayer isn't going anywhere, unfortunately, but he's been rolling snake-eyes over the last couple.

Your post is on defenders and I hesitate to just come out and say this but this probably applies to forwards too, right?

In which case, Brad Richards is the perfect example of multiple years of bad EVSV% behind, if he hadn't had such high offensive numbers this season some team might have been able to victimize Dallas in a trade.

5/13/2010 10:53 am  
Blogger Vic Ferrari said...

I think with forwards the mind tends to gravitate more towards the offensive numbers. So on-ice shooting%. There is a very real effect with that, though for any particular player it's not possible to sift out the noise.

I don't think that GMs are as swayed by the luck numbers as fans though. Before he was traded from Tampa Brad Richards was on almost every fan's list of worst contracts in the NHL. In the whole of the Oilogosphere, only me and jon k were hoping the Oilers would try to acquire him. Yet there was a bidding war for the guy when T.B put him on the trading block.

And Richards has no beef with the hockey gods since the trade. Especially in the Dallas playoff run. He's been in an all star game as well. Hit a lot of posts last year, but overall he can't bitch.

That's a guy I hope the Oilers find a way to acquire, though. With Vis gone and Souray all but gone ... they need a good powerplay point man, and Richards is one of the best. Plus he's a complete player that can take a lot of the load off of Horcoff and send a positive ripple through the lineup.

I don't know if the Stars are trying to unload salary or not, it's been rumoured that they have. If Richards is traded, fandom is going to be shocked by how much DAL garners in return, methinks.

People can point to the counting stats all they want ... Richards is a hell of a hockey player. Just is.

5/13/2010 11:13 am  
Blogger R O said...

Richards is also on my wish list, and I've heard the Dallas salary dump rumor as well. I thought a Kiprusoff package would do it, then Dallas went out and got Lehtonen. Still, the former had a year of pucks hitting him, and the latter is either made of glass or in the 25th percentile as far as injury luck goes, so there's hope yet.

In any case, I don't know if this is off the board but if I ran the Oil right now I would probably trade every kid in the bigs (Gagner, Brule, Cogliano, Smid) for Richards. Only Gagner seems to be worth a damn anyways, and if he ever turns into Richards then the Oiler fans should probably rejoice.

5/13/2010 11:34 am  
Blogger mc79hockey said...

YOu beat me to the punch on a post I spent a great deal of time painstakingly collecting data from your site for. I was trying to figure out why you'd use Gabe's site when it suddenly became clear to me that it's a hell of a lot quicker.

Bastard. Good post though.

5/13/2010 11:35 am  
Blogger Vic Ferrari said...

Tyler:

Yeah, it takes about ten minutes total to grab the data from Gabe's site, post it in a spreadsheet, calc the save% score for each season ... then vlookup() to get the paired data. And I'm lazy that way :)

I just came up with the idea for the post a couple of hours ago.

Your way is miles better though, these empty net goals are fucking things up. It's fine brushstrokes, but I'm not even sure how they would effect things.

Using the timeonice.com data will be labourious, but clearly better. If you've already started, please carry on. It will be better than my hasty post, for certain.

I'll build a random effects model (hyoergeometric, a deck with marked cards, essentially) for you as well, then you can modify it to use in the future. Sunny uses Bernoulli (weighted coin flipping) models, and I have no beef with that, near enough the same result anyways, that's what Jim Albert generally uses. I usually prefer the Calvinist approach though, that was Bill James' bent before he got distracted and influenced by lesser minds with more statistics education.

5/13/2010 12:09 pm  
Blogger Vic Ferrari said...

RO

I don't know what the market would be if BR were available, and maybe it's my Oiler bias, but I'd be heartbroken if the Oilers dealt Gagner. The pucks haven't crossed the line for him as much as we'd all like, but he's a really good hockey player. Still really young too, so he's going to play like a dink at times, and he'll get a lot better yet.

I'd like to see them move Brule while he's hot though, or at the very least play hardball on a short term contract this summer. We'll see.

I PVRed the WHC game from yesterday and watched the first couple periods last night, the Canadian team were dominant but snakebit. Anyhow, Ray Whitney can still play. He's as old as dirt, but he must have avoided serious injuries. He doesn't cheat for offense at all any more, kind of important when you're playing with Duchene and Stamkos. And Brooks Laich is really good. Granted maybe I've seen him good, he always plays well against the Oilers. That's a guy who might get lost in the shuffle in Washington, and he's the kind of guy who helps you win games, methinks.

5/13/2010 12:23 pm  
Blogger R O said...

On the WHCs:

Why am I not surprised? Bruce wrote this lovely bit...

Canada outshot the Swiss but in no way shape or form did they outplay them. Switzerland played aggressive, bend-not-break defence that kept Canadian snipers to the outside for the most part, and former Dallas backup Tobias Stephan did the rest in his very first World Championship start with an extremely solid night's work. Switzerland always gets the great goaltending, don't they? Meanwhile they also counterpunched effectively, generating a handful of odd-man rushes and a plateful of ten-bell scoring chances, only some of which they converted. And as the game wore on, they carried larger stretches of the play.

... and I didn't catch the game but there it is, the narrative written right after the fucking game ended.

And it might just be me but the centre depth chart for the Canadian team (at least when Scott listed them out as centres) looked just horrific. Duchene, Tavares, Stamkos, who probably still think the game like its their junior team.

The wingers are of course awesome although for some reason Smyth ended up on the "fourth" line according to Scott. He's never been one to stay healthy for long stretches though, and he didn't look so good in the LA-VAN series so prior injury seems likely. One wonders though why he accepted the invite if he was banged up. Olys, they ain't.

5/13/2010 12:31 pm  
Blogger Vic Ferrari said...

I think that Smyth and Brown were both playing hurt in the series against the Canucks. Neither were winning puck battles the way that they normally do.

As for the narrative you quoted, that must be from Bruce at C&B, no? I avoided the article because I didn't want to know the score, but Canada outchanced the Swiss by a wide margin. Some guys were playing lazy, though. Perry in particular, cherrypicking like a fiend. I have trouble recognizing a lot of the guys, though. Just not familiar enough with their numbers after one game.

I thought Myers was impressive, and you can't miss him because he's the size of two people. Wore the goathorns on the first two goals against ... but not really fairly. If the first goal isn't a perfect deflection we don't remember it, that guy could try that in practise all day and he wouldn't do that very often. The second one looked like it bounced off Myers' skate. Shit happens, he was where he should be, and that's only because the other two deep defenders were where they were supposed to be ... strong post and on the man, though the latter kinda missed, didn't catch his number. Again, shit happens. If not for those two plays, and Canada buries some chances ... Myers is probably a star of the game.

That's a young squad though. Beauchemin was the only guy who made you feel safe on the back end.

Cool to see MacTavish and Messier in clips. Both looking every inch the high stress guy. Messier has really aged. I suppose we all have.

Team Canada is a really good team though, better than I thought they would be. Hockey gods get the last say of course, but you have to like their chances in this tournament.

I just love the WHCs, next time they have a good location I'm going. If it's Finland and it laps over Mayday ... it's a can't miss. Until you've drank with the Finns, you don't really know what humanity is capable of, RO.

5/13/2010 1:05 pm  
Blogger Vic Ferrari said...

Jebus, dude put the score in the headline. Whether I read it or not ... that's bad form.

5/13/2010 1:19 pm  
Blogger R O said...

I've never met a Finn in my life, it appears I've been missing out. Noted though.

Myers, from all indications, looks to be a fine young defenceman. Related: your head is screwed on tighter than most with regards to these sorts of things, what do you make of Doughty?

I think he'll be a good defencemen yet but every year the hype spotlight centres itself on somebody. Doughty seems to be a decent young player but he's prone to getting beat cleanly (at the Olys he was victimized quite a few times).

I don't know, he just reminds me of all the incomplete young stars of today who obviously ooze skill out of their pores but haven't quite matured into the complete and trustworthy players that the fans or counting stats hype them up to be.

5/13/2010 3:44 pm  
Blogger Sunny Mehta said...

Funny, I too was thinking about tackling the defensemen-Sv% thing. I was first going to do the usual coin-flip model, but thought for sure it would show no skill whatsoever. I'm not thrilled about the idea of setting a minimum of games played, for the usual survival bias reasons. (Though you seem to think the effect is less with NHL defensemen than say NHL goalies or MLB batters, eh? I haven't looked into it.)

Off the top of my head, teams seem pretty freakin' fickle about who they give D minutes to. The Devils have had a rotating carousel of guys as well as a few youngsters who seemed to all be in line for "top pairing minutes" depending on who was "playing well." I don't know, that's just one team though, and maybe the empirical evidence says something different.

My next thought was to separate out the population of first-pairing defensemen and compare them to a different population of defensemen. (And perhaps qualify first-pairing as the guys who played the most.) But the problem there is two-fold: One that you might see the same survival bias effect I just mentioned, Two is the issue you bring up in your post that top-pairing guys are gonna be playing against good forwards night after night, so it might actually negatively impact their Sv%.

And we're back to square one.

What's the Calvinist model? (I.e. how does it differ from the coin-flip model?) I'm not religious.

5/13/2010 3:49 pm  
Blogger Vic Ferrari said...

Sunny,

In most cases it works out the same, such as your save% model, it's big underlying numbers there.

By way of example, if Pronger had been on-ice for 500 shots and 40 goals against, and off-ice for 1000 shots against and 75 goals against ...

... we'd model that as a deck of 1500 cards, with 115 of them marked (40 + 75). Then in a trial we'd shuffle the cards and deal Chris 500 cards, then count the number of marked cards in his stack as a measure of his on-ice goals in the sim. The remaining marked cards represent his off-ice goals against.

Do the same for everyone in the sample ... plot it out, and it will be strikingly similar to the actual one in the post above, at least most of the time. Do it enough times, eventually we'll get one that plots out as the image of Jesus, but it doesn't matter.

Run that sim enough times and you'll see that the scatter plot above is a smidge too tight, it's actually the opposite of popular opinion. The players are closer in terms of "shot quality ability" than the universe allows. By an amount probably too small to see with the eye, but plainly visible in the light of math.

So either the universe is imploding or good players play a lot against good players in this league. So you check with qualcomp and it all makes sense.

Simple stuff.

The effect of defenders on the save% behind them is clearly minute, and the effect of players on the scoring chance % when they are on the ice is clearly enormous ... some I'm done at this.

If you were keen, you could check to see the expected save% behind a guy, given who took the shots when he was on the ice. Then you could, for example, deal Pronger 502 cards, instead of 500, to compensate.

Makes sense, no?

5/13/2010 4:29 pm  
Blogger Black Dog said...

Good stuff Vic, good to see you have returned, at least briefly.

My Dad's best friend was a Finnlander and there are plenty up in Northern Ontario, especially in Sudbury. Tough folks, the Finns. Miners and woodsmen back when Sudbury was a collection of huts and muddy paths.

Tough and a hell of a lot of fun, at least the ones I've known. I've heard they are a morose lot but I haven't seen it.

Michalek would be a sweet get but I read somewhere he's asking for 4 per.

Plus why the hell would you come play for this sadsack club?

As for Richards, yes please.

5/13/2010 8:45 pm  
Blogger JLikens said...

Nice post, Vic.

Do you suspect that the same holds true on the PK (4-on-5)?

I recall a post of yours where you found that skaters have very little control over their on-ice PK SV%.

Oh, and count me in as someone fond of Finland and its inhabitants.

I have an interest in population genetics and they're very peculiar in that regard.

5/13/2010 11:35 pm  
Blogger speeds said...

Vic:

So, for skaters, there is an ability to influence/increase on ice shooting percentage, but that ability is not nearly as significant as the ability to increase shots/scoring chances?

5/14/2010 1:10 am  
Blogger Vic Ferrari said...

speeds:

There is a real effect of skaters to affect on-ice shooting percentage. If you use this exact model you get a scatterplot that has an obvious trend ... bottom left to top right. Just change the last letter in the linked urls above from "D" to "F".

r=.4, if you know what an ordinary scatterplot with a correlation in that range looks like. That is ALL that r means here, never, ever use it in math.

Also the correlation of qualcomp to on-ice shooting percentage is significant ... in the r=.3.

Now obviously that does NOT imply that playing against good opposition helps your on-ice shooting%. It's the result of good players usually playing against good players in this league.

That's a first blush, give-you-a-feel-for-it thing. This model has some problems with the forwards and shooting. Notably that fighters should be excluded from the population, and there are also significant survival bias problems.

Teams seem to know that this is largely a random thing, moreso than fandom for sure. I mean there is no way in hell that a guy like Goc is still in the NHL after 08/09 if the talk radio callers from each city were running the show.

But unless you're famous ... two bad years of this and it's Europe for you (SEE Thoresen. And he brought other things to the table. This past year he put up gaudy totals in the KHL based on unsustainably high shooting percentages, and presumably the same for his on-ice numbers, based on the +/-. Lady luck is a fickle bitch.

Just do the copy and paste thing speeds, and look at the guys who fell out of the league ... it's a pretty long list of "Christ, he was a good player, where is he playing now?" guys.

5/14/2010 9:20 am  
Blogger Vic Ferrari said...

speeds:

Also, as a quick check for the presence of survival/censorship bias ... look at the survivors in the population (say guys who were NHL regulars playing top-9 ice time two seasons in a row) ... if you remove the bonafide stars, the remaining group will collectively show a drop in performance from the first season to the next. This for any pair of seasons.

Or just randomly selecting 30 players from the group of survivors ... would the smart money be on them dropping off in performance? Or increasing?

It's a bit more obvious in NHL goaltending stats and in MLB stats, pretty much all of them. But it's clearly present with shooting% and on-ice shooting% as well.

5/14/2010 9:26 am  
Blogger Vic Ferrari said...

Pat:

Michalek is asking for 4 mil?! Wow. I wonder if he's looking at his brother's paycheck and thinking he should get the same. I remember when someone gave Dmitri Mironov a stupid contract Bobo Mironov suddenly thought he was getting jobbed on his deal.

In any case, I tend to be surprised by how high almost every contract is ... I must be getting old. Have I told you that wigwag chocolate bars were only 15 cents when I was a kid, and they were freaking huge! A pack of nibs was a nickel, and a box of candy cigarettes cost a dime. Based on that alone Zbynek deserves half of what he's asking!

5/14/2010 9:31 am  
Blogger Vic Ferrari said...

Jlikens:

Re: the PK ... I think so too. Not so for the PP though.

Shooters decide when to shoot. I think that's he central theme in all of this.

5/14/2010 9:44 am  
Blogger Sunny Mehta said...

Vic,

re: marked card model

So instead of giving all players the same league average p (as we do in the coin-flip models), you're essentially saying to give each player his own p according to his team's total Sv% in games he played (i.e. on-ice plus off-ice Sv%)? Then you give him the number of flips according to his on-ice shots against (like we would in the coin-flip model).

If the sims are wider than the observed distribution it shows that basically players' on-ice Sv% is the same as their team's Sv%, i.e. there's no evidence of players affecting their team's Sv% beyond what we'd expect from binomial chance variation.

Do I have that right?

5/14/2010 10:00 am  
Blogger Vic Ferrari said...

Sunny,

With Bernoulli trials your parallel universes have the same number of shots and the same p value as our universe.

With Hypergeometric trials you parallel universes have the same numbers of shots AND the same number of goals. There is no p value per se. The goals are sprinkled randomly over the shots, or with a bias, depending on the model.

Using goaltending as an example, the p value in your trials is the aggregate save% in the league for that year. Whose to say it isn't .003 higher than true ability that year, based on chance alone?

So in your goaltending sim you have deemed the shots against for each goalie as an irrevocable decision of God that extends through all universes. If you also choose to make the total goals scored in the league an irrevocable decision of God that extends through all universes ... use step by step reasoning and you'll end up with the card model.

The total number of cards in the deck is the total number of shots in the league. The total number goals in the league is the total number of cards that get marked. The number of shots-against for each goalie is the number of cards he is dealt.

Simple as that. And it that case it should end up with nearly identical results.

5/14/2010 10:53 am  
Blogger Sunny Mehta said...

Thanks, got it. Sampling without replacement vs. with replacement.

Conveniently R has a command to run sims using the hypergeometric distribution that works exactly the same way as the one for the binomial distribution.

And I just tried it by running 10,000 sims on the past three years worth of ES goaltender Sv%. I got the exact same answer as with the binomial. Sweet.

5/14/2010 2:55 pm  
Blogger Black Dog said...

Vic - yeah I read that somewhere on the interweb (so it must be true ;)) - can't remember where but it was someone pretty reasonable, not some of those way out madmen with the rumours

I remember a bag of chips or a can of pop for a quarter, my best friend claims he remembers them for 15 cents, so yes I agree on the contracts. I'm happy to see the players get paid but its rare that I look at a contract and think 'wow, that's a reasonable deal'

After the '06 run I remember predicting what I thought all of the Oilers would get - Horcoff, Hemsky, Stoll, Pisani, Roloson - I was way low on all of them.

Every year I figure a guy will get X and I'm always low. The next year I think higher and I'm still low.

I'd be surprised in Michalek got that but again, what the hell do I know.

5/15/2010 6:41 pm  
Blogger Scott Reynolds said...

Good stuff Vic. This is roughly what I believe to be the case with regard to save percentage. It seems to me the way defenders usually help the goalie most is by limiting the number of shots against (whether by clearing rebounds, breaking up rushes or just keeping the puck in the offensive zone) rather than consistently limiting quality. That said, I'd be interested in your thoughts on Gabe's WOWY study (http://www.behindthenethockey.com/2009/07/do-defensemen-influence-save-percentage.html) which suggests that defenders do have an influence on save percentage.

5/15/2010 11:55 pm  
Blogger Triumph said...

wow, i was just thinking about this today and wondering if anyone had done any research on defensemen affecting ES SV%. the internet is a wonderful place. great work.

5/17/2010 10:53 am  
Blogger Vic Ferrari said...

Scott:

Thanks for the link. Gabe posted that 5 days after I wrote an article called 'The Shot Quality Fantasy', that's not coincidence, methinks.

Jeff J's comment hits the massive problem on the head. And your request to show predictive value ay well have been the item that swung Desjardins into a different belief set, I dunno. Any road, your request from that comment thread has essentially been answered with this post.

5/17/2010 4:17 pm  
Blogger FancyStatsfan said...

Hi there, dont think Ive ever commented here which would make me new, but I read this post recently and spotted some areas I have concerns about regarding the integrity of the study. I appreciate that a huge amount of hard work had to go into a very ambitious post like this and I think its great work other than a few areas that I feel undermine the study's. Here is what I mean by that, just what I think, feel free to disagree, and not looking to offend.

Vic Ferrari, I think you have this wrong. "And if a guy is playing tough minutes and getting shelled in terms of save percentage score, coaches tend to give them a break from the tough gig, probably just to make sure that they don't start losing confidence. So the correlation of Corsi QualComp to save percentage score is only r=-.01. But this defender was playing tough minutes in the first place because he's a good player, the coaches know that, so next year he'll be back to facing tougher comp. So the correlation of 08/09 Corsi QualComp to 09/10 5v5 save percentage score is a touch stronger, r=-.09."

I don't think when Chris Pronger had a couple tough weeks on the ice, his coaches checked his on-ice save percentage score, noticed it was low, and decided to give him easier match-ups the rest of the year, but then, because he's a great player, gave him tough match-ups again the next year.

That's not how it works, not even how it would make sense logically. Chris Pronger in his prime is going to face the toughest competition always and forever with very few exceptations. To suggest the greater qualcomp he faces is balanced out by his coach "giving him a break from a tough gig, probably just to make sure he does not start losing confidence," is an ironic way for a stats blog to skirt over a real issue with the study.

I mean, advanced stats stats have often been used to disprove those exact type of myths. Someone says something like, "How can you say our player was lucky? Our player had a great second half because he was confident! Because the GM finally shored up the backup goaltending position and our player knew there would be support for him defensively if the #1 goaltender got injured, and that security is what gave him the confidence that allowed his shooting percentage to go up to 28% for the last three months of the season." Blogs like this have a field day disproving opinions like that!

But now we come to a real issue with this study, and that's the same type of reasoning given as to why qualcomp is only a marginal issue determining whether defensemen can impact opponent S%. "Well, sure, qualcomp could be an issue in determining whether the best defensemen can affect opponent shot percentage in addition to shot quantity, but I bet the coach will look out for the player's confidence and give him easier match-ups and it will probably all just even out, or somewhere in there."

Doesn't work like that. Even going by the scenario you gave, wouldn't the coach replace the goaltender based on merit if he noticed poor goaltending over a notable period, as in not just based on luck, or, the more logical version of your theory, replace the goaltender for a period of time to save his confidence if the goaltender was simply having an unlucky streak?

Why replace the defenseman if by looking at his on-ice save percentage he has come to the same conclusion as you, that the defenseman has simply been unlucky? If he wanted to help the defenseman's confidence, he could simply tell the defender of his findings, that he's simply having bad luck, and tell him to keep playing his game.

But even if your description was logical, and it obviously seemed logical to you, it's still an assumption that a majority of the NHL coaches would have done what you would have, or even done what was logical and objectively correct, if there is such a thing in this scenario, period.

continued in c2

8/03/2012 2:51 pm  
Blogger FancyStatsfan said...

continued


Coaches make all sorts of strange decisions for many reasons. You can't just assume a majority of the coaches of the dmen in your study would have balanced out the difficult qualcomp of dmen like Chris Pronger by by easing up on their Chris Pronger's for as significant an amount of games as needed to balance out that difficult qualcomp. And if they had, then Chris Pronger's (or whoever's) qualcomp would not show up as overall being difficult in the first place, because many of his games would have been played against easier competition to spare his confidence, and so it's a paradox and the theory collapses on itself.

But I don't want to get too complicated, the point is you can't come to scientific conclusions based off assumption. You were right to point out that this study is flawed because of the tougher qualcomp better defenders face, but I think your second-guessing of that conclusion saying it's only a small factor in the ultimate analysis is incorrect.

I think it would also be wise to point out, I appreciate the subtlety of your conclusion even if no conclusion can be trusted here based on the flawed study. I refer to this sentence of yours. "The ability of defensemen to affect shot quality against does exist in the population, but it is so small that we will never be able to sensibly apply it to any player in particular."

But I thought you would be interested to hear that everyone who has referenced this post to me claims that shot percentage is non-existant. They have forgotten the subtlety of your conclusion, that shot percentage exists but is difficult to quantity, and this majority of the advanced stats community have instead jumped to the conclusion that it does not exist at all, and that every time a team performs over a 7 game series with an above average shooting percentage or PDO, they were lucky, and vise versa. Player performance, health, line combinations, how teams match up against one another, pretty much 99% of the hockey games themselves are being ignored by the stats community because they seem to be misinterpreting blogs like this. Just thought you would like to know your words are being twisted.

8/03/2012 2:52 pm  
Blogger FancyStatsfan said...

Excuse me, should read "non-existent" there in the last paragraph and not non-existant.

8/03/2012 2:57 pm  
Blogger Jim Philips said...

It is impressive how they have been doing that great in the defense. I really take under consideration them when I am in the price per head free trial

5/30/2013 11:21 pm  

Post a Comment

<< Home