Monday, March 27, 2006

Interesting draft post on HF

Within the thread titled "McKeen's New Top 100, March edition" is some interesting information regarding how teams go about scouting and compiling their lists.

I'm speaking specifically about replies from steblick (#31, 87), Blackshad (#32), timlap (#33,35), Dr.Sens(e) (#34, 72), West(#36), Chief (#68, 110), Big Daddy (#82)

In particular responses 31-36,68,82,87, and 110.


I've been wondering how I can seemingly make a better draft list than NHL teams can. Or even if I can, if I haven't just got lucky due to sample size (if my list has been better at all). If I can beat/perform as well as actual NHL teams in preparing a draft list, without seeing many/any prospects play, I think it speaks to a systemic flaw in the scouting system - overvaluing the opinion of scouts who see a limited sample of games.

When you see a player only a few times (say, between 5 and 10) in his draft year how can you possibly expect to have a great read on him? Perhaps team A sees player X (on average) on better days than team B sees him, thus player X is higher on team A's ranking than on team B's? Maybe that is mainly what accounts for much of the variation among scouting rankings, and scouting staffs? In creating my list I don't have to worry about the bias of "saw a player on good days", that should average out among all the sources (to some extent).I guess this isn't really my theory, since it appears as well in "Moneyball" in some way as it releates to baseball.

*Note* if this looks familiar I'm stealing some of it from my old blog, which will be undergoing periodic updates to back up some draft materials, new and old, and some Oilers prospect information


To go further, I don't need to draft better than average for this way of scouting to make sense; by drafting at average I save the money spent on amateur scouting that can now either be spent on current player salaries or kept as profit.

we talked about this on the Oilers board in this thread, might be an interesting refresher to those interested in this, and those who haven't seen the thread before.


Of course, it's impossible to know how my list actually compares with the list of any team, since the list of a team isn't available. This leaves a problem. I can't know my list is better simply by looking at who I would have picked vs. who EDM (or any team) picked, for at least 2 reasons.

(1) When I pick someone different from who EDM actually picked it would have changed the entire direction of the draft in an unknowable way. At least, unknowable without the lists of all 30 teams, assuming they all have lists.

(2) sample size. To illustrate...

team X's top 10 draft list, they pick 4th overall

BUST
BUST
BUST
HOME RUN
BUST
BUST
BUST
BUST
BUST
BUST

team Y's top 10, they pick 5th

HOME RUN
HOME RUN
HOME RUN
HOME RUN
BUST
HOME RUN
HOME RUN
HOME RUN
HOME RUN
HOME RUN

Team X picks a great player, team Y picks a dog, but without having their lists we can't see that team Y is CLEARLY the superior scouting team.

12 Comments:

Anonymous lowetide said...

Terrific stuff speeds, thanks for doing this. I find the entire draft process to be probably the most interesting thing in the sport and have been following it for decades.

Central Scouting grew out of a belief that good talent was being missed, as explained on Hockey Draft Central in their 1976 Quick facts page (halfway down the page).

From what you've written and the pages you've posted it seems as though these drafting sources (ISS is my personal favorite) are ahead of the curve and that a team may well benefit from taking less of a dim view.

Suspect it's all about bias and each team's definition of "best player available."

3/28/2006 12:43 pm  
Blogger Dennis said...

Mike we know the guys you've liked over the years, ie it seems to be mostly guys the Sens select.

But who are the guys you've been totally wrong about?

3/28/2006 3:28 pm  
Blogger speeds said...

there are of course a number of guys.

I had Pouliot 10 OV in 2003, that's likely to be wrong even if he turns into a decent player. I had Jessiman high as well, and he looks worse than Pouliot at this point.

In 2004 I had Alexandrov something like 17th OV, he ended up going in the 3rd round and looks like a big swing and a miss. Here's a link to my 2004 rankings...

http://hfboards.com/showthread.php?t=86629

and my 2005 rankings:

http://hockeysymposium.blogspot.com/2005_07_01_hockeysymposium_archive.html

the july 25th rankings were near my final ranking, but I think I fine tuned it a bit (as in I may have moved 3 or 4 guys on this whole list up or down one or two spots - one of them was Trukhno as I noted in my july 31st entry - Oilers 2005 draft review)

steblick, in a response to my response in the HF thread I mentioned, noted that most teams will end up with the player they get in the 7th round being rated something like 70th overall. For me, the last guy I theoretically would have got, Mathieu Roy, was rated 39th on my list. For what it's worth Roy ended up going undrafted, not a good sign (though the draft did drop 2 rounds), but his stats this year were good/great (10th in Q scoring, team leading 104 pts) so it remains to be seenif/where he's drafted this summer. I'll be curious, anyways.

3/28/2006 4:36 pm  
Blogger speeds said...

In 2003 I also had Phaneuf rated below Pouliot, I think I had him 12th or 14th, or something like that. I also had Coburn 5 or 6 OV, and he hasn't even cracked ATL's blueline yet.

In 2004 I had Schemp 20, and Wolski 21, both lower than they in hindsight probably should have been, but still higher than they went in the draft (in wolski's case I had him ranked 21st, and he went 21st, but COL probably had him ranked much higher).

In 2005 I had Bourdon and Lee much lower than they ended up going, and I probably look wrong there at this early stage.

3/28/2006 4:42 pm  
Anonymous lowetide said...

speeds, a question.

Would you prefer the Oilers to trade up for a first rounder this season, or keep their second rounder (Niinimaki pick) and acquired another 2?

Just reading about the draft so far it looks very much like 2002 where the Oilers did pretty well after the first round (Stoll and Greene).

3/28/2006 5:52 pm  
Blogger speeds said...

I don't think there's really good answer to that LT without knowing what it would cost, and who is avail.

They made their decision to move the first, and the 2nd. There is an element of going for it here, but when you move your top 2 picks it tells me that they maybe they don't think they are missing much, or that they so much like what they have now?

What picks were involved in teh Peca trade, I thought I remember reading their were conditional picks in that deal, from EDM to NYI? Or was it just a 4th, flat out?

3/28/2006 6:16 pm  
Anonymous lowetide said...

I think the Peca pick (can we work "peppers" into this?) was a conditional but I'll check it out.

I bet they deal for a pick, maybe signing Spacek and dealing Staios.

3/28/2006 6:50 pm  
Anonymous lowetide said...

TSN has it as a "conditional" pick.

3/28/2006 7:00 pm  
Blogger Vic Ferrari said...

Some fascinating comments on that HF thread, speeds. I could not know less about amateur scouting, but it surprises me how confident steblick was in his assertion that seeing a prospect a few times was enough to form a reliable opinion on the player.

I'd think that more sets of trained eyes, assessing the same player separately, that this would be the better way to go. Obviously cost comes into play. And if you have agencies like Red Line, CSS, ISS and whoever else ... you'd think that they would count these opinions in with their own, no? But it kind of sounded to me like some of these teams don't use those rankings much. Is that the impression you got?

3/28/2006 11:04 pm  
Anonymous lowetide said...

I know the Oilers use ISS Vic, and I read somewhere over 20 teams subscribe (or something like that).

Part of the problem a team would have is resisting the urge to get an edge.

If all 30 teams have the same information, how tempting would it be to go get better information?

GMs may have different priorities at the draft (they all say best player available but they're a lying bunch if you ask me) but I don't know if you could find a team willing to give up that kind of an edge.

Too much bias toward the "saw him good" crowd I suspect.

By the way, on Guy's Pipeline Show the other night a very interesting item from Bob McKenzie.

McKenzie said that at the 2005 draft Lowe was contemplating taking Steve Downie over Andrew Cogliano. He hadn't seen Cogliano play well (I think that was the wording) but was well pleased with Downie.

Went with his scouting staff, which to me is a very interesting point. Downie was a much bigger factor at the WJC and fits in with Lowe's Coke Machine drafts (there are about 10 of them since 2001).

Cogliano is/was an interesting "skill" pick by this organization.

Sorry to derail this thread, I love this stuff.

3/29/2006 6:47 am  
Blogger speeds said...

yeah, I don't think many teams use the rankings. they probably consult with them in case there's a guy in RLR, or ISS's top 100 that they've never seen/heard of, but I 'm not sure how much they use it to form their final list.

I wasn't surprised to hear that scouts feel they get a read so quickly, but it doesn't make me think my theory is wrong at this point. It may be, but if anything I'm more confident in it now than I was before.

I think most NHL teams do the same thing as each other but each one thinks they are smarter at it than the next guy. Which one reads the tea leaves the best, according to their own standard of how tea leaves are read?

3/31/2006 12:36 pm  
Anonymous autralian job sites said...

I am here because of search results for blogs with a related topic to mine.
Please,accept my congratulations for your excellent work!
I have a autralian job sites site.
Come and check it out if you get time :-)
Best regards!

7/14/2006 11:43 pm  

Post a Comment

<< Home