As seen by the Buffalo Sabres never-ending rebuild, the ability to find future talent at the NHL Draft can affect a franchise for years. This is why teams should always be looking for future market inefficiencies. They should look to correct for past biases at the NHL Draft. Various kinds of biases have been shown to causes groups of prospects to be undervalued. For example, I previously looked at age and showed forwards with later birthdates tended to be undervalued. In a similar vein, today we will be looking at how the league which these prospects have come from affected their value. More concisely, we will see, have players from certain leagues have been undervalued at the draft table?
NHL Draft: Undervalued Leagues
Evaluating Drafted Players
First and foremost, we must assign an “expected value” to each draft pick. If we don’t, we may conclude certain leagues are “undervalued” when in reality they just produce higher draft picks.
So, how will we define the expected value of draft picks? Well, let’s start with “value”. We will need to define the NHL player’s contributions to their teams in some way. Then we can look at how that value changes with draft position, on average. We will be using what I call “SPAR Index” to evaluate player output. This simply takes each player’s Standing Points Above Replacement (SPAR, found at EvolvingHockey.com), and their Expected Standing Points Above Replacement, then average the two of them.
From there the SPAR Index has been prorated over a full season. This means the values are per 82 games for skaters and per 60 games for goalies. This was to account for the uneven playing field in the data. I am using the value players produced in the seven seasons after the players draft as their output. This is because teams only retain a player’s rights for seven seasons after they drafted them.
A note on sample sizes
Ideally, we would have the data far enough back that I could use the player’s first seven seasons in the NHL. After all, this is when the team’s rights to a player expire, but that would greatly reduce the sample size for this analysis. So to increase the sample I have prorated the output to a per-season basis. Prorating data can give players who performed well in small samples extreme values, so the SPAR Index for each player has been regressed towards replacement level up to each player’s 100th game.
The Expected Value of NHL Draft Picks
Once we have defined player output, it’s (reasonably) easy to evaluate the expected value of each draft pick. We can simply look at the trendline of how much value (SPAR Index Per Season) is generated by each draft position. For example, we can see a general trend where output is usually high for the top few picks. After those first few picks, the value falls off much more slowly towards.
In our sample of drafts from 2007-2013, the first overall pick tends to add 2-3 SPAR Index per season. (1st overall picks add 2-3 standing points above replacement per season for 7 seasons following their draft, on average). Then the values begin to fall off very quickly. By the end of the first round, the players are usually only producing a fraction of what the first few picks do. After pick 30 or so, average values decreased at a slow rate until the end of the daft.
Using this data, I fit a logarithmic regression line to the data to generate out expected draft pick values.
This way we see a classic draft pick value chart. The first picks have large differences in expected value. Then after the early first-round value begins to fall off very slowly.
Finding League Biases
With production and expected value defined, we can begin looking for league biases. The idea here is simple. If the NHL Draft was efficient, no league would tend to produce players who perform better or worse than expectations. On the contrary, if players coming from certain leagues generally perform better or worse than expectations, perhaps NHL teams are not properly valuing certain leagues.
So which leagues, if any have been over or underrated? My guess was that the CHL leagues have been overvalued, while top European leagues have been undervalued. To test my theory I graphed cumulate value vs. expected value of all skaters (because the drafting of goalies is largely random, especially in a 7-year sample). The skaters-only data shows it’s not so simple. Note that above the black line will mean that league has produced more value than expected, while the opposite is true for leagues below the line.
Testing the Hypothesis
The first part of my hypothesis appears to be correct. Intuitively you might expect teams to draft most efficiently out of the leagues they are most familiar with, but the opposite has been true. The three CHL leagues produce the most prospects, and they all fall into the same category. Each league has produced far more expected value than actual value based on the pick number where its prospects were selected. In other words, NHL teams draft CHL players far higher than they should have, on average. Since a few leagues produced most of the prospects, let’s look at the value above expected on a per pick basis instead.
Over Valued Leagues
On a per pick basis, we see the most overvalued league was the USA Hockey National Team Development Program (USDP). Players who primarily play in the USDP underperformed their draft position by 0.28 standing points above, replacement per season, on average. The sample size is reasonably small, however this reminds me of the debates before the 2019 NHL Draft. Corsica creator Emmanuel Perry (@mannyelk) was flammed on twitter because his draft model was relatively low on USDP players such as Cole Caufield. Perry defended himself by saying that production in the USDP has been overvalued historically. (Note that due to the uniqueness of the USDP, we have separated these players from the rest of the USHL). Since that has been the most overvalued team/league at the draft table, it looks like Manny was right.
The CHL
Then there are the three CHL leagues. Prospects from the Western Hockey League (WHL) were the most overvalued in our sample. They underperformed their draft position by 0.27 standing points above replacement. It’s basically splitting hairs between the WHL and USDP for the most overvalued team/league. Given the much larger sample of players coming from the WHL, this is damning evidence that WHL prospects have been overrated at the draft.
After the WHL, the next most overvalued CHL league has been the Quebec Major Junior Hockey League (QMJHL). The QMJHL is generally considered the worst of the 3 CHL leagues. And yet teams have still picked its players too high, on average.
Finally, there is the Ontario Hockey League (OHL). Often considered the best league in the CHL, it produces the most prospects of any league by far. Unfortunately, with over a 250 prospect sample size, OHL prospects have produced far less value than expected in total. And that is despite the OHL often having the best prospects on earth at the time such as Patrick Kane, Steven Stamkos, Taylor Hall, Tyler Seguin, and John Tavares.
Under Valued Leagues
On the flip side, we see not all Junior leagues were undervalued. The SuperElit, Sweden’s best junior hockey league, was the primary league for many skaters and those skaters outperformed their draft position, on average. The sample size is small enough to be skeptical, however, it seems reasonable to me that players who primarily played in Sweden’s top junior league are at least more undervalued than their peers in the CHL.
The samples are too small to say anything definitive about any other league. Even with the small sample, it’s worth discussing the Kontinental Hockey League (KHL). Prospects from the KHL are few and far between but have produced far more value than their draft position implies. This is, at least in part, selection bias.
NHLe
My plan is to use player scoring rates to produce NHLe for prospects. As a result, I had to make some judgment calls. Some players played in two leagues in their draft year. So, I would simply record their statistics from whichever league they played in most, with some exceptions. Sometimes skaters (especially defencemen) would play most of their games in the KHL, but produce 0 points. These players’ true talent is almost certainly not an NHLe of 0. So for these players, I used their statistics from whatever league they played in the next most. That means the takeaway is not as simple as, prospects from the KHL have been dramatically undervalued (although this is likely true too). The real takeaway is that the prospects who manage to primarily play and score at least 1 point in the KHL have dramatically outperformed their draft position.
It is also worth noting that the KHL group includes Alexei Cherepanov. Cherepanov was one of the best teenagers the KHL has ever seen and drafted 17th in 2008 as a result. Because the sample for the KHL is so rare, Cherepanov’s expected value represents more than 1/10 of the KHL’s expected value. Unfortunately, Cherepanov passed away in 2008. His early passing means his production of 0 SPAR Index per season says nothing about the quality of draft pick he was. If he played in the NHL, he almost certainly would have performed extremely well. This means yes, the chart shows us that prospects who primarily play in the KHL and score at least 1 point there have dramatically outperformed their draft position, and yet this is likely an understatement.
Closing Thoughts
My hypothesis was a sort of blanket statement. I thought CHL prospects have been overvalued while prospects playing in Europe have been undervalued. While this isn’t entirely true, it’s very close. Each of the CHL leagues has produced a large number of prospects. Collectively, those prospects have produced far less value than you would expect given the player’s draft position. Of the CHL leagues, the WHL was the most overvalued in our sample by a large margin.
We know it isn’t simply a problem with prospects playing in junior leagues either. Drafted prospects whose primary league in their draft year was the SuperElit in Sweden have outperformed their draft positions, on average. Then finally there is the KHL. Most people say the KHL is the second-best hockey league on earth. And yet, the teenagers who manage to play and score in the KHL have been the most undervalued. Perhaps teams are too confident that CHL scoring is projectable and too afraid of Russian KHL players will stay overseas instead of coming over to North America.
NHL Draft Photo: VANCOUVER, BRITISH COLUMBIA – JUNE 22: A general view of the 2019 NHL Draft at Rogers Arena on June 22, 2019, in Vancouver, Canada. (Photo by Rich Lam/Getty Images)
