I had a good question this past week on the podcast from one of my listeners about the value of the various draft rankings. It reminded me of a small project I had worked on last year looking at the consensus draft rankings which I think I discussed on a podcast but never really posted about. For those unfamiliar with the consensus draft, it is a massive project that has been undertaken each year by Arif Hasan where he compiles a large number of draft rankings into one big board to get an idea as to where draft analysts project the player rankings. Some teams seem to completely discount all the mock draft stuff while others say it can be useful so I wanted to go back and look at how the consensus board performed in terms of 2nd contract value and in accuracy over a small timeframe which for these purposes is from 2015 to 2018.
Here is the average outcome of the two drafts over the four years broken down by the pick range in the draft and consensus draft:
| Pick Range | Actual Pick 2nd Contract | Consensus Pick 2nd Contract | Difference |
| 1 to 10 | $16,378,009 | $14,580,975 | $1,797,034 |
| 11 to 20 | $12,449,592 | $9,432,415 | $3,017,177 |
| 21 to 32 | $7,424,359 | $11,011,351 | ($3,586,991) |
| 33 to 47 | $8,102,223 | $7,942,507 | $159,717 |
| 48 to 63 | $6,115,502 | $4,922,047 | $1,193,455 |
| 64 to 79 | $4,420,657 | $5,611,106 | ($1,190,449) |
| 80 to 95 | $5,500,377 | $3,846,620 | $1,653,757 |
| 96 to 111 | $3,366,245 | $3,817,416 | ($451,171) |
| 112 to 127 | $3,172,717 | $2,645,682 | $527,035 |
| 128 to 143 | $3,673,474 | $4,015,333 | ($341,859) |
| 144 to 159 | $2,870,332 | $3,683,713 | ($813,381) |
| 160 to 175 | $2,460,984 | $2,325,162 | $135,823 |
| 176 to 191 | $1,330,456 | $1,481,068 | ($150,612) |
| 192 to 207 | $2,147,430 | $2,161,168 | ($13,738) |
| 208 to 223 | $894,750 | $2,267,029 | ($1,372,279) |
| 224 to 239 | $1,153,906 | $1,846,243 | ($692,337) |
| 240 to 255 | $1,269,091 | $1,496,979 | ($227,888) |
| Con. UDFA/Drafted | $2,513,431 | ($2,513,431) | |
| Unlisted Con. | $1,726,103 | ($1,726,103) | |
The top of the draft skews in favor of the actual draft which looks like it would largely be from the NFL being more bullish on players the consensus board may have ranked in the 20’s being pushed forward a bit and a similar outcome likely occurring near the top of the 2nd round. While I am sure this is a pretty random outcome the players actually selected from 208 to 223 on average have been pretty bad which led to a big gain in the consensus board there. Somewhat interesting are the players who would be considered high level UDFA’s by the consensus board who the NFL actually drafted (I did not calculate those who went undrafted) averaging $2.5 million per year on a 2nd contract so there are clearly some good players that land between 256ish and 300. Also a solid result is that the players who were not listed at all on the consensus board but were drafted only averaged $1.7M per year on their 2nd NFL contract. While that is better than the performance of the late rounds it is fair to say that those the consensus have missed are, on average, not turning into superstar players and generally perform at the level of a 6th round draft pick.
The second thing I looked at was the differential between the consensus board and the actual draft. For this I broke the differences up into ranges. A 10 spot change would be a near perfect hit with the player’s draft position coming within 10 picks of the consensus ranking. A “jump” would indicate that the NFL saw that player as worth a higher pick than the consensus while a “fall” would indicate some range lower than the consensus.
| Differential Range | Pct of Picks | Avg. 2nd Contract Value |
| 10 spot change | 18.9% | $8,039,019 |
| Not listed | 15.4% | $1,715,588 |
| 25 to 50 jump | 11.7% | $3,410,558 |
| 10 to 25 spot jump | 10.9% | $6,431,445 |
| 10 to 25 spot fall | 10.1% | $5,644,023 |
| 25 to 50 fall | 9.7% | $2,801,835 |
| 50 to 75 jump | 6.6% | $3,535,751 |
| 100 to 200 jump | 4.2% | $3,947,371 |
| 50 to 75 fall | 3.6% | $2,991,946 |
| 75 to 100 jump | 3.5% | $2,907,472 |
| 75 to 100 fall | 2.9% | $1,814,892 |
| 100 to 200 fall | 2.4% | $693,604 |
Nearly 19% of the picks in this timeframe fell within 10 spots of the consensus ranking which is pretty good. Players that fell or rose between 10 and 25 spots were both about equal at 10.9 and 10.1%.That basically puts the consensus board at around 40% of the players who are likely to be selected within 1 round of the consensus ranking. The bigger change is in the 25 to 50 spot changes. Nearly 12% of the draft picks rise by at least 25 picks from the consensus board which is a pretty high number. Just under 10% fall by that much. Still that is probably going to lead to a round and a half change which isn’t too bad. That would put us around 60%.
The number of players who are big risers or droppers is not huge. About 10% have a difference of 50 to 75 selection spots and 6% are between 75 and 100 spots. 6.5% is the number of massive changes over 100 spots. The players unlisted by the consensus would be the big miss at 15.4% of the actual draft. Still that means nearly 85% of the players drafted were on the consensus boards which is pretty solid with 60% of those being within a round.
I think one thing worth noting here is the salary outcomes of those who fall in the draft. While the draft does influence salary outcomes, the results would certainly indicate that those draft day fallers who will get touted as “great value” often are not performing at that level. Player’s who fell over 100 spots from the consensus had an average 2nd salary of just $694K which basically is the minimum during that timeframe. Players who fell 75 slots had just $1.8 million and the 25 spot fall worked out to $2.8 million. These were three of the four worth performers on the board. The other one was the $1.7M for those who were unlisted on the consensus. That tells me that while it’s a miss to not have those player’s listed, on average they are not standouts in any way shape or form.
On the other side of the spectrum the draft “reaches” have performed better than the ones we consider great value. Those who jump over 100 spots have an average 2nd salary of nearly $4 million. The 50 spot jump is at $3.5 million. So the NFL is probably getting something right there over where the consensus sees the draft.
Finally I wanted to look at these same hit rates based on the consensus board rankings just to get the best idea for where we find the risers and fallers.
| Pick Range | 10 spot change | 10 to 25 spot jump | 10 to 25 spot fall | 25 to 50 jump | 25 to 50 fall | 50 to 75 jump | 50 to 75 fall | 75 to 100 jump | 75 to 100 fall | 100 to 200 jump | 100 to 200 fall |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 to 10 | 80.0% | 0.0% | 15.0% | 0.0% | 2.5% | 0.0% | 2.5% | 0.0% | 0.0% | 0.0% | 0.0% |
| 11 to 20 | 53.8% | 10.3% | 17.9% | 0.0% | 15.4% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 2.6% |
| 21 to 32 | 47.9% | 22.9% | 14.6% | 0.0% | 10.4% | 0.0% | 0.0% | 0.0% | 4.2% | 0.0% | 0.0% |
| 33 to 47 | 39.1% | 20.3% | 17.2% | 6.3% | 10.9% | 0.0% | 3.1% | 0.0% | 0.0% | 0.0% | 3.1% |
| 48 to 63 | 22.2% | 20.6% | 25.4% | 11.1% | 12.7% | 0.0% | 3.2% | 0.0% | 1.6% | 0.0% | 3.2% |
| 64 to 79 | 26.6% | 10.9% | 21.9% | 15.6% | 6.3% | 0.0% | 3.1% | 0.0% | 7.8% | 0.0% | 7.8% |
| 80 to 95 | 17.7% | 19.4% | 9.7% | 16.1% | 14.5% | 3.2% | 6.5% | 0.0% | 8.1% | 0.0% | 4.8% |
| 96 to 111 | 21.7% | 10.0% | 10.0% | 21.7% | 15.0% | 1.7% | 8.3% | 1.7% | 3.3% | 0.0% | 6.7% |
| 112 to 127 | 11.9% | 13.6% | 11.9% | 20.3% | 16.9% | 5.1% | 5.1% | 0.0% | 8.5% | 0.0% | 6.8% |
| 128 to 143 | 13.0% | 13.0% | 11.1% | 16.7% | 16.7% | 5.6% | 9.3% | 5.6% | 5.6% | 0.0% | 3.7% |
| 144 to 159 | 15.4% | 17.3% | 5.8% | 13.5% | 9.6% | 15.4% | 9.6% | 1.9% | 7.7% | 1.9% | 1.9% |
| 160 to 175 | 4.8% | 14.3% | 7.1% | 16.7% | 19.0% | 9.5% | 9.5% | 7.1% | 7.1% | 4.8% | 0.0% |
| 176 to 191 | 7.9% | 5.3% | 7.9% | 21.1% | 23.7% | 21.1% | 5.3% | 5.3% | 0.0% | 2.6% | 0.0% |
| 192 to 207 | 11.4% | 8.6% | 8.6% | 17.1% | 11.4% | 5.7% | 5.7% | 22.9% | 0.0% | 8.6% | 0.0% |
| 208 to 223 | 3.4% | 13.8% | 10.3% | 10.3% | 10.3% | 20.7% | 0.0% | 10.3% | 0.0% | 20.7% | 0.0% |
| 224 to 239 | 2.9% | 5.7% | 5.7% | 20.0% | 2.9% | 28.6% | 0.0% | 20.0% | 0.0% | 14.3% | 0.0% |
| 240 to 255 | 13.6% | 4.5% | 0.0% | 18.2% | 0.0% | 31.8% | 0.0% | 9.1% | 0.0% | 22.7% | 0.0% |
| UDFA | 0.0% | 7.1% | 0.0% | 21.4% | 0.0% | 23.2% | 0.0% | 12.5% | 0.0% | 35.7% | 0.0% |
I think looking at a change of no more than 25 spots we can see that the most accurate consensus picks are the top 10 (95% hit rate), picks 11 to 32 (mid 80%), top of the 2nd round (77%), and bottom of the 2nd round (68%). Obviously our biggest risers will come from those ranked by the consensus at 6th round picks and later.
The big misses come in the mid round projections. Picks 64 to 79 had just under 16% with a 75 slot or greater fall. Pick 80 to 95 was at 13% while 96 to 111 was 10% and 112 to 127 was at 15%. This is probably where we have our biggest disconnect so it is probably fair to say that the 1st and 2nd round projections are pretty good with the top of the 3rd round being decent but with a much higher variance and then things get a little chaotic in the 2nd half of the 3rd round through the 4th round. Basically we are going to find a number of players who jump up from those late consensus projections to replace these players. It would probably be worthwhile to do this on a positional basis but perhaps that will be something to look at next year.
Overall, regardless of what extra information teams have and all the extra prep work that goes into the draft, the “amateur” draft process is pretty solid at identifying most players who will be drafted and producing overall outcomes that are not that far off from the actual NFL results despite having less resources on hand and should be capable of producing some pretty accurate simulations of draft outcomes.