Can We Pick Deep QB Classes in Advance?

Comparing QB draft projections to outcomes from 2011 through 2022

I am afraid that my last two articles might have caused undue levels of angst among Hogs Haven readers as they look forward to the Commanders taking a shot at a starting QB with the second overall pick in 2024 draft.

The first article pointed out that no team picking second or third overall in the decade from 2012 through 2021 had hit on a starting QB. I followed that up by showing that, had the teams picking second or third chosen other QBs who were taken a few picks later, the hit rate could have risen, but only as high as 38%.

Quite a few readers pointed out that past draft results have no bearing on what Adam Peters will do when the Commanders are on the clock on the evening of April 25^th. In particularly, Peters will be picking in an unusually deep draft class with somewhere between 4 and 6 QBs expected to be drafted in the first round.

That sounds reassuring, except for one thing. How certain are we that it’s a deep QB class?

NFL teams don’t readily share their draft boards, so the information we are relying on must come from one of two sources. Most of us are relying on draft experts like Chris Simms, Mel Kiper and Lance Zierlein. A few among us might be doing their own scouting.

I can’t speak to the latter, because Hogs Havens’ resident draft experts have steadfastly resisted my best attempts to get them to commit their predictions to writing in my annual Bold Draft Predictions Challenge. As to the former, do we know whether the media draft experts are any good at predicting when it’s a deep or a not so deep draft class for QBs?

That seems like a pretty crucial information gap, considering how much time we all spend debating which QBs are guaranteed day 1 starters and which ones are sure to be busts. To help my fellow Hogs Haven readers out, I took it upon myself to figure out just how much stock we should put in draft experts’ predictions that 2024 will be a deep QB draft class.

To get to the bottom of this pressing question, I compared media draft expert projections to actual draft outcomes for every draft class from 2011, when the rookie wage scale came into effect, to 2022, which is the most recent draft class for which we can get a handle on how the QBs turned out.

Using data from 12 draft classes I asked how often the media analysts correctly identified deep QB classes ahead of time and how often they missed them. To make it even more fun, I compared the ability of the media experts to pick the deep and not so deep QB classes to that of the NFL teams making picks.

Set Number: X157561 TK1 R2 F155

Assessing QB Draft Classes from 2011 through 2022

QB draft classes were assessed as deep or not using the following criteria.

Media Expectations – How the media saw the draft class in the lead up to each draft was assessed using a Consensus Draft Board. The Consensus Draft Board concept was perfected by Arif Hasan, originally writing for Vikings fan site Zone Coverage. In 2019, Hasan’s creation moved to The Athletic, where it has outlasted its originator. The first version contained average prospect rankings from over 40 well known draft analysts, and today it draws on over 80 big boards. It is one of the best predictors of where players will get picked in the actual draft.

In this analysis, Consensus Rank is used to estimate where the media expected a player to go. The mapping between prospect rank and expected draft position is imprecise, because QBs tend to get picked higher than their rank due to their extreme position value. This analysis assumes that a QB ranked in the top 16 prospects was expected to be one of the first few players off the board, and QBs ranked in the top 32 carried a “first round” grade. A player ranked after 32 had a “first or second round” expectation.

Prior to 2014, things were more challenging. In 2012, Walterfootball published a Consensus Board analogous to Hasan’s, so I used that. In 2013, I rolled my own by scouring the internet for remnant big boards and calculating average player ranks.

I concluded there was an expectation of a Deep QB Class if the Consensus Draft Board had a higher than average number of QBs ranked in the top 32.

NFL Expectations – this is a lot more straightforward to assess. The pick number where a QB was drafted tells us what at least one NFL team expected of him. QBs picked in the top 15 are expected to start early, with above-average to elite expectations. Teams picking QBs from around pick 20 through 50 are taking gambles on prospects with starting upside but bigger questions. After that, it’s anyone’s guess.

I concluded that NFL teams expected a Deep QB Class if a higher than average number of QBs were drafted in the first round.

Long Term Starters – this is tricky because I had to sort out some flash-in-the-pans and late bloomers. My judgement is weighted toward the early part of a QB’s career, which more often than not aligns with peak productivity. Therefore, I used number of Starting Contracts following the rookie contract, taking into account reasons for premature termination, as my primary indicator of whether a QB was a long-term starter or not. Singing a single long-term extension following the rookie contract usually indicates a long term starter. Very few QBs make it to a second extension. But a team quickly realising they made a mistake due to play on the field (as opposed to injury or off-the-field issues) could indicate that he wasn’t.

There are some tough judgement calls, such as Geno Smith, Jimmy Garoppolo, Colin Kaepernick and maybe even Baker Mayfield. Feel free to flag any of the calls that you think I got wrong in the comments.

Draft Outcomes – the assessment of draft outcomes was based on how many long term starters were turned up in a draft class. A draft class was deemed “deep” if it contained more than the average number of long-term starters. I have also listed a few indicators of how good the drafted QBs were to provide some depth beyond my minimal criterion for a successful QB draft. These include:

• Number of Seasons Starting – this is based on the number of seasons as the primary starter for the team, rather than expectations at the start of the season. If a player was expected to start, but missed time to injury and played fewer games than his backup, it was not counted as a starting season.
• Playoff Success – this is indicated by the Playoff Finishes throughout a QB’s career. The Playoff Finish is the highest playoff game that the QB won, starting for his team. No credit was given for playoff losses or for playoff games won by another QB on the team (sorry Carson).
• Peak QBR – the peak Total QBR achieved in any season as a primary starter in the QB’s career. Total QBR is an advanced metric, developed by ESPN, which measures total efficiency at advancing their team to scoring position on a per play basis, taking into account passing rushing and negative plays, with weighting to prevent inflation from garbage time. Values are scaled from 0 to 100, with 50 indicating average play. Values above 60 typically land in the top-12, and anything above 70 is heading toward elite territory. Anything below 40 is essentially a fail. It should not be confused with the NFL official stat Passer Rating, which is based on a few simple passing metrics and ranges from 0 to 158.3.
• Honors – I listed all the usual indicators of premium QB play including Pro Bowl and All-Pro team nominations, MVP, Super Bowl MVP, Rookie of the Year and Comeback Player of the Year.

QB Draft Classes 2011 – 2021: The Deep and the Shallow

Media Expectation: 2 first rounders, one borderline

NFL Expectation: 4 first rounders

Draft Outcome: 3 long-term starters, 1 premium LTS in the first round, 2 LTS in round 2

The consensus first QB off the board ahead of this draft was Blaine Gabbert (Missouri), who ticked all the boxes as a perennial All-Pro, to some respect evaluators. Cam Newton was the ultimate boom or bust QB, getting near to “generational” reviews for his athleticism and arm talent, but also raising concerns about his readiness for the pro level. The general zeitgeist was that there was no “sure thing” in this draft, but at least two prospects worthy of a first round pick. Riverboat Ron’s Panthers took the gamble on Newton and hit the jackpot, while the Jaguars crapped out with Gabbert.

Apparently, head coach Jack Del Rio was not consulted about the pick:

I had no idea we were going to draft Blaine Gabbert. No idea. In fact, I left to go get something to eat because our pick wasn’t for much longer in the draft. I go and then I’m sitting there filling my plate thinking, ‘Oh great, we’ve got a couple more hours until we pick.’ Then I see, ‘The Jaguars are on the clock.’ I’m like, ‘What the blank is going on?’

I walk into the draft room, and I could see it on the faces of the people in the room. They knew how uncomfortable that was, how wrong that was.

The Titans and Vikings picked Jake Locker and Christian Ponder earlier than the analysts expected, with limited success. Day 2 picks Andy Dalton and Colin Kaepernick were both extended as starters by the teams that drafted them. Kaepernick reached greater heights as an NFL starter, but his career ended prematurely for off-the field reasons.

Media Expectation: 3 first rounders

NFL Expectation: 4 first rounders

Draft Outcome: 4 long-term starters, 1 high-end backup/Super Bowl MVP, 1 all-time bust

One of the deepest QB drafts in the modern era was not really viewed that way at the time. There were two clear elite talents, followed by fairly usual depth after the top of the first round.

Andrew Luck was a consensus “generational” draft prospect, in the modern sense of the term (best player at his position in a decade). While he might not have lived up to the generational tag in the NFL, he met expectations of his draft status as a premium starter until his career was cut short by injuries. Dan Snyder’s Redskins traded a record haul of draft capital to move up and draft the consensus second best player in the draft, Robert Griffin III, who delivered an electrifying rookie season before going on to become one of the greatest busts in NFL draft history and a cautionary tale against betting the future of the franchise on a single draft pick.

The first round produced another long term starter, Ryan Tannehill, who was still available at the Commanders’ original pick. Tannehill is widely viewed as a mediocre starter, despite having had greater success in the NFL than most QBs picked in the first round.

The real depth of this draft was in the middle and later rounds. Russell Wilson has gone on to become one of the greatest value picks in NFL draft history, leading the Seahawks to an amazing run of 8 playoff appearances in 10 seasons as the starter, with Super Bowl wind and another Conference Championship finish. Wilson was widely recognized as a stellar passer, but dropped in the rankings for no other reason than lack of prototypical size. Bleacher Report’s Matt Miller gave the Seahawks’ draft a D for wasting a third round pick on a QB when they had bigger needs elsewhere.

Whatever you think about his play on the field, Kirk Cousins has become the undisputed master of extracting long-term starting money from NFL teams. Nick Foles deserves honorable mention as a high-end backup/marginal starter with one Pro Bowl nod and a well-deserved Super Bowl MVP trophy.

Deciphering how many long-term starting contracts the later round QBs signed is a job better left to the cap/contract experts. Wilson and Cousins are undoubtedly long term starters. Foles inked a four-year $22m APY contract with the Bears in 2020, but only played 2 years. Not enough to qualify, despite the Super Bowl pedigree.

Media Expectation: 1 first rounder

NFL Expectation: 1 first rounder

Draft Outcome: 0 long-term starters (despite 1 late-career comeback story)

There was a wide range of opinions on Smith, with some rating him as a first round pick and others projecting him to the late first or second round. Nobody predicted that he would languish on a dysfunctional Jets team, roam the NFL as a journeyman, and eventually make good on a series of one year deals as Russell Wilson’s backup, 8 years later. He was not a long term starter for the team that drafted him or the next two teams he played for.

The Bills’ picking Manuel in the first round was a surprise as most analysts saw him as a Day 2 pick.

Media Expectation: 4 first rounders

NFL Expectation: 3 first rounders

Draft Outcome: 2 long-term starters from Day 2

It might come as a surprise now, but 2014 was seen as a deep QB class, with three quality options for the top of first round, and some more intriguing options toward the end of Day 1 and into Day 2. Bridgewater profiled as a Day 1 starter, while Johnny Manziel added excitement as a high-risk, high-reward option, and Derek Carr was seen as having the best arm in the draft.

The two hits in this draft came from the second round. Carr went later than expected, apparently over concerns about his pocket presence and whether he could adapt to an NFL offense. Carr has now played 9 seasons and been to 4 Pro Bowls as a starter for the Raiders and the Saints. Garoppolo was drafted by the Patriots to back up Tom Brady, and traded 3 seasons later to the 49ers where he signed 5 year, $27.5m extension in 2019. His time with the 49ers was impacted by injury, but he did lead them to a Super Bowl and a Divisional playoff win. He was released in 2023 and signed a 3 year, $24m low-end starting deal, which was terminated after the first season.

Manziel flamed out due to poor lifestyle choices. Bridgewater’s early NFL career was impacted by injury, but he has managed to stay on in the league as a journeyman.

Media Expectation: 2 first rounders

NFL Expectation: 2 first rounders

Draft Outcome: 0 long-term starters

Media analysts and NFL teams saw this class the same way: two elite prospects for the top of round 1 with nothing behind them. The were right about the second part.

Both QBs were allowed to walk by the teams that drafted them and have made a living on short term contracts as high-end backups/marginal starters.

Media Expectation: 3 first rounders

NFL Expectation: 3 first rounders

Draft Outcome: 2 long-term starters, 1 from round 1, 1 from Day 3

For the second year in a row, media analysts and NFL teams picking in the first round saw eye to eye.

Goff and Wentz were both seen as early first round picks. They were followed by a distant second tier including Connor Cook, Paxton Lynch, Christian Hackenberg who were expected to go from the late first round to Day 2.

Goff has succeeded as better than average starting QB, despite being traded by his drafting team.

Aside from Goff, the other quality starter to come out of this draft was Dak Prescott, who was generally viewed as a Day 3 pick and did not crack the top 100 Consensus Board for this draft. Walterfootball rated Prescott higher than most.

Despite some early success, with help from backup Nick Foles, Carson Wentz failed to catch on as a team’s long-term starter. After the Eagles extended and then traded him to the Colts, he signed 4-year mid-level starting deals with two teams, each of which recognized their mistake after one season and moved on.

Media Expectation: 3 mid-to-late first rounders

NFL Expectation: 3 earlyish first rounders

Draft Outcome: 1 generational talent, 2 long-term starters

In hindsight, this draft featured the greatest QB talent in the draft for over a decade and another premium starter. It was not seen that way at the time.

UNC’s Mitch Trubisky was a one year starter who profiled as having potential to excel as a pocket passer, despite having played in a spread offense in college. Trubisky was essentially tied with two-time Heisman finalist Deshaun Watson. Watson wowed analysts with his mobility and leadership, but concerns were raised about throwing accuracy and whether he was a system QB.

The dark horse in this draft was Patrick Mahomes, who played in Kliff Kingsbury’s Air Raid offense at Texas Tech. Opinions were mixed on Mahomes with some analysts like Lance Zierlein recognizing his potential to become a plus starter with further development; while Rivals national recruiting director famously tweeted that the Chiefs made the “dumbest move in the draft” by trading up for Mahomes with Watson still available. As best I can tell, the tweet was later taken down.

It was not a deep draft, but it had a higher peak than anyone but the Chiefs’ front office expected.

Media Expectation: 5 first rounders

NFL Expectation: 5 first rounders

Draft Outcome: 3 long term starters

The 2018 draft was seen as one of the deep QB classes in years by media analysts and NFL teams alike. In fact, it tied with 1949 for the most QBs selected in the top 10 picks in draft history.

The field was led by UCLA QB Josh Rosen, who profiled as an NFL-ready pocket passer, with some minor qualms about his leadership and technique issues to clean up. Darnold was close behind, with projections as a top-5, or even top-3 pick.

Heisman winner, Baker Mayfield rounded out the lead pack. Mayfield was more polarizing than the QBs ranked ahead of him, with concerns about his lack of prototypical size, wild off-the field behavior and college system detracting from his otherwise stellar profile as a highly accurate passer and dynamic playmaker.

The second tier of QBs in this draft comprised mobile QBs Lamar Jackson and Wyoming’s Josh Allen. Jackson was recognized as an elite playmaker at the college level, but was also raw and inconsistent as a passer. Two NFL MVP awards later, that’s still how a lot of people describe him. Allen was seen as a boom or bust prospect, dripping with raw natural talent, but needing intensive coaching to succeed in the NFL. As prophesied by ESPN’s Dan Graziano, NFL teams saw more to like in Allen than media experts and fans. The Buffalo Bills were rewarded for taking him earlier than the pundits expected.

The first round of this draft reflects the pattern of drafts throughout the decade. Teams picking first overall and in the second-tier of prospects hit gold, while the team taking the second QB off the board struck out.

Media Expectation: 2 first rounders

NFL Expectation: 3 first rounders

Draft Outcome: 2 long term starters, for now

A bit of mythology has developed about this draft. Kyler Murray was far from a consensus first overall pick in the eyes of draft analysts and reporters. He was a polarizing prospect. While everyone recognized he was an extremely talented exemplar of the by then in-vogue mobile QB, there were also major concerns about his small stature, personality and limited amount of quality game tape.

While many analysts rated Murray as a top of the draft prospect, others had him graded much lower. Walterfootball’s Charlie Campbell reported that NFL teams had him rated in the second round because he would require an offense to be built around him due to his size, and because of durability concerns.

What may have confused matters is that reports emerged early in the draft process that the Cardinals’ Kliff Kingsbury was high on Murray and that it was pretty much a done deal he would be going first overall to the Cardinals from the point that he decided not to play baseball and declared for the NFL draft.

After Murray, there was little for teams to fall in love with in 2019. Ohio State’s Dwayne Haskins was generally projected as an early starter. When the Redskins picked him 15^th overall, it raised few eyebrows because that’s where he was expected to. During the 2019 season, reports began to emerge that head coach Jay Gruden had not wanted to draft Haskins. Gruden himself later confirmed the owner Dan Snyder made the pick over the strenuous objection of his front office executives.

The surprise pick in this draft was the Giants picking Daniel Jones sixth overall, since he had been expected to be a second round pick. The move was not universally embraced by Giants fans:

Jones signed a 4-year, $40m APY extension with the Giants before the 2023 season, but it is being reported that they might be having second thoughts. In hindsight, a better option for either NFC East team might have been Gardner Minshew in the 6^th round.

Media Expectation: 3 first rounders

NFL Expectation: 4 first rounders

Draft Outcome: 5 quality starters

Nobody seems to have expected the 2020 draft to be the QB draft of the century. In fact, if the five QBs listed above continue on their present trajectories, it could turn out to be largest crop of premium starting QBs ever in one draft class, surpassing the class of 1983, which produced four.

Four of the first five QBs off the board have been to at least one Pro Bowl and two have led their teams to championship round playoffs. Jordan Love had to serve a two year apprenticeship backing up Aaron Rodgers, as is the Green Bay way. When he finally got the opportunity, he played at a high level, leading the Packers to a Wild Card Playoff win in his first season starting.

Love arguably outplayed the analysts’ expectations and Jalen Hurts certainly did. Both Love and Tua should be extended this season unless something unexpected happens.

Media Expectation: 5 first rounders

NFL Expectation: 5 first rounders

Draft Outcome: 1 long term starter expected

After missing the deepest QB draft class in decades, the analysts seemed to overcompensate in 2021, with 5 prospects ranked in the top 27 of the Consensus Board, headed by Trevor Lawrence who was billed as the best QB draft prospect since Andrew Luck. This was the year when NFL teams began to panic about the escalation of starting QB salaries, so it was little surprise that they followed suit, picking 5 QBs in the top 15.

Both analysts and NFL teams seemed to agree that 2021 was an historically deep draft class. The haul of 5 QBs picked in the top 15 tied with 1949 for the most in draft history. That is where the similarities with the 1949 draft ended. That draft produced Hall of Fame QBs Norm Van Brocklin (pick 37), Jim Finks (pick 116), and George Blanda (pick 119).

As of two weeks ago, Trevor Lawrence is the only one of the five first round QBs who is still starting for the team that drafted him. He and Wilson are the only QBs who are still with their drafting teams. Lawrence has played like a quality starting QB in his second and third seasons, but has yet to live up to the expectations of being a “generational talent” which followed him through the draft process. He still has plenty of time.

Media Expectation: 1 marginal first rounder

NFL Expectation: 1 mid-first rounder

Draft Outcome: 1 high-end late round starter

Ordinarily, two seasons is not long enough to judge outcomes of a QB class. But 2022 was an unusual QB class in many ways.

Both media analysts and NFL teams judged 2022 to be an exceptionally weak QB class. Only Kenny Pickett was projected to have a chance to be picked in the first round. Unsurprisingly, he was picked in the second half of the first round. After a promising rookie season, Pickett took a step back in 2023. A few weeks ago, he was traded in a player-pick swap with Philadelphia, where he will back up Jalen Hurts.

With Desmond Ridder being traded to the Cardinals and Sam Howell to the Seahawks, it seems unlikely that any QB drafted before the last pick in the 2022 draft will get to start for an NFL team any time soon.

The last player picked in the 2022 draft, Brock Purdy, has quickly ascended to the elite ranks, leading the 49ers to Divisional and Conference Championship playoff wins in his first two seasons in the league. In 2023, he took over from departed starter Jimmy Garoppolo and earned a Pro Bowl nomination, while leading all NFL QBs in total QBR.

Adam Peters had this to say about the pick:

How Well Did the Experts and NFL Teams Predict the Deep QB Classes?

Now that we’ve been through each draft class in excruciating detail, we are getting close to some answers. But before we get there, we have one more question to answer: How many long term starters make up a deep QB class?

The following table summarizes the data from each of the 12 draft classes, including the expectations of media draft experts and NFL teams, as well as the numbers of long term starters that were picked.

The average (median) draft class produced around two long term starters, with one from the first round and one from the later rounds about every other year. That brings us to the answer of our first question. (Don’t worry about the numbers not adding up. Using medians can lead to rounding errors). A deep QB draft class is one that produces a higher than average number of starters. So, three or more long-term starters is a deep class.

Interestingly, if we take the number of players drafted in the first round to indicate the number of expected starters, both the media experts and the NFL teams seem to predict a deep class in the average year. That is not overly surprising because not all projected starters pan out. That is just the nature of the draft.

Now, to answer the question I started out with. How good are the experts at picking when it’s a deep or a weak QB class? The following table summarizes the results, showing when the media and NFL teams expected deep or weak QB classes as well as the actual deep and weak QB classes. I deemed an expectation of a deep or weak class to be when the prognosticators’ expectations were higher (>3 long-term starters) or lower (<3 long-term starters) than average. A deep or weak class is when the actual number or long-term starters was higher (>2) or lower (<2) than average. Numbers of expected and actual long-term starters are in parentheses.

Deep QB Drafts

The media experts expected deep QB classes in 2014, 2018, and 2021. They were right about 2018, and wrong about the other two classes. They also failed to anticipate the two deepest QB classes in the period examined, including 2020, which could possibly become the deepest QB draft in history. They failed to pick 2012 as a deep class, for the same reason everyone else did. Two of the long-term starters were picked in the third and fourth rounds. Late round starting QBs are unpredictable.

Not surprisingly, NFL teams were better prognosticators than the draft experts. NFL teams anticipated all three of the deep QB classes. They overshot the mark in 2021, just like everyone else.

Weak QB Drafts

The media experts were better at anticipating weak QB classes. They correctly identified three of the four weak QB classes. They correctly picked 2022, 2015 and 2013. But they overshot the mark by also calling weak classes in 2019, which was normal, and 2011, which turned out to be deeper than average.

To be fair, 2011 was a particularly challenging draft to get right, because the highest rated QB (Blaine Gabbert) was a bust, along with two of the other three QBs drafted in the first round; And nobody could have predicted the success of Andy Dalton and Colin Kaepernick in round 2. Some readers are sure to disagree with my classification of Kaepernick as a long-term starter, but he was extended long-term in 2014 on the back of a higher peak than most starting QBs ever achieve.

Once again, it should surprise no one that the experts working for NFL teams were better at predicting weak classes than the media pundits. NFL teams correctly anticipated three of the four weak QB classes. The only one that fooled them was 2021, which seems to have been a total anomaly.

TL:DR Summary

The fact that media experts rate the 2024 as a deep QB draft class means precious little. They have not been very good at calling QB classes in advance throughout the modern draft era, starting in 2011. The sample size is obviously too small to calculate meaningful hit and miss rates, but their results don’t appear to be much better than a monkey throwing darts.

NFL teams have been better at anticipating deep and weak QB classes. Unfortunately, though, we only get to find out how they view the QB class after the draft gets underway. Sadly, they are no help to us in figuring out what Adam Peters should do on opening night of the draft.