QB Skill: A Narrative of Numbers (pt. 3)

Thanks to the nflFastR project and NFL NextGen Stats for the timely sources of data.

The point of this series has been to present an in depth examination of QB efficiency stats, so that I can properly frame what Colts fans can expect from Carson Wentz in 2021. In part 1, I showed how these stats are able to capture multiple QB skills such as accuracy, mobility, vision, etc. and therefore are good measures for Wentz’s abilities.

In part 2, I demonstrated that while QB efficiency is impacted by the rest of the team and coaching, it is still primarily a result of QB performance and so, what Wentz did on the Eagles will, in large part, carry over to what he does with the Colts.

In this segment, I will address an objection I often hear, which is that an efficiency stat can’t tell the whole story. And to those people I say . . . well . . . yeah . . . you’re right.

For example, EPA is an excellent measure of the true value of a passing offense, but it does not tell you how that value was achieved. It doesn’t say if a QB is a good deep ball passer or has poor accuracy etc. For a deeper understanding of a QB’s value, you have to look at a suite of stats.

With the right stats, you can get a very good picture not only of how good a QB is but what their strengths and weakness are as well as their style of play. You can even get a decent glimpse into the future by seeing if their performance is supportable long term. I’ll demonstrate this by showing some of the stats I compile to judge a QB’s overall performance and I’ll use Jacoby Brissett as an example.

After the 7th game of the 2019 season, Brissett had led the Colts to a 5-2 record and had the 10th highest scoring offense in the league(1). For a backup, who just a few weeks prior to the start of the season didn’t know he would be a week 1 starter, that is pretty impressive and you would think the entire fanbase would be ecstatic. However, many fans, including myself, were skeptical of his abilities and while I can’t speak for anyone else, the following is a good explanation of why I was certain that his success was going to be short-lived.

EPA Efficiency

All of the following graphs will show metrics for the 32 QBs with the most pass attempts through week 8 of that year and how those measures were distributed around the league average (the bold black line). Each dot represents a QB and Brissett is highlighted along with his rank (you can ignore the color). His value for that metric is displayed at the bottom chart. The graphs are mostly empty because I am going to highlight only a few stats at a time.

The stats shown above are Passer Success Rating (psr) and Epa per Dropback (epa/d), where Jacoby ranked 11th and 12th respectively, which puts him at the edge of the upper third of QBs. At the beginning of the season, I would have bet a large sum of money that half-way through the year he wouldn’t have been anywhere near that efficient.

Clearly, he had been successful, but other stats tell a more complete story.

The Colts were a run first team that had faced easy passing defenses

Opponent Passing Defense (opd) is the epa efficiency that opponent defenses have allowed QBs to accumulate. This shows that the defenses that Brissett had faced had given up some of the highest numbers in the league (0.15 epa/d) to opponent QBs. So, not only did I take his good efficiency with a grain of salt, I also expected future opponents to be harder and thus bring his production down.

It was clear the Colts weren’t relying on Brissett to carry the team as Early Down Passing (ed%) showed the Colts offense had the 3rd lowest % of passing plays on early downs(2) and their Weighted Rushing Success Rate (wrsr) showed a slightly better than average run game through the first 7 games, both of which are factors which can boost passing efficiency.

He was indecisive and couldn’t find open receivers downfield

Time To Decide (ttd) is like time to throw, but includes time to sack and time to scramble as well. Jacoby held the ball the 7th longest of any QB, which would make sense if he was waiting for long plays to develop. However, his Average Depth of Target (adot) only ranked 25th, so he wasn’t throwing the ball far (7.1 air yards per attempt). The length of his average completion ranked even lower (28th ay/c).

This is really the worst of both worlds. For example, Deshaun Watson had the 9th slowest decision time, but that was due to the 8th longest pass attempts. Or you can go the other way like 2020 Rivers, who had the 25th shortest pass attempts, but the 2nd quickest decision time of any QB.

A QB that takes a long time to decide and then throws short either can’t find open receivers downfield and/or is hesitant to throw to them, opting for check downs.

He had good protection and could be successful downfield, but was too cautious

About 36% of Brissett’s dropbacks were under pressure (pr%), which was basically league average. Relative to how long his decision time was, that means the O-line was giving him plenty of protection and only further illustrates his indecisiveness.

Even with all of that time, he only threw passes of 20 or more air yards on less than 8% of his attempts (20+), the 6th lowest attempt rate in the league. When he did throw far, he had the 12th ranked epa efficiency (20+e), so he could be successful downfield: He just rarely tried it.

Against average pressure, his reaction was to abandon attempts (throw-aways, sacks or scrambles) on 16% of dropbacks (aa%), which was the 13th highest rate and indicative of an unwillingness to actually throw the ball.

He was not accurate with above-average receivers

Completion % Over Expected (cpoe) measures what a QBs completion rate should be based on distance thrown and a lot of other variables. Brissett’s completion rate was 3.4% below expected, so he ranked 23rd in accuracy.

When he did complete passes, his receivers showed the 9th best ability to gain Yards After the Catch (yac) with 0.3 yac over expected (yacoe).

He was mobile and managed the ball well

On the plus side, Brissett did not turn the ball over often (22nd to%). This number includes 2 sack fumbles, so it is a bit higher than his INT rate alone, but it is still comfortably below league average.

He had a lot of throwaways (9th ta%) and scrambled at the 6th highest rate (scr%), both of which helped his ability to avoid sacks (26th sk%).

His ability to move the ball with his arm was below-average

Driven by low values in previously discussed stats (cpoe, adot, ay/c), his Yards per Attempt was far below average (23rd ypa). His low sack rate lifted his overall yardage efficiency, but Net Yards per Dropback (ny/d) was still only 18th in the league. This made it hard to gain first downs through the air (19th 1st%).

He threw a lot of TDs and was very good in the red zone

Even though most of his numbers so far have been average to bad, he had the 3rd highest Red Zone Epa Efficiency (rze) of any QB. That was in large part due to the league’s 5th highest TD rate (td%).

This is really the sole reason that he had so much buzz around him. When the field got short, he was throwing TDs and winning games. However, every silver lining has a cloud and Brissett’s was sustainability. The question wasn’t whether he had been successful through 7 games (he was), it was whether he could maintain that success the rest of the season.

It is unusual for a QB to maintain such high red zone efficiency when his supporting stats are not good. If a QB’s red zone efficiency ranking is a lot higher than his efficiency ranking outside of the red zone (orze) then that is a red flag that he is due for a regression.

After week 8, Brissett was 3rd in red zone efficiency and 16th outside of it, for a 13 spot differential (or 0.98 z-score differential) making him 1 of 7 QBs who had a rank differential of at least 10 spots. In the following 9 weeks, 2 of those QBs were benched (Mariota, Keenum) and the other 5 all saw their red zone ranking crash back down to a level at or below their ranking outside the red zone.

THE WHOLE PICTURE

Putting it all together, the data showed me a mobile QB, who managed the ball well, but when facing poor pass defenses without heavy pressure: held the ball a long time, wouldn’t throw downfield, was too cautious, couldn’t find open receivers, checked down a lot, wasn’t accurate and relied on the run game to move the ball. When the team did reach the red zone, he had success that was likely not sustainable.

So, what happened after that?

In his remaining games, he spent the same amount of time to decide (3rd ttd) while facing slightly harder opponents (0.10 opd vs 0.15) who dialed up the pressure (2nd pr%). Against that pressure, he scrambled a lot (8th scr%), but threw the ball away less often (13th ta%) and took a lot more sacks (13th sck%). He was forced to make longer throws (8th adot) but his downfield success disappeared (32nd 20+e).

He still only completed primarily short passes (22nd ay/c), but completed fewer of them as his accuracy plummeted (31st cpoe). This caused his net yardage efficiency to fall (26th ny/d) along with his passing conversion rate (30th 1st%). Unsurprisingly, his red zone efficiency fell in line with his numbers outside the red zone (30th rze, 28th orze) as his TD rate dropped off a cliff (32nd td%).

Brissett defenders will point to a week 9 injury as the reason for this fall-off, but nothing really changed from the first 7 games, except his numbers got worse against increased pressure and his unsupportable TD rate came back in line with the rest of his play. The Jacoby Brissett after week 8, looked very similar to the Jacoby Brissett from 2017, which was not a coincidence and pretty much foreseeable.

*weeks 9 – 17 only

CONCLUSION

So what can this tell us about Carson Wentz? I looked at the cumulative numbers from his 2018-2020 seasons to see if they can tell us anything about his decline from an MVP-like 2017.

First the bad news. He was 28th in epa/d, driven by an inability to pass for first downs (27th 1st%). This was a result of struggling to gain passing yardage (30th ny/d) which in turn was partially because he took a lot of sacks (6th sk%) while rarely throwing the ball away (27th ta%). Also, he was a little careless with the ball (11th to%). His red zone success (10th rze) was not from throwing TDs (24th TD%) and is at odds with other supporting stats, so he is likely due for a regression in that part of the field if nothing else changes.

Now the good news. He has faced harder than average passing defenses (12th opd) and played in a “pass first” offense with an inefficient run game (9th ed%, 21st wrsr) all of which can artificially depress a QBs numbers. He was not slow to pull the trigger (15th ttd) yet still faced the 11th most pressure (pr%). Combined with the high sack rate, that suggests some offensive line struggles. Additionally, even though his accuracy was not terrible (19th cpoe), his receivers provided the lowest adjusted yac of any offense (32 yacoe), which hints at a lack of talent downfield.

In part 2 of this series, I talked about how the surrounding players usually don’t impact a QBs efficiency much on average. I think the data above shows that Wentz is one of the exceptions as he has been in a kind of worst case scenario the last few years (O-line problems, poor receivers, tough defenses, below average run game). I think the data points to him doing much better with the Colts.

How much better? I’ll talk about that in part 4.

GLOSSARY

opd: The epa/d given up by opponent defenses in all games other than the QB/team being measured

ed%: The % of plays on early downs(2) that are QB dropbacks.

wrsr: The % of designed carries that earn more epa than the median league value in similar game situations (down, distance, field position etc.), adjusted for 4th qtr game script and weighted by result (TD, first down, other)

ttd: The average time from snap to the point when a QB throws, scrambles or is sacked.

pr%: The % of dropbacks where the QB was pressured (per Pro Football Focus)

adot: The average air yards thrown per attempt.

20+: The % of attempts >+ 20 air yards

ay/c: The distance between the line of scrimmage to the point of reception.

cpoe: Completion % over an expected amount based on game situation (air yards, down, distance, field position etc.)

yac: The distance between point of reception and the spot of the football at the end of the play.

yacoe: The yac over the league average yac for a given game situation (yards thrown, down, distance, field position etc.)

ypa: Yards per Attempt

aa%: The % of dropbacks that result in a throw-away, sack or scramble.

ta%: Throw Aways as a percentage of dropbacks

scr%: Scrambles as a percentage of dropbacks

sk%: Sacks as a percentage of dropbacks

to%: Interceptions and QB lost fumbles as a percentage of dropbacks

ny/d: Net Yards per dropback. (Passing Yards – Sack Yards + Scramble Yards) / (Att + Sacks + Scrambles)

1st%: Passing first downs as a percentage of dropbacks

td%: Touchdown as a percentage of dropbacks

rze: Expected Points Added per dropback in the red zone

orze: Expected Points Added per dropback outside of the red zone

20+e: Expected Points Added per dropback on passes >=20 air yards

psr: The % of dropbacks that have epa> 0

epa/d: Expected Points Added per dropback.

FOOTNOTES

1) Points per drive excluding drives that end with time expiration.

2) Pass attempts in game neutral situations on 1st or 2nd down with at least 3 yards to gain. Game neutral is defined as plays with a Win Percent between 20% and 80% and outside of the last 2 minutes of either half.