NFL Draft 2023: Will Levis never had a 99.9% chance of going in the 1st round, and ESPN’s model gives advanced stats a bad rap

The NFL Draft’s quarterback storylines looked like they would be resolved in a flash. Three quarterbacks — Bryce Young, C.J. Stroud and Anthony Richardson — flew off the board in the first four picks. That left Kentucky’s Will Levis, the last member of the group frequently mentioned and mocked as first-round picks, in the green room awaiting his call.

Levis eventually went 33rd overall to the Tennessee Titans, the second pick of Friday's second round. But he'll long be remembered for his Thursday slide, and how it was covered.

As possible landing spots came and went, ESPN’s broadcast began to tout an amazing, ever-shrinking number: the chances, according to “ESPN Analytics,” that Levis would remain available. By the end of the night, he was still in the green room — and suddenly the butt of a joke. ESPN’s model said he had come into the night with less than a 0.1% chance of not being picked.

Presented as an NFL Draft facsimile of win probability, and available to tinker with online, the model asserts itself as a sort of weather report for how draft picks will play out. (It now foresees a 10% chance of Levis at the Pittsburgh Steelers’ No. 32 selection but increasing chances of Levis as we roll into the Detroit Lions’ No. 34 pick. The entire idea of a second-round landing spot, of course, was pegged as borderline impossible prior to Thursday.)

That makes for an eye-popping graphic, but this ESPN model, which makes Levis sound like a historic draft flop, has put its “probability metric” shirt on inside out. The useful models that produce win probability or playoff odds to contextualize games or seasons derive their immediate insights from past events fully played out, from settled facts. The ESPN draft model is “based on expert mock drafts, team needs and Scouts Inc. grades and provides a probabilistic forecast for how the NFL Draft will go.” It is crunching a variety of famously fallible human predictions about what will happen and presenting the results as a quantified statement about what is most likely to happen. In other words, it’s dressing up a mock draft — a perfectly fun exercise explicitly understood as a person’s opinion — as a statistical finding.

On the most basic level, that casts an unnecessarily harsh shadow of implied failure over a college athlete who will still most likely wind up being deemed one of the 50 or 60 most attractive players in his class. On a broader scale, it threatens to kneecap the credibility of good advanced metrics — many of which are still overcoming reflexive skepticism of the unknown or full anti-analytics bogeyman sentiments — that could help fans have better, more interesting conversations about sports.

Why the NFL Draft can’t be quantified by a probability metric

The big, honking problems with trying to apply probabilities to who will be chosen when in the NFL Draft are apparent in the contrasts with win probability, a good application of data.

The first thing to know about win probability, something frequently misunderstood, is that it doesn’t make predictions. The numbers you see on the screen — the Los Angeles Chargers have a 98.5% chance of winning vs. the Jacksonville Jaguars in the playoffs, let’s say — are essentially working backward. They start with a big database of how past NFL games have played out, paying attention to the remaining time, score and downs, and they fit the game at hand into the spectrum. The 98.5% is not a statistic saying, “I’m 98.5% sure this Chargers team is going to win this game.” That, as you might recall, would've been a bad prediction.

Instead, it’s actually saying, “Based on the dataset, teams up 24 with two downs left on the opponents’ 5-yard line and 34:36 left in the game should be expected to win 98.5% of the time.” That’s helpful. It provides avenues to descriptive context for whatever happens — pinpointing which mistakes allowed the Jaguars to claw back and showing just how monumental the collapse proved to be.

If you’re trying to port that over to Thursday, Levis after pick No. 4 becomes the Chargers’ status up 24-0, and the rest of the draft equates to the remaining game time.

Maybe you’ve spotted the two main problems.

There has never been another Will Levis — and not just because of his odd banana-eating and coffee-drinking habits. The probabilities you could apply to college players entering the NFL Draft entail projecting their future performance in the NFL based on college statistics and/or physical traits. Those sorts of tools — prominently used to assess player valuations in baseball and other sports — are worth exploring, and I imagine many NFL teams use them. But projecting where Levis will be picked isn’t the same task. It relies much more on soft factors (did he jel with a team’s quarterbacks coach?) and outright hidden information (does he have a lingering toe injury?).

All of those pitfalls are true before you factor in the influence that agent maneuvering, team smokescreen ploys and literal anonymous Reddit posts could've had on the observers whose predictions apparently form the backbone of the ESPN probabilities.

There’s not a consistent set of rules. These metrics need to define the game they are trying to break down. In your basic win probability metric, the participants have singular motivations (score more than the other team), and the model confidently makes ironclad assumptions about how that can and can’t happen (teams must progress at least 10 yards every four downs or lose the ball). Those just aren’t possible in the NFL Draft.

So ESPN’s model invents guardrails where there are none. By ascribing “team needs,” the system is creating a logical box that the human beings in war rooms don’t actually have to stay within. Per the draft predictor spitting out these numbers, the Detroit Lions didn’t “need” a running back, but they took one anyway. The Las Vegas Raiders “needed” a corner, a defensive tackle, an offensive guard and a quarterback — where Levis was the top remaining talent when they picked. They didn’t take a quarterback, nor did they take any of the other needs.

NFL teams might not pursue their goals successfully in game action, but you can be pretty sure that if they face a second-and-8 situation, they are not going to pull out a 5-iron and aim for the fairway.

How ESPN could erode trust in advanced stats

I don’t mean to ascribe malice to the ESPN folks behind this. It was surely a good-faith effort to provide a new way to understand the machinations of the NFL Draft and feed the content machine around one of football’s most fascinating, most dramatic events. In the grand scheme of things, a number on a screen isn’t a societal evil — maybe Levis will become a Pro Bowler and the graphics from Thursday night will be trotted out on Future Twitter as memes. Who knows.

But the misdirected deployment of advanced-sounding statistics isn’t helping a cause to which ESPN has otherwise made excellent contributions. Helping fans embrace newer numbers and ways of communicating about football is a worthy endeavor. Playing fast and loose with which numbers sprout from serious, trustworthy analysis and which ones are actually translated sad face emojis to plaster over a dejected quarterback’s face undermines that cause.

If you don’t believe me, just peruse the quote tweets here. You’ll get the gist.

Fans are, rightfully in this case, pushing back at the idea that a player’s draft fate can be reduced to a probability. Whether they are inherently skeptical of data in sports or newly perturbed by this instance, it’s a disservice to the painstaking work of analysts and commentators — many of whom work for ESPN — who use numbers to issue more informed commentary and guide sports talk away from its lesser tendencies.

Advanced stats in sports are by no means sacred. There are plenty of valid reasons to skip the deep dives and go along for sports’ emotional ride. There are plenty of valid metrics that understandably inspire send-ups. Still, a lot of fans simply don’t know what they’re missing because the names look daunting or the numbers don’t carry traditional heft. Ideally, proponents of seeking more knowledge in sports — that’s all “analytics” really means — keep their doors wide open and welcome in all who are curious.

Not everyone wants to learn. But those who do shouldn’t be scared off by conjecture in a stat’s clothing.