Fourth-down analysis met with skepticism

http://static.espn.go.com/nfl/columns/garber_greg/1453717.html

Thursday, October 31, 2002
Updated: November 1, 1:45 PM ET
Fourth-down analysis met with skepticism
By Greg Garber
ESPN.com

BERKELEY, Calif. — Wander down Telegraph Avenue, through the bohemian landscape of bistros and shops. Cross onto the eclectic campus of the University of California. Pass through Sather Gate. Make your way past dozens of tables set up by student organizations in their fall recruiting mode. Shuffle through the falling leaves. Eventually you will come upon Evans Hall, a hulking, utterly charmless stack of concrete.

Up in a fifth floor classroom, David Romer, a Cal-Berkeley professor of economics, writes a series of hieroglyphs on the board — with a Sharpie, which, unlike a certain wide receiver from across the Bay, he did not pull from his khaki sock:

Ei Di(gt) Vi = Pgt + Bgt Ei Di(gt+1) Vi – egt

This, in the unimpeachable terms of the Bellman Equation, is the definitive proof that NFL head coaches should go for it more on fourth down. Really, no kidding.

Bill Belichick
Bill Belichick is one of several NFL head coaches who has studied David Romer’s work.
“Teams should be a lot more aggressive on fourth down,” Romer says with the precise air of a scientist. “On average, you’d be better off going for it, essentially, in any short-yardage situation, and then there are some cases even in longer yardage when you’re in that position where neither punting nor a field goal looks very attractive.

“The usual assumption of profit-maximization implies that in their on-field behavior, teams should act to maximize their probabilities of winning. This isn’t happening.”

This, of course, is heresy. In the button-down world of the NFL — the last bastion of fascism, according to one enlightened former player — a conservative approach is almost mandatory. Survival of the safest, if you will. Fourth down, the ultimate push-comes-to-shove, all-or-nothing moment in football, is not generally judged to be worth the gamble. Not only are points often at stake — passing on a three-point field goal is sacrilege in Coaching 101 — but there is also the momentous matter of field position, roughly 40 yards worth. Coaches are terminally terrified of risking that stretch of precious real estate.

But the risk, Romer insists, is almost always worth the potential reward.

“This is a professor from Cal-Berzerkely?” asked Giants head coach Jim Fassel, in the true tradition of a former Stanford man.

Fassel turned a sheet with the equation on it sideways, then upside down in a humorous attempt to absorb its subtleties.

“What does the professor coach?” Fassel asked. “Maybe,” he added, “he needs a few more classes to teach. Too much free time?”

Steve Mariucci, the 49ers head coach, has considered Romer’s thesis. The difficulty, according to Mariucci, is maintaining a level head when everyone in the stadium wants you to go for it.

“The crowd is going ‘Go for it,’ and they’re just drinking beers and just going for it,” Mariucci said. “Sometimes you get swayed a little bit. So you’ve got to block them out and you’ve got to make sense of it all.

“So then you start thinking about that article the guy from Cal wrote and then you say, ‘Well, what would he do in this situation?’ ”

Along with Mariucci, Bill Belichick of the Super Bowl champion New England Patriots is one of several NFL head coaches who has actually studied Romer’s working paper, which, with references and tables, runs a stout 33 pages. Maybe it’s because Belichick graduated from Wesleyan University with a bachelor’s degree in economics.

“I think, basically, he was saying that if you get down there and don’t score, you’re putting the other team 80, 90 yards away from the goal line anyway, and the chances of them scoring aren’t very good,” Belichick said. “You’ll probably get the ball back in good field position. And the percentages added up to his conclusion, which was to go for it.”

Bill Walsh, another Bay Area professor with some serious tenure, is a believer.

“To this day, I can close my eyes and see 22 players,” he said in his San Francisco 49ers office, where he still serves the team as a consultant. “I can see them moving and I can see the equation. My indicators would be somewhat different than (Romer’s), but this (equation) is what it takes.”

Walsh looked closely at the tangle of letters and numbers.

“It would be fine if I completely understood it,” he said. “But I don’t think you can get all the variables on one page. I don’t care what the equation is, you can’t get them all.”

That said, Walsh agrees with Romer’s basic premise.

“I think (coaches) can tend to be too conservative,” Walsh said. “They can tend not to trust themselves — in a sense, take the easy way out and, in this case, punt the ball.”

Not the ‘obvious’ call
About five years ago, Romer was listening to a Raiders game on the radio when a pivotal fourth-down situation came up.

“The team kicked a field goal and the announcers discussed it long enough to say, ‘It’s obvious, it was clearly the right call,’ ” Romer said. “That bothered me. I was thinking about it some and then I realized that some of the tools I use in my economics work were going to be useful.”

NFL Countdown
ESPN’s Greg Garber spent time talking to David Romer and went around the NFL talking to coaches about the professor’s study. Tune in to ESPN at 11 a.m. ET Sunday to watch.

Romer applied the Bellman Equation, which is used in economics and math to help simplify dynamic, long-term problems into the immediate payoff and the future ramifications of the situation.

Based on statistics from actual NFL games from 1998 through 2000, Romer assigned a value to having the ball first-and-10 on each of the yard lines of the football field in terms of points scored. Since teams rarely go for it on fourth down — last year, the league’s 31 teams went for it a collective 468 times, an average of less than once per game — Romer decided it was not a representative sample. He elected instead to use third-down statistics in assessing probability. In addition, he used only first-quarter statistics because, generally speaking, score and time left did not influence decisions that early in a game.

The findings of Romer’s analysis:
A team facing fourth-and-goal within five yards of the end zone is better off, on average, trying for a touchdown.

At midfield, on average, there is an argument to go for any fourth down within five yards of a first down.

Even on its own 10-yard-line — 90 yards from the end zone — a team within three yards of a first down is marginally better off, on average, going for it.

Of the 1,575 fourth downs in the sample where the analysis implied that teams were, on average, better off kicking, teams went for it only seven times. However, on the 1,100 fourth downs where the analysis implied that teams were, on average, better off going for it, they kicked 992 times.

Translated, in the 1,100 instances where teams had more to gain in Romer’s estimation by going for it, they did so only 108 times, or less than 10 percent.

The primary objection to the concept revolves around field position. A punt will generally net around 40 yards, and that’s a solid number coaches feel is worth that decision not to gamble.

As Tampa Bay’s Warren Sapp says, “I’d rather defend an 80-yard field than a 40-yard field.”

Romer freely admits there are many variables he doesn’t account for. This is the area football coaches wonder about.

“Where are you in the game?” Fassel asked. “What’s the score of the game? Are you running the ball or passing?”

“ It’s easy to sit there and apply a formula, but it’s not always the easiest thing to do on a Sunday. There’s so much more involved with the game than just sitting there, looking at the numbers and saying, ‘OK, these are my percentages, then I’m going to do it this way,’ because that one time it doesn’t work could cost your team a football game, and that’s the thing a head coach has to live with, not the professor. ”
— Bill Cowher, Steelers head coach

“Do we have the personnel to get that yard?” Walsh asked. “Do we have the personnel to stop the play they may run?”

“Do we punt and use our timeouts?” Belichick asked. “Do we have confidence in our field-goal kicker? Are they going to blitz or not?

“If I don’t get the first down, what are the repercussions?” asked Packers head coach Mike Sherman. “Are they moving the football? If you’re on the road and don’t get that fourth down the momentum is going to change over to the other team.”

Momentum, according to coaches (see sidebar), is a matter of some consequence in fourth-down situations. But Romer — a man as serious as an economist can be — doesnt’ pretend to offer an infallible system, just a guideline. That’s why he couches his conclusions with the words “on average.”

The skepticism, he allowed, does not surprise him.

“That makes sense,” Romer said. “If they came to me and said they could use their ability to design plays to run monetary policy better, I would be skeptical.

“I can’t tell them specifically what to do in particular cases, but on average they would be better off if they went for it in some cases where they now are always punting.”

An intuitive sense
On fourth down, it’s either-or, in or out, liberty or death. It’s the ultimate game of chicken.

“Everybody sitting in the stadium knows, hey, this is a big play,” Belichick said. “If you go for it, that you’re going to maintain possession and it’s going to give you another opportunity to score, or you’re going to give the ball to the other team and then you’re going to have to go through another series of plays to get it back — if you get it back.”

Brian Billick, the cerebral head coach of the Baltimore Ravens, isn’t so sure about all of the professor’s numbers.

“There are only two numbers,” Billick said. “And those are 50-50. You either make it, or you don’t.”

And believe it or not, 50-50 is almost exactly the success/failure rate.

Last year, teams made 203 of their 468 attempts on fourth down, or 43.4 percent. This year the number is up. Teams have converted 97 of their 191 fourth-down attempts, a .508 success rate. The number suggests Romer might be onto something. Nineteen of the league’s 32 teams are at 50 percent or better, and 23 teams have a better conversion rate on fourth down than third down.

As you might expect, teams with their backs against the wall tend to go for it more often.

As Miami head coach Dave Wannstedt says, “The key is to go on fourth down early in the game — when you don’t have to.”

The winless Cincinnati Bengals, for instance, lead all teams with 15 tries (and only four successes) on fourth down. The Washington Redskins are second with 11 attempts on fourth down and five conversions, while Minnesota (5-for-10) is next. The quality of a team has a lot to do with its success rate. Last year, the St. Louis Rams, at 8-for-11, were the league’s best, with an astounding percentage of 72.7. The Tennessee Titans, at 2-for-15 (13.3) were the worst.

Some teams are more aggressive than others. Under head coach Tom Coughlin, Jacksonville has gone for it 10 times and made seven. Philadelphia’s Andy Reid has already made six of nine tries. Last year he was an impressive 9-for-13. Only two AFC teams went for it more than Belichick’s Patriots a year ago. This year, the Patriots have already converted four of seven tries.

What it really comes down to, Belichick said, is confidence. Confidence that your offense can get that yard or two, confidence that your defense can hold the opponent if the offense fails.

“The more third-and-ones you make, the more likely you are to go for it on fourth-and-1,” Belichick said. “You also factor in the defensive side of the ball. How good of a short-yardage or goal-line team is your opponent? When you add those two up and you’re real good and you don’t think the other team is that proficient at it, that’s one thing. If it’s vice versa, then maybe that skews you.

“In the end, it’s about intuition and recalling all of the experiences you’ve been through.”

Said Billick, “It’s all about matchups. Can you win that matchup and get the yard or yards you need to? Do they have the wherewithal to stop you? In the end, it’s how good do you feel about your team that’s on the field executing.”

Fourth-Down Decisions
Editor’s note: Here are a few big fourth-down plays so far this season:

Week 1 — Rams at Broncos: Trailing the Broncos 16-13 in the third quarter with the ball on Denver’s 9-yard line, the Rams went for it on 4th-and-1. Kurt Warner’s pass was incomplete and the Rams ended up losing their opener 23-16. “We will be aggressive in our approach,” Rams coach Mike Martz said. “Some may question it, but that’s how I’m going to approach it.”

Week 2 — Giants at Rams: Facing fourth-and-inches with 2:53 left in the game, the Rams trailed the Cowboys 26-21. Despite the absence of starting left guard Tom Nutten, the Rams ran Marshall Faulk straight up the middle and he was stuffed for no gain before fumbling. The Rams did get the ball back but ended up losing 26-21.

Week 5 — Giants at Cowboys: Trailing 21-17 and facing fourth-and-nine from the Giants 48 with 2:03 left in the game, Cowboys coach Dave Campo elected to punt. Dallas never got the ball back, losing 21-17. “I’m a little surprised they punted. I didn’t expect them to put it on their defense like that,” Giants defensive end Michael Strahan said.

Week 8 — Bengals at Titans: There was no controversial decision involved here, but this fourth down decided the game. Trailing 30-24 with 1:08 left and facing fourth-and-one with a yard to go for a touchdown, Corey Dillon had a clear path to the end zone but tripped on pulling guard Matt O’Dwyer, coming up short as the Bengals remained winless.

Week 8 — Bills at Lions: Trailing 24-17 and facing fourth-and-inches at the Buffalo 20 with 1:52 left in the game, Lions running back James Stewart was stuffed for no gain. The Bills took over, converted a first down and ran out the clock.

Already this season, there have been some classic fourth-down decisions. When the Broncos decided to try a long Jason Elam field goal on Monday Night Football, Ravens cornerback Chris McAlister returned the miss 107 yards for a touchdown and an NFL record. Remember when Rams coach Mike Martz was looking at a field goal to tie a game in Week 1 against the Broncos and elected (unsuccessfully) to go for the touchdown? He’s still hearing about it. Caught in the nether region between the Patriots’ 30- and 40-yard lines, the Packers went for it on fouth-and-three. Brett Favre made it. Another advertisement for going for it? The Tennessee Titans, up by only two points, went for it on fourth-and-goal at the Jacksonville 1. Eddie George scored what turned out to be the clincher.

“A lot of economics is about optimization,” Romer explained. “A lot of our models say that people are good optimizers, and we don’t think that people all know the Bellman Equation and they can’t all do calculus in their heads. We think that trial and error, survival of the fittest, imitation is going to cause people to get toward optimizing behavior. So the question is, why doesn’t it seem to be working here? Why are they being so much too conservative?

“There’s got to be some force pushing them the other way. And what it looks like is there’s a natural human tendency, it may be a genetic one, to worry about the worse case. If you’re teaching your kid about crossing the street or you’re hunting a saber-tooth tiger a few thousand years ago, the worst case is really something terrible.

“In this situation, the worst case really isn’t that bad — you go for it and you don’t get it, so they have the ball in better field position. There’s a medium-sized cost to failure and there’s a medium-sized reward to success — mainly, you get to keep the ball and the drive keeps going. Sometimes, what you need to say is, ‘Let’s just step back and look at the probabilities, look at the cost and the benefits.’ Work it all out and see what you get. Sometimes, your intuition is leading you very far astray.”

The Steelers’ Bill Cowher, in his 11th season, has enjoyed the longest consecutive tenure of any NFL head coach. He has made the playoffs seven of his first 10 seasons and looks like a good bet for eight of 11. He didn’t get this far by randomly adapting the flavor of the month in the fashion-conscious NFL.

“It’s easy to sit there and apply a formula, but it’s not always the easiest thing to do on a Sunday,” Cowher said. “There’s so much more involved with the game than just sitting there, looking at the numbers and saying, ‘OK, these are my percentages, then I’m going to do it this way,’ because that one time it doesn’t work could cost your team a football game, and that’s the thing a head coach has to live with, not the professor.

“If we all listened to the professor, we may be all looking for professor jobs.”

Greg Garber is a senior writer at ESPN.com.