Patriots' Bill Belichick's facing a problem much bigger than the fallout of grabbing a ref's arm

Bill Belichick reached out his arm in an effort to stop referee Esteban Garza. But like too many instances Sunday night in Baltimore, the New England Patriots saw someone blow right past them.

That was a ref and a coach, the video of the postgame incident becoming the symbol of the Patriots' frustrating 31-30 loss to the Ravens as well as the potential tipping point in the NFL's use of replacement officials. The NFL is investigating and Belichick could be fined for what he says was an attempt to find out if the game-winning field goal was going to be reviewed. In related news, the league and its referee union are scheduled to continue meeting with a league of frustrated players, coaches and fans desperate for a deal.

More troubling now for the 1-2 Patriots was the ease in which Baltimore stormed back from a nine-point, fourth-quarter deficit by moving the ball up the field in the final minutes. The Ravens closed the game with a 92-yard TD drive that took just 3:42 and then needed just six plays to move 72 yards to set up the final, close-range field goal.

The NFL's referee disaster will get solved [we hope]. The bad calls went both ways – had the Ravens lost, it might have been John Harbaugh fruitlessly trying to stop Garza. Belichick either will or won't pay a fine, either way he remains a multimillionaire. The team has dealt with bigger controversies than this. However, the Patriots' inability to stop quality teams, especially in a late-game, pass-first situation, is both more troubling and not nearly as simple to remedy.

[Related: NFL must put an end to intimidation of replacement refs]

"In the end it wasn't really good enough," Belichick noted in his abbreviated postgame media session.

New England's secondary was torn apart and the most obvious problem spot was cornerback Devin McCourty, who was targeted and then exploited by Ravens quarterback Joe Flacco.

The Patriots invested heavily in defensive playmakers during April's NFL draft. They traded up in the first round to land defensive end Chandler Jones and linebacker Dont'a Hightower. Safety Tavon Wilson went in the second and defensive end Jake Bequette in the third.

Yet against Baltimore, the unit gave up 503 total net yards and couldn't come close to stopping Flacco when it mattered most. The most glaring issue, somewhat ironically, but not at all unexpectedly, was McCourty. He was a 2010 first-round pick that had an impressive rookie season. Then he hit a sophomore slump that saw opposing quarterbacks target him for over 1,000 yards.

After two solid games, Sunday was a reversal. He dropped two interceptions. He had a poor-tackling effort on a touchdown by Ravens tight end Dennis Pita. Then on the final drive, Jacoby Jones first beat him cleanly for 24 yards and then drew a clear pass-interference flag, good for 27 more yards, on a critical third-and-9 that would've forced a 51-yard field goal attempt.

[Related: Jim Harbaugh cons refs out of extra replay challenges]

Instead Baltimore set up inside the New England 10-yard line and was able to run down the clock. The Ravens were close enough that even Justin Ticker's inevitable hooking kick was able to get over the top of an upright to win the game.

"You have to make plays, plain and simple,"

told reporters after the game. "There were more plays, not just on the last drive, but all throughout the game that I can make and my team can make. And it's simple, I have to make those plays."

One player doesn't lose a game. There are countless moments in an NFL contest that dictate the outcome. That shouldn't reduce the cause for alarm in New England as the Patriots find themselves with a rare losing record and headed back on the road to Buffalo (2-1) on Sunday, a place they lost a year ago.

Belichick's push for defensive difference-makers was an obvious effort to shore up one side of the ball for the final few seasons of Tom Brady's brilliant career. An inability to get a stop on a final drive has cost New England in its last two Super Bowl appearances.

The rookies [other than Bequette who hasn't recorded a tackle] have been strong. Jones, in particular, has the makings of a star. Even after allowing 31 points Sunday, the Patriots are third in scoring defense in the AFC and boast a ton of guys gaining experience by the snap.

[More: Torrey Smith's peers help him get through tragic day]

This hasn't been a complete disaster.

Against Baltimore, though, the final drive was so easy the Ravens had time to move the ball to the center of the field before burning clock. And they were clearly going after McCourty and the outside coverage to get there.

"We did a great job on the outside [Sunday night] making plays," Flacco told reporters afterwards.

McCourty, among others, took their lumps and vowed to work hard and use the frustration of the loss to improve. They didn't deflect the blame or hide from the issue. "I had two balls in my hand," McCourty said of his would-be interceptions. "I have to make those plays."

That's nice but those are just words. No one accused McCourty or the defense of trying to fail on Sunday.

This is a question of ability. This is a question of whether the revamped, infused-with-youth-defense has enough to hold up its end of a Super Bowl bargain. Brady and Belichick are in this to win it all and nothing less.

Right now they're a game under .500, headed back into hostile territory and so frustrated the coach is dealing with a replacement-ref-arm-grab playing on a seemingly endless nationally-televised loop.

[More: Morning Rush: Cardinals are the NFL's biggest September Surprise]

The issues with the refs will presumably get worked out. What about the Patriots defense, though?

Related NFL video from Yahoo! Sports:

Other popular content on the Yahoo! network:
• Matt Schaub loses piece of ear on vicious hit
• New York Jets CB Darrelle Revis out for season with torn ACL
• NFL Week 3: Winners and losers | Injury update
• Dirtiest cities in the U.S.