At 2:26 a.m. Tuesday, Milos Raonic is denied a U.S. Open quarter-final spot by Kei Nishikori

Milos Raonic returns a shot to Kei Nishikori during the fourth round of the U.S. Open Monday. AP Photo/Jason DeCrow)

NEW YORK – By the end of the four hours and 19 minutes Milos Raonic spent on Arthur Ashe Stadium court on Labour Day Monday, his legs were gone.

The dividing line, that moment when they just said no más, came after Raonic was broken at 2-2 in the fifth set, a break that would prove to be the decider in a 4-6, 7-6 (4), 6-7 (6), 7-5, 6-4 loss to Kei Nishikori of Japan in the fourth round of the U.S. Open.

By then, Nishikori's down-the-line backhand was landing in the corner uncontested time after time; the four-plus hours the Canadian spent over in the ad-side corner looking for forehands finally taking its toll.

"It was a combination of fatigue, pain in some ways, and just not feeling like I would expect or hope to feel that late in a match," Raonic said. "Just stuff I sort of been dealing with a little bit and you're sort of compensating over. You sort of find parts where you're not normally sore are getting sore. In that kind of aspect. Nothing really drastic."

It wasn't of the slightest comfort to Raonic, who looked so young and so disappointed in his post-match press conference even as he tried to be eloquent in defeat, but the 2:26 a.m. finish was the latest in U.S. Open history.

Well, tied for the latest-ever along with two other matches: Philip Kohlschreiber and John Isner in 2012, and Mats Wilander and countryman Mikael Pernfors in 1993.

"I thought in general it was a good match if you sort of step away from it and you look at the whole thing. I wish I could have been, let's say, more efficient in many aspects, like my serve and other things. Especially in the beginning of the match I was just not figuring it out. I was losing my serve way too many times. This whole tournament, actually," Raonic said. "But I kept fighting, and that's all I can ask. I fought through that third set. Then I felt like he sort of just got a little bit over me and I was just sort of the whole time playing to just stay alive."

Kei Nishikori, of Japan, cools himself in a break before his decisive fifth set service game against Milos Raonic, of Canada, during the fourth round of the U.S. Open tennis tournament Tuesday, Sept. 2, 2014, in New York. (AP Photo/Jason DeCrow)

Raonic had his chances. Winning the second-set tiebreak would have given him a two-sets-to-none lead, awfully difficult to come back from.  He also led two sets to one.

Nishikori, a 24-year-old who is seeded No. 10 in the tournament, can only dream of doing what Raonic does on a tennis court. But Nishikori does other things extremely well and in the end, one of the best serve returns in the game narrowly trumped one of the best serves in the game.

Nishikori had his chances as well. In that third set, he had four break points when Raonic was serving at 4-4, and couldn't convert one. In the tiebreak, he was up a mini-break at 4-3 – and double-faulted to give it right back. Raonic ended up winning it 8-6.

"Hard to lose like that. I was up and make double faults and, you know, very tough to concentrate again. But tried to fight every point, and when I have to, you know, play well, I did. Took some chances. I broke his serve almost I think every set. And especially I was playing like normal," he said. "I was playing terrible in the first and second and I was a little bit tight. I mean, so was him too, I think. Very happy to win today."

It was a great opportunity for both to reach their first U.S. Open quarter-final, and Nishikori was the one who made it – the first Japanese player since 1922 to do so.

In the final analysis, two things hurt Raonic. The first was that he said he didn't serve well – in fact, he felt he really didn't serve up to his standards the entire tournament.

"I think in general just his foot speed was probably the most difficult part. He was taking the ball very early, controlling the centre of the court. He was keeping himself in a lot of situations where someone might be out of position. He was getting himself in good position and giving himself good opportunity to swing at the ball properly," Raonic said. "He does return well. I believe I could have served much better."

Raonic had 35 aces, not a huge total in a five-set match on a relative fast court for him. But he served at just a 55 per cent clip. And he fell below the 50 per cent threshold in terms of points won on his second serve.

The second area was Raonic's return.

It's no secret that the serve return may well be the weakest part of the Canadian's game. But the ATP Tour stats have him ranked No. 46 in first-serve return points won, just 62nd overall on second-serve return points won.

That weak area really showed up in this match. Nishikori is a good server, but hardly a great one. His second serves were mostly coming in between 80-90 mph, some of them even in the 70s. And the number of balls Raonic failed to put in court – and he wasn't even going for broke trying to hit return winners – was almost shocking. Those were moments when he could have jumped on his opponent, but he kept letting him off the hook. Nishikori won 66 percent of his second-serve points and against most people, that is going to win you the match.

As Raonic said, as the match progressed, eventually he got to the point where he was just trying to stay alive. Meanwhile, Nishikori's legs remained strong. That doesn't mean he wasn't feeling it physically. Even at 3:30 a.m., long after the match was over, the heat outside was still stifling. But generally speaking, in conditions like these, the big dog feels it a little more.

After coming into the tournament only having played points on the practice court a few days prior, because of a toe injury, Nishikori has gone further than he ever thought he would.

Conversely, Raonic believes that his best chances at a major lie in New York and Australia, the two hard-court Grand Slams. This year will not be the year.