Mauricio Pochettino's team had scored six in their previous outing, dispatching Middlesbrough to reach next month's Carabao Cup final, but found Unai Emery's Premier League high-flyers a more obdurate hurdle as a combined 23 shots from both sides failed to yield a winner.
It was a theme of the night in west London as a replay at Villa Park beckoned.
Visiting supporters were on their feet when Douglas Luiz tapped in from almost on the goal line, but their delight was curtailed after VAR determined the midfielder had handled as the ball deflected to him off Alfie Gilchrist.
Palmer fed Madueke who might have done better with his low finish, aimed towards the corner but instead fired against Martinez.
Palmer was then the recipient of a ludicrous gift from Clement Lenglet, the defender's square ball easily cut out by the former Manchester City man who again found only the goalkeeper with his shot.
With virtually the final act of the first period, Moussa Diaby raced down the right and centred for McGinn, who with a sweep of his right boot hit a curling strike that cleared the crossbar by inches.
Villa ended a run of eight straight FA Cup defeats with their third-round victory at Middlesbrough but here they were made to weather a first half in which Chelsea had the better of the chances, though as so often this campaign Pochettino's side lacked cutting edge in front of goal.
Palmer tried to benefit from another Villa howler at the back midway through the second half.
Martinez hit a clearance with insufficient height that struck Chelsea's top scorer, who seemed in too much of a hurry to execute the finish. Trying to catch out the backtracking goalkeeper, he missed his kick entirely, as Emery breathed a sigh of relief.
Thiago Silva flung his head at a goal-bound Luiz effort as the game started to become stretched.
Ollie Watkins took a touch and hit a fizzing drive that Petrovic beat away, then the goalkeeper made the save of the match diving low with a stunning reach to claw Cash's left-footed piledriver from the foot of the post.
McGinn had the chance to settle it and dodge a replay four minutes from time, arriving at the near post to meet Nicolo Zaniolo's cross at the near post. However, he got too much of a glance on the ball under pressure from Silva, as the chance and the game fizzled out.