Adam Matthews' first goal for Celtic proved just enough to send Neil Lennon's team into the fifth round of the William Hill Scottish Cup but their replay win also owed much to a controversial disallowed goal by Steven Doris.
Matthews fired a wonderful strike from 20 yards in the 19th minute but Celtic subsequently failed to take control at Gayfield.
And they were grateful for a 66th-minute decision by referee Craig Charleston, who presumably ruled out Doris's close-range finish for an earlier infringement on Celtic goalkeeper Fraser Forster, but a foul did not appear obvious.
Loading article content
Forster himself averted extra-time when he saved a one-on-one with Connor Birse as Celtic edged through to set up a trip to Raith Rovers.
Celtic made three changes following their 3-1 win at Kilmarnock on Saturday but retained the 3-5-2 formation and a strong team.
Mikael Lustig and Beram Kayal came in for Joe Ledley and Victor Wanyama, the Swede's introduction allowing Efe Ambrose to push into a holding midfield role. Georgios Samaras replaced Gary Hooper to start up front alongside Lassad Nouioui.
Arbroath manager Paul Sheerin, who was part of the Inverness team that shocked Celtic in 2000, started for the hosts, who earned the replay after a late deflected free-kick from Doris earned a 1-1 draw at Parkhead.
Samaras got the first effort on target with a header from Lassad's cross but it was straight at home goalkeeper Scott Morrison.
The Greek soon returned the favour after a backheel from Scott Brown, sending over a low cross that Lassad swept home but the offside flag was up.
Arbroath hit back with a well-worked move that ended with Sheerin hitting a goal-bound strike from 18 yards, but Kelvin Wilson got his head in the way to clear.
Arbroath were the architects of their own downfall though when Matthews won the ball off left-back Colin Hamilton following a pass from Sheerin.
The Wales international burst away but still had plenty to do, and lashed a powerful strike that flew over Morrison's outstretched hands and into the far top corner.
Celtic appeared to take their foot off the gas in an attacking sense and Arbroath were able to push play away from their penalty box but some decent attacks all broke down on the edge of the Celtic box, and they failed to trouble Forster from several set-pieces.
Celtic only had one more chance of note in the first half when Samaras set up Lassad after a wonderful diagonal ball from Charlie Mulgrew. But the Tunisian failed to produce an emphatic finish from eight yards and Morrison blocked the bouncing effort.
The hosts had the first sniff of goal of the second half when Sheerin's free-kick caused problems but Stuart Malcolm was penalised for a high challenge on Forster, and harshly booked for going for a ball that was there to be won.
Paul Currie and Brown fired wide at either end and Celtic were soon stretched again before Emilio Izaguirre headed away under pressure following Sheerin's cross.
Sheerin soon caused more problems with the corner that led to the contentious decision to chalk off a goal.
The Arbroath player-manager delivered the ball right into the heart of the goalmouth and Forster could not collect. Hamilton won the high ball and headed down towards goal with Doris forcing the ball home on the line.
But Charleston, who was standing on the edge of the box, signalled for a free-kick.
Kayal came close at the other end and Forster saved from Lee Sibanda.
Wanyama and Hooper came on as Celtic manager Lennon switched to a 4-4-2 in a bid to regain control of the tie.
But there was another major escape when Birse won the ball in the middle of the Celtic half and took a return pass from Doris that sent him clean through.
But the substitute was quickly closed down by Forster, who got just enough of his diving body in the way of the shot to divert it wide.
Lennon was growing increasingly frustrated with his side's slack passing and Forster was soon scrambling again as Currie fired just wide from long-range.
And the visitors were grateful again for the sound of Charleston's whistle at the end of three minutes of injury-time.