A mostly uneventful stalemate in 90 minutes ensured another half-hour's play and five minutes into the extra period, the Ton midfielder Dougie Imrie scored from the spot after referee Bobby Madden adjudged Celtic defender Efe Ambrose to have handled when challenging Greenock substitute Archie Campbell.
The Hoops bombarded the Morton goal but despite 27 corners and numerous efforts on goal, they could not draw level leaving the delighted Championship side to take the plaudits from the travelling support at the final whistle
The Celtic boss said: "I have got to be philosophical about it. It wasn't for the attitude or want of trying in the game.
"It just wasn't our night for whatever reason.
"The only disappointment for me is we had 26 corners and we didn't convert one of them with the height we had in the team.
"We have to take our chances.
"We ended up putting Virgil van Dijk up to give us something different and certainly we created a lot more."
The Northern Irishman continued: "If you don't take your chances you don't expect to win the game.
"I thought we would at least get one tonight and that is a surprise and disappointment to me.
"But I am not going to hammer the players, it has been a very demanding start to the season for them and I think they are looking forward to the game on Saturday.
"It is a frustration but I will get over it very quickly. I have too much to look forward to, to dwell on it."
Morton boss Allan Moore described the result as "incredible", especially in in light of his side's Championship form which has seen them slump to second bottom of the table.
Moore, who revealed midfielder David O'Brien dislocated his shoulder in a challenge with Dylan McGeouch, said: "There aren't many times I've come here even in my playing career and got good results.
"But to a man we were excellent, especially after the league form we've had over the last couple of weeks. What a dressing room it was, that's why you come to play football.
"They were talking before the game about swapping tops with the Celtic players. I told them if they swap it to make sure it was full of sweat.
"If they did that, they'd have put a shift - and to a man they did that. I'm over the moon and hopefully it's the catalyst to kick on in the league."