A dramatic stoppage-time strike from John Sutton gave the Steelmen a point after the champions had fought back from two goals down and a penalty miss by Kris Commons to forge ahead through substitute Leigh Griffiths with only six minutes remaining.
Afterwards the Hoops boss, who sent his players out early for the start of the second-half after giving them a "bit of a rocket", said: "I am annoyed. We were poor defensively first half, gifted Motherwell two goals with poor defending.
"There was definitely a psychological edge missing first half. We got all that back in the second half, we were fantastic, and we should have won the game.
"We dominated the second-half from start to finish but switched off. Motherwell had one attack and scored from it.
"On the counter attack we had three four or five really good chances to make the game safe.
"The keeper has made a great save from the penalty, so overall, attacking wise, we were really very good, defensively we were sloppy.
"We were done with a straight ball for the first one but I think the keeper should save it, the second one we have missed the flight of a cross and the third one Emilio (Izaguirre) is sleeping and they have got in behind us.
"But I am really pleased with the quality and character in the second half."
Goals from Sutton and Zaine Francis-Angol either side of a Gunnar Nielsen penalty save from Commons, had put the home side ahead before Anthony Stokes reduced the deficit just before the interval.
Georgios Samaras levelled 10 minutes after the break before Griffiths, on for Beram Kayal, grabbed a third in the 84th minute but, as the game moved in to stoppage-time, Sutton grabbed his second from a yard out to take the Lanarkshire side to within three points of second-placed Aberdeen.
Stokes stressed he had meant to chip Nielsen from inside the box for his goal.
The Republic of Ireland striker said: "I went for goal but it's a one in a thousand. I didn't have much on and couldn't cut it back so I just went for it. I wouldn't say I was lucky but I'll take it as a good finish.
"The half-time team talk was quite short and had we not scored just before half time it could've been even shorter."
Motherwell boss Stuart McCall had few complaints about his side's performance.
He said: "I really enjoyed that, I'm so proud of the lads today. I thought we played our best football at 2-2.
"We've done it so often this season, we lose silly goals right on half-time but it was a great goal by Stokes. We knew Celtic would come at us in the second half and they got an early goal.
"But we got what we deserved. There were some massive performances from the players.
"They looked dead on their feet but they dug out a point and next week's game against Dundee United is massive."
Sutton claimed Motherwell's quest for second-spot has gone unnoticed this season with the focus on their 'best of the rest' rivals, Scottish League Cup winners Aberdeen and William Hill Scottish Cup finalists Dundee United.
"We are not one of the sexy teams this year," he said. "Everybody is talking about Aberdeen and Dundee United, but our league form has been really good and we have just to keep it going.
"It is going to be really big game up at Tannadice for us next week and then we have two massive home games before Aberdeen so there's lots of points to play for and hopefully we will be in the mix going up there."