Niall McGinn had slotted Derek McInnes' side ahead after a quarter of an hour at Ibrox.
But when Adam Rooney failed to take a second chance soon after, the League Cup holders blew their best chance to claim the trophy double they had been longing for.
That left it down to 25-goal May to make himself the hero again for Saints. He levelled the match after an hour before firing the famous goal which sent the McDiarmid men into their first-ever Scottish Cup final, ending a seven-match losing streak in last-four clashes in the process.
St Johnstone manager Tommy Wright, whose side will now face Dundee United at Celtic Park on May 17, was forced into a late change to his team as winger Lee Croft was ruled out with a hamstring pull, losing his place to Michael O'Halloran.
McInnes, on the other hand, had the luxury of a fully-fit squad and named the same side which beat Hibernian at Easter Road last Monday.
But they failed to click into gear in the same way they had when they thrashed Wright's team in January's League Cup semi-final despite netting the opener.
The first 15 minutes were scrappy but amid the loose touches and heavy tackles Peter Pawlett found a moment of quality to create the opener.
The Scotland Under-21 playmaker collected Russell Anderson's pass out of defence under pressure on the edge of the Saints box before spinning to pick out McGinn's run through the middle of the opposition defence.
The pass was weighted perfectly and McGinn only had to poke the ball past Alan Mannus with his right toe to give the Dons the lead.
St Johnstone knew the cost of letting Aberdeen score early after shipping four in that crushing Tynecastle defeat.
But they almost handed McInnes' men a second soon after as Steven Anderson went to ground far too easily just as McGinn slipped by him before slotting in Rooney, but Mannus was quick off his line to block.
The Perth side did at least threaten for the first time in the 23rd minute as May got a foot to a bouncing ball in the Reds box but saw it deflected away from goal by Jamie Langfield's outstretched foot.
David Wotherspoon put a free-kick on target but the Dons stopper saved comfortably, while a May set-piece four minutes before half-time was just as easily gathered.
The second half again started with a messy affair but another hasty attempt to win the ball by Anderson let in McGinn once more. The Northern Irishman's cut-back for Pawlett was perfectly timed but a second goal did not come as the midfielder miskicked his shot horribly wide.
And the Dons youngster would soon regret that mistake as May put St Johnstone back on level terms after 61 minutes.
Aberdeen conceded a needless corner and despite Barry Robson clearing the initial delivery, James Dunne knocked the ball into the box for May who turned one way then the other before rifling a low shot past Langfield from an improbable angle.
Robson should have hit back for Aberdeen immediately though when he was picked out by Willo Flood but his weak header dropped into Mannus' grasp. McGinn then headed wide at the back post as Robson sought to make amends with a terrific delivery from wide on the right.
With the game entering the final 10 minutes, there was no sign that a winner was on the cards.
But May, so often Saints talisman this season, worked a clever one-two with strike partner Steven MacLean on the edge of the box before galloping through the Dons defence to smash a low shot underneath Langfield.
The 83rd-minute goal sent the Perth support among the 19,057 crowd into raptures but there was soon a loud cry for a penalty from the Aberdeen support as McGinn and Frazer Wright collided at the other end.
Referee Willie Collum waved away the appeal and Saints saw out the remaining minutes to set up what could potentially be the biggest date in the club's history.