Pittodrie skipper Russell Anderson hit the post and both sides had penalty claims turned down in regulation and then extra time but neither side could find the breakthrough.
Caley striker Billy McKay had the first penalty saved by Jamie Langfield and Greg Tansey ballooned his over the bar before Nicky Ross and Aaron Dorran scored for John Hughes' side.
However, after Barry Robson, Nicky Low and Scott Vernon had all slotted in for the Dons, it was down to former Caley striker Adam Rooney to blast his spot-kick past Dean Brill to spark wild celebrations among the 40,000 Dons fans who had taken over the east end of Glasgow and outnumbered their Caley counterparts by around four to one.
It was the perfect end to the week for Dons boss Derek McInnes, who, along with assistant Tony Docherty, signed a new deal in midweek taking them up until 2017. It also means the League Cup is on its way to the Granite City for the first time since the Dons beat Dundee in the 1995 final at Hampden.
McInnes had earlier been dealt a blow when attacking midfielder Peter Pawlett failed to recover from a hip injury and there was further disruption to his plans when he had to make his first substitution with only five minutes gone.
With just seconds played, Caley defender Josh Meekings left Jonny Hayes prostrate and requiring treatment with the first challenge of the game and although it looked like the former Inverness wide-man had recovered, he lasted only a few more minutes before being replaced by Cammy Smith.
There first moment of real drama came in the 23rd when Anderson sent an shot crashing through a packed penalty area and off the post, the ball looking like it was heading in until whipped away by Inverness skipper Richie Foran.
Moments later, Rooney, who had made his name at Inverness, volleyed over the bar from close range.
There was an Aberdeen claim for a penalty in the 35th minute when Rooney went to ground after stealing the ball from the toes of Meekings but referee Steven McLean was unimpressed.
Brill was called into action when he had to save a long-distance effort from midfielder Robson four minutes later but in fact, amid the honest toil on show, neither keeper was seriously worked in the first half.
There was a comedic moment in the 58th minute when Caley's Marley Watkins went on a great run from right to left only to mis-kick his shot and fall over. However, more seriously for the Dons fans, Caley midfielder Tansey then tested Langfield with a free-kick from distance, another indication that Hughes' side were gaining in belief.
As nerves jangled, it looked like one goal would win it and it almost came in the 78th minute when Aberdeen midfielder Ryan Jack fired in a powerful shot from 25 yards which Brill could only deal with at the second attempt with Rooney ready to pounce.
Three minutes from time Foran complained bitterly to referee McLean that he had been pushed at a corner by Dons defender Andrew Considine before Aberdeen raced to the other end where a last-ditch challenge from Meekings denied Niall McGinn.
Both sets of players understandably looked tired as the game entered extra ime.
Aberdeen remained on top but Langfield pulled off a decent save from Caley substitute Doran just before the first period ended with the resultant corner again coming to nothing.
Tension increased further after the players swapped ends but penalties seemed inevitable and when they arrived Rooney, ultimately, was the Aberdeen hero.