Ally McCoist's men had come out on top against Clydesdale Bank Premier League opposition in the last round of the competition when they dumped then league leaders Motherwell.
But Caley Thistle proved to be too strong, with Andrew Shinnie, Gary Warren and Graeme Shinnie all on the scoresheet.
Rangers named an unchanged team from the one who defeated Clyde in the Irn-Bru Third Division at the weekend for the clash at Ibrox in front of a crowd of 28,033.
The Highlanders also opted for the same 11 players who stretched their unbeaten run to seven games with a draw against St Johnstone on Saturday.
Rangers were out of the traps quickly and Dean Shiels sent an angled drive across the face of goal and just past the post as the home side pushed for an early opener.
But Inverness also looked up for the tie and a full-stretch Neil Alexander just managed to gather from an Aaron Doran shot, before blocking low at the near post from Billy McKay.
Having survived the double scare, Rangers set about cranking up the pressure and Lewis Macleod forced a decent save from Antonio Reguero.
The Inverness goalkeeper then did well to deny Andy Little's strike from 10 yards with an outstretched foot after fellow Northern Irishman Shiels teed up the effort.
Macleod had another go at breaking the deadlock with a powerful shot from distance that dipped just over the crossbar.
For all the home side's pressure, it was Inverness who surged into the lead after 27 minutes through Andrew Shinnie.
The former Rangers youngster pounced on a pass from Doran before rifling past the grasp of Alexander to hand the visitors the advantage.
At the other end, the home crowd screamed for a penalty when Shiels tumbled over Reguero in the box but there was no spot-kick from referee Steven McLean.
Rangers then passed up the chance to level the scoreline before the break when Chris Hegarty threw himself in front of Lee Wallace's free-kick only to nod agonisingly over.
Ross Perry was swapped for Anestis Argyriou at the break before Lee McCulloch squandered a great chance to haul Rangers level when he blasted wide from a lovely Little ball.
Hopes of a comeback were dealt a blow when the Highlanders doubled their lead after 59 minutes.
Graeme Shinnie floated a corner into the box and the Gers defence failed to react as Warren was allowed a free header which he directed into the back of the net.
Barrie McKay and Kevin Kyle were thrown into the action for Shiels and Kyle Hutton as Rangers desperately tried to salvage the tie.
But a blunder by Hegarty almost gifted Inverness a third, with Alexander coming to the rescue to save one on one from McKay.
The Rangers goalkeeper was called into action again moments later, this time blocking with his legs from the same player as Caley Thistle pushed for the killer goal.
There was no way back for the Light Blues when the referee pointed to the penalty spot, ruling Argyriou had fouled Andrew Shinnie in the box.
Graeme Shinnie stepped up for the spot-kick and sent Alexander the wrong way after 78 minutes as Inverness booked their place in the last four.