United scored just once from their five kicks, with only skipper Darren Fletcher converting, as Sunderland's two successes proved enough to book a Wembley date with Manchester City on March 2.
In a season of utter misery for United boss David Moyes, it was yet another hammer blow.
His side has seemed to be on the verge of victory thanks to Jonny Evans' first-half goal as the tie ticked towards the final minute of extra time.
But when Phil Bardsley let fly from 20 yards, David de Gea inexplicably allowed the former United man's shot to slip through his grasp.
Amazingly, United earned themselves a reprieve in injury time as his players summoned up the spirit for which they are so famed and with virtually their last attack, Adnan Januzaj crossed for Javier Hernandez to bundle home from close range.
Moyes might have hoped fortune was to smile on him.
Even in a ridiculously bad shoot-out, De Gea's save from Adam Johnson meant Sunderland had only scored twice.
United could not match it though. And whilst it is anticipated Juan Mata will be having a medical tomorrow ahead of a club-record transfer from Chelsea, he will do so against a backdrop of even more gloom.
Sunderland boss Gus Poyet dedicated his side's dramatic win to the club's long-suffering fans.
"The feelings are for them," Poyet told Sky Sports 1, pointing to the visiting fans at Old Trafford
"A difficult season, nothing to enjoy, look at them, they deserve that. Let's see if we can go one more and win it."
Sunderland will be rank underdogs in the final against Manchester City, but, having knocked out Chelsea and now United, that is a status Poyet is happy with.
And asked if his side could win, the Uruguayan added: "No doubt, for sure."
Sunderland goalkeeper Vito Mannone, who saved the decisive penalty from Rafael, said: "Unbelievable, just unbelievable. I just want to celebrate with my team-mates.
"We deserved to get to the final."