Hull had stunned the Gunners as they went 2-0 up inside eight minutes through James Chester and captain Curtis Davies, who both capitalised on poor defending.
Arsenal, though, rallied and cut the deficit through a superb 30-yard free-kick from Santi Cazorla. After several Arsenal penalty claims were turned down by referee Lee Probert, including notably a pull by Tom Huddlestone on Olivier Giroud, the equaliser came on 71 minutes when Laurent Koscielny knocked the ball in from a corner.
Allan McGregor saved well from Giroud as the match went into extra time after the France forward sent a header against the crossbar. But it was left to Ramsey to complete a remarkable turnaround by crashing in a loose ball from the edge of the penalty box on 109 minutes.
It handed Arsene Wenger his first success since 2005, and might just convince him to sign a new contract. There was, though, more drama as goalkeeper Lukasz Fabianski - in most likely his last appearance for Arsenal - rushed out and was beaten to a loose ball on the left by Sone Aluko.
The Hull forward looked to sweep it into an empty net, but agonisingly for Tigers fans massed behind the goal, it rolled along the line and wide.
"We wanted to make history tonight and win the game, and we made history in both ways - how not to start a cup final and how to come back," said Wenger.
"I'm very happy for the players because I think it is a turning point in the lifetime of this team, and to know they can win in that way was very important today."
Wenger added: "I've praised many times the spirit of this team and I'm very proud of that today. Congratulations to Hull, they played fantastic for the whole game. We won it in the end but they played very well."
Few would have predicted a flying start for Hull, but that is what arrived as the Tigers swept into the lead on four minutes.
A corner from the right by Stephen Quinn was floated out to Huddlestone.
His scuffed shot bounced through to Chester, who flicked it back into the bottom left corner just past Fabianski. Rather than respond, the Gunners quickly found themselves 2-0 down. Again it was a delivery into the penalty box which caused problems, this time from Quinn out on the left.
Alex Bruce's header was deflected onto the post by Fabianski and Davies reacted quickest to smash the ball home from a tight angle.
It was almost 3-0 on 13 minutes when another corner was headed back in by Bruce and as the ball looked to be dropping in, but Kieran Gibbs knocked it off the line.
After 17 minutes, Cazorla hauled Arsenal back into the game. He was tripped 30 yards out by Bruce and lashed the free kick into the top right corner.
The Gunners had renewed belief: Mesut Ozil just failed to connect with a ball through the Hull six-yard box from Lukas Podolski and Giroud's shot was blocked, before Podolski's bullet follow-up was deflected behind.
Arsenal continued to ask questions after the restart, with Ozil's clever dragged cross through the six-yard box just too far ahead of Podolski.
Giroud felt he should have had a penalty when he was tugged back by Huddlestone, but referee Lee Probert was not convinced.
Cazorla was the next Arsenal man to go down in the penalty area, as he turned Davies, but again the officials remained unmoved, to the anger of Wenger, but on 72 minutes, the Gunners were level. Yaya Sanogo's flick at the near post was blocked for a corner and the ball was swung into a crowded six-yard box where it hit two bodies and dropped to Koscielny.
Aluko replaced Quinn to give Hull impetus in attack and there was more incident before full time as Gibbs crashed the ball over the bar after it was dropped to him six yards out and Cazorla looked to have been nudged over, but the referee waved play on.
With neither side able to make a breakthrough, the tie went to extra-time, and Wenger sent on Jack Wilshere and Tomas Rosicky, for Ozil and Cazorla, for the last 15 minutes, Arsenal emerging victorious after Ramsey's instinctive finish.