The second of two crashes on the route from Ypres to Arenberg Porte du Hinaut led to Froome withdrawing to the Team Sky support car.
He wrote on Twitter: "Devastated to have to withdraw from this years TDF. Injured wrist and tough conditions made controlling my bike near to impossible. Thanks to the team & support staff for trying to get me through today. Wishing @richie-porte & @TeamSky the best for the rest of Tour!"
It has been reported Froome began the stage with a fractured left wrist, suffered when the 29-year-old tumbled on to the tarmac on Tuesday's fourth stage.
The stage, designed to commemorate 100 years since the start of the First World War, featured many of the cobbles used in the Paris-Roubaix one-day race nicknamed 'the Hell of the North', but torrential rain and wet roads served only to make the fifth stage even more challenging.
Organisers removed two of the nine cobbled sections because of the conditions but Froome's falls came before the first section of cobbles he had been dreading since the route was announced last autumn.
The sight of Froome, dominant in winning the 2013 Tour, grimacing by the roadside was reminiscent of Sir Bradley Wiggins' withdrawal with a broken collarbone in the first week of the 2011 Tour.
Froome crashed early on the route before a second crash with around 70 kilometres remaining ended his defence. He consulted the Team Sky doctor Alan Farrell and sports director Nicolas Portal before taking his place in the back of the team car.
Despite the atrocious conditions, which made for an epic day's racing won by Belkin's Lars Boom, Froome was the only withdrawal as the peloton was reduced to 193 riders.
Only two Britons are left - the Welshman Geraint Thomas (Team Sky) and England's Simon Yates (Orica-GreenEdge) - after Froome joined Mark Cavendish, the Manxman who injured his shoulder in a fall at the end of the first stage, in exiting the Tour. Thomas played a key role in supporting Richie Porte's challenge, with the Australian inheriting the lead role from Froome within Sky.
Porte finished 20th, 2min 11sec behind stage winner Boom to move up to eighth overall, 1min 54sec behind Astana's Vincenzo Nibali, who retained the yellow jersey with a third-placed finish.
Froome's absence means the Tour's wait for a first back-to-back winner since the disgraced Lance Armstrong won seven titles, all since removed, between 1999 and 2005, goes on. It is the first time since Bernard Hinault in 1980 that a defending champion has exited the Tour prematurely.
Porte wrote on Twitter: "Going to miss having @chrisfroome around for this TdF but sure he'll be back next year stronger than ever #lionheart"
Sir Dave Brailsford, the Team Sky principal, said on letour.com: "He's out of the race. Obviously it's devastating for Chris and for the team. We knew it was going to be a tough race. We really believed in Chris and his ability to win this race. But it's not to be this year.
"I'm sure he'll be back. It's part of this sport: sometimes you get knocked down. He was fit to start; he was in pain; there's no denying it. But the injury he sustained today was on the other side.
"In Richie Porte, we have a very capable, viable leader in the team. When you have a day like today, when you have a setback you have to roll ahead and go again, you have recalibrate your goals and you go again. Richie Porte came on the Tour to be the team leader No.2 and he showed great ability to ride the cobbles the way he did."
Brailsford backed Porte for the challenges ahead. "The fight in the mountain lies ahead," he added. "Richie is in great shape. He had a slower start to the season than usual but he's fresh. The reason we decided to call him as our second leader was his climbing abilities."
Froome's supremacy in the mountains earned him the 100th Tour title in 2013, but a duel with two-time winner Alberto Contador will now have to wait until 2015, at least.
Contador's Tinkoff-Saxo team wrote on Twitter: "@chrisfroome we wish you a swift recovery #TDF. We were looking forward to the battle in the mountains @letour"
Contador endured a challenging day, finishing 37th, 2:54 behind, to fall to 19th overall, 2:37 adrift ahead of Thursday's 194km sixth stage through Champagne country, from Arras to Reims.
The Spaniard added on Twitter: "From here I want to give support to @chrisfroome, months of work and everything gone. A big loss for the Tour. See you here next year."