Miller, 47, is bowing out of the BBC1 series after two years playing strait-laced British police officer Richard Poole.
He told the Radio Times that he promised himself he would get into the sea every day while filming in Guadeloupe, where the show is set.
But Miller told how he lost "every scrap of dignity" in the water when he first joined the series, after ignoring a warning sign.
"I had a terrible experience at the beach barbecue the crew held for me when I arrived. There was a sign - in French - that said on no account go into the water on this beach," he told the magazine.
"I remember thinking, 'What absolute rubbish, it's clearly for tourists'.
"I wandered out there casually to cool my feet and all of a sudden it was like a man under the sand grabbed my ankles and threw me over.
"I was rolled into the waves and thrashed to within an inch of my life, while frantically waving for help. Nobody noticed.
"I lost everything on me - my glasses, my wallet... every scrap of dignity I had.
"I finally found my feet and limped back to my room to get changed. When I returned to the barbecue the producer asked, 'What on Earth kept you?' I never admitted how stupid I'd been."
He also admitted another mishap, telling the magazine: "I went snorkelling on one of the reefs over on Petit Havre, off the main island, and it was stunning, like a lagoon... until I saw a five-foot reef shark. I didn't react in a particularly macho way.
"Reef sharks probably think that one of our defence mechanisms when under attack is to defecate in the water."
The comedian and actor said that he decided to step down from Death In Paradise because his baby son, Harrison, had no idea who he was when he returned to the UK.
But he joked that little had changed since coming home.
"He still doesn't know who I am. I see him every day and he still cries when I pick him up. Turns out it's nothing to do with me being away, he just doesn't like me!," he said.
Miller, who is being replaced by actor Kris Marshall playing a new detective in the lead role, collapsed from heat stroke while filming the first series, but said he was now better prepared.
Referring to his wardrobe, he said: "In any given shot, if there's an item of my clothing you can't see, I'm not wearing it - will that put people off watching, do you think?"
Marshall told the Radio Times that he had been kidnapped while making a film, Oka!, in the Congo in 2011.
"I actually got kidnapped while making a film in the Congo jungle, where army warlords are ten-a-penny," he said. "I mean, I use the word kidnapped very loosely.
"Basically, the army used to turn up at our unit base and take the producer away whenever they were bored and we would have to pay to get him back.
"And then one day, when they found out I was the main actor on the job, they took me instead.
"So, I say kidnapped, but it involved me being led into the back of a jeep, driven away to some army/police office by drunk guys with machine guns standing around for about three hours, before they bought me back for about 200 dollars."
Asked how much was paid for the producer, he replied: "250!"