The renowned athletics commentator worked for the BBC for almost 50 years, covering 11 summer Olympic Games, his final one in Sydney in 2000.
He also covered six football World Cups.
His family said in a statement: "We regret to announce the death of David Coleman OBE, after a short illness he died peacefully with his family at his bedside."
Coleman, who was born in Cheshire and joined the BBC in 1954, presented Grandstand and also hosted quiz show A Question Of Sport from 1979 to 1997.
He also became affectionately known for on-air gaffes, giving his name to the term Colemanballs.
Click here to read some of his most famous slip-ups.
He was awarded an OBE in 1992 and retired from the BBC in 2000.
Later that year he became the first broadcaster to receive an Olympic Order medal to recognise his contribution to the Olympic movement.
Fellow commentator Brendan Foster, the former Olympic 10,000 metres bronze medallist, hailed Coleman as the "greatest sports broadcaster that ever lived".
Foster said: "David enriched so many lives and that was down to his brilliant commentary and presentation at all the major sporting events of the world.
"In my view, everybody had a David Coleman quote they could use. It could have been about Pele, Charlton, Toshack or Keegan, or just 'one-nil'.
"It was a privilege to know him, to have him commentating on races during my career, to work with him and to call him a friend."
British Athletics chairman Ed Warner said: "David has been the voice of some our most memorable moments over the years. A truly iconic broadcaster."
Gary Lineker paid his own tribute.
The Match of the Day host and former England striker wrote on Twitter: "Sad to hear, David Coleman has died. A giant of sports broadcasting. Brilliant, gifted, precise and concise. Much more than 'one-nil' #RIP."
Coleman, whose first Olympics was in Rome in 1960, began presenting Grandstand in 1958 and worked on the magazine programme for 10 years.
In 1971 he became the BBC's senior football commentator, covering five FA Cup finals before handing over to John Motson in 1979.
BBC director-general Tony Hall called Coleman "one of this country's greatest and most respected broadcasters", while the corporation's director of sport Barbara Slater described him as "a giant in the sports broadcasting world, an iconic and hugely respected figure".
Steve Cram, another athlete turned BBC broadcaster, credited Coleman with helping him in the early stages of his career.
Coleman commentated on many of Cram's races and his tussles for world middle-distance supremacy with fellow Britons Sebastian Coe and Steve Ovett.
Cram said: "When I first came into the British team as a youngster, I would watch back my races and I could tell from his commentaries that he knew what he was talking about.
"When I met him at major championships, such as the Olympics in Moscow in 1980, he would say things that turned out to be incredible helpful, such as advice on travel and how to deal with the media.
"He had a reputation within broadcasting for being tough and demanding, but I always found him an incredibly generous bloke.
"Yes, he had high standards but I think that came from his athletics background. Broadcasting, like athletics, is in a sense about performance and he wanted to produce the best he could. He used to always tell me that I should endeavour to work with the best to get the best results."
Former Home Secretary David Blunkett described the commentator as "a thoroughly decent guy", having been quizzed by Mr Coleman 45 years ago on BBC1's Feedback show.
Mr Blunkett said: "He made me feel at home in my first ever TV interview 45 years ago when I found myself bizarrely confronting a very young David Dimbleby on the programme Feedback, which David Coleman was chairing.
"David Coleman had to deal with a man who couldn't see talking about a film which David Dimbleby had produced and which had caused enormous controversy by displaying dead and naked bodies. Why I ever wrote in I shall never know, but it was certainly a way of being blooded in terms of future interviews over the past 45 years.
"I know that as well as his family and friends, many of us will mourn him as someone who represented the best in broadcasting and of decency in public life."
Prime Minister David Cameron wrote on Twitter: "Sad to hear David Coleman has died - the voice of BBC Sport for as long as I can remember."
Former BBC colleague Barry Davies said Mr Coleman was "the master who set the standard for sports broadcasting on television".
Mr Davies said: "He had such authority in his voice which could bring even the most mundane event to life. And at the big events he was superb.
"I feel privileged to have known him, worked with him and occasionally stood in for him when he was in his prime."