His son Adam said the American star, also well known for his role in The Odd Couple, had a great life and he had "enjoyed every moment of it, and he would encourage others to do the same".
Klugman's wife, Peggy Compton, was at his bedside in Los Angeles. Although the exact cause of death was not known, his health had been in general decline in recent years.
Cancer had robbed him of his voice in the 1980s, although he had taught himself to speak again.
He was still working and was not thought to be suffering from the disease at the time of his death.
His raspy voice helped make him the mainstay of US television shows during the 1970s and early 1980s.
He played the sloppy sports writer Oscar Madison in The Odd Couple alongside Tony Randall.
His son added: "He went very suddenly and peacefully ... he was there one minute and gone the next."