The 77-year-old, famed for his booming delivery, said he was offered the role in the early years of his acting career, although he now says he would not hesitate in playing The Doctor.
Blessed went on to take a guest role in the programme in 1986 in a story called Mindwarp, playing the character of King Yrcanos, at a time when Colin Baker had been cast in the lead.
In an interview for Radio Times, he explained that he was approached to star in the show in the mid-1960s, suggesting he was asked to step in as a replacement for William Hartnell who launched the show, and was eventually succeeded by Patrick Troughton in 1966.
Blessed said: "After I was in Z Cars, the head of BBC serials took me aside and said, 'We're thinking of having a young Doctor Who and we'd like to cast you'. But it clashed with other things.
Asked what his reaction would be now, he said: "I'd jump at it."
Blessed - who is also fondly remembered for another sci-fi role, appearing as Prince Vultan in the movie Flash Gordon - appeared to be a little confused about The Doctor's surname, inaccurately suggesting the "Who" of the title was actually the character.
He said: "I think they need to re-examine his surname - "Who" is very oriental. It's about time they had an Asian actor as the Doctor. And a female one."
Doctor Who returns to BBC1 on August 23 with Peter Capaldi starring as the latest regeneration of The Doctor.
Blessed - who examines his family roots in a new series of BBC series Who Do You Think You Are? - said he still has a dream role he would like to take. "I'd like to play the last 20 years in the life of the England cricketer WG Grace, because I look exactly like him," he said.