The presenter, who announced this month he is stepping down as presenter of the hit BBC1 series after a decade, told the Michael McIntyre chat show he got "tired doing live telly".
Speaking of the pressure of live television, Sir Bruce, 86, said: "It's the biggest show on television and I had to think very seriously about giving it up ... but I know it's better for me.
"It's better for me physically, I'll be able to spend more time with my wife and my family. I'll be able to have little breaks, which are good.
"It's live every week, and if I made a mistake with the autocue, this is what I was saying earlier on, it's big news: 'Oh he's 86 years old and past it'.
"I was getting all that which you don't need it. Who needs it?"
The all-round entertainer, who co-presented the celebrity dance contest with Tess Daly and performed a one-man show at Glastonbury last summer, said he would continue to appear on the Christmas and charity Strictly programmes, but that live television had become too strenuous.