The former cabinet minister, who now chairs the BBC Trust, said it was a "delusion" to think changing the way the BBC was governed would solve all its problems.
Recent scandals including revelations about Jimmy Savile's sex crimes and large pay-offs for senior staff have seen the BBC make the headlines for the wrong reasons.
Lord Patten said he hoped the focus would soon shift to the programmes made by the BBC, adding: "If that happens, I suspect arguments about governance would seem less important in the next year or two."
He told members of the Culture, Media and Sport Committee he was astonished at the amount of media "attention" over the BBC.
Director general Tony Hall was also giving evidence to the session, which will examine the fall-out from Savile and executive pay.