The broadcaster defended the show's record for inviting women guests and pointed out the proportions were higher than those for many areas of public life.
Dimbleby's comments come after O Briain turned on BBC1's political discussion show in a Radio Times interview after being asked about a recent BBC decision to ban all-male line-ups for comedy panel shows.
O Briain said: "It's remarkable that this amount of time is spent debating women on comedy shows rather than, say, Question Time."
But stung by the suggestion that his show had done anything wrong, Dimbleby said: "Seeking to deflect attention from the gender imbalance on comedy panel shows Dara O Briain suggests that critics look instead to Question Time. He has picked the wrong target.
"Of the 104 panellists in the present run 46 have been women (44%). Given that we invite guests prominent in public life this is not a bad record. Only 18% of the cabinet is female. Only 23% of MPs are female. Only 20% of FTSE board members are female. Our aim is to achieve a 50/50 balance but British public life has to catch up before we can."
O Briain had said scrapping all-male comedy panels would mean viewers would see any female guest as a "token woman".