The Commons Public Accounts Committee (PAC) said the scale of the losses was worryingly high, and expressed concern about the impact the deficit was having on the public finances.
The Whole of Government Accounts (WGA), drawn up by the Treasury, show that in 2011-12 about £13.2bn had to be written off due to fraud and error. However, the figures do not include local government or public corporations, and the National Fraud Authority estimates the true cost of fraud to the public sector was £20.6bn.
Meanwhile, HM Revenue and Customs said the "tax gap" - the difference between the taxes it is meant to receive and those it manages to collect - was £35bn.
PAC chairwoman Margaret Hodge said: "Taxpayer losses due to fraud and error are worryingly high. It is staggering that, in one year, the public sector was defrauded of more than £20bn and the tax gap rose to £35bn."
The committee called on the Treasury to develop a strategy to tackle fraud and error in the public sector.
The committee also voiced concern about how the figures in the WGA were complied and presented. The WGA - published for the first time last year and covering the combined financial activities of 3,000 bodies - is intended to provide the most comprehensive accounting picture of the public sector in the UK.
However, the PAC said its credibility was undermined by the "poor quality" of some data, and the Treasury needed to do more to explain discrepancies between some of the WGA figures and those produced by the Office for National Statistics.