It comes as French company Atos, one of the providers, revealed it was seeking to pull out of its contracts to carry out fit-to-work assessment for the disabled and terminally ill.
Mr Penning told the Commons: "As we bear down the new benefit and the new policies there will be issues within the department, issues within the contractors and issues not only with people giving advice to claimants but claimants themselves."
Answering MPs who were complaining about the inefficiency of suppliers in making their decision, he said : "There are contractual requirements of both providers (Atos and Capital) and if they are not meeting that they will be financially penalised.
"But I also have to say that there are internal delays within the Department for Work and Pensions, so I'm not going to pass all the buck onto them."
Mr Penning's admission came after members criticised the lengthy delays that thousands of individuals had experienced in waiting to hear the outcome of their assessments and whether or not they had been successful with their Personal Independence Payments (PIP) applications; in some cases, claimants have been waiting as long as six months.
Labour's shadow education minister Kevin Brennan said MPs were being inundated with complaints that were "clogging up a lot of MPs' surgeries as a result".