Borrowing rose to a higher-than-expected £13.3bn though it edged down by £200m, or 1.5 per cent, on an underlying basis.
Experts pointed out this was much narrower than the 11 per cent fall in the deficit for the current financial year, pencilled in by the independent Office for Budget Responsibility (OBR).
Samuel Tombs, of consultancy Capital Economics, said the figures "contain tentative signs the coalition may be beginning to struggle to bring down the deficit in line with the fiscal plans."
Tax receipts had "continued to grow disappointingly weakly", he added. "So, while the economic recovery may now be fairly strong, it still appears to be struggling to have much of an impact on the borrowing numbers."
Treasury coffers were partly boosted by surging stamp duty revenues, which were 28 per cent ahead at £1.2bn. This was primarily driven by higher receipts from land and property transactions rather than share deals to which it also applies.
The £13.3bn borrowing figure for May was £4.6bn up on the same month last year.
A Treasury spokesman said: "Today's figures continue to be in line with the budget forecast."