Theresa May is under renewed pressure to abandon Brexit talks with Labour amid warnings she risks losing the “loyal middle” of the Tory Party if she gives ground on a customs union.

Some 13 former ministers, together with the chairman of the backbench 1922 Committee, Sir Graham Brady, have written to the Prime Minister urging her not to concede Labour’s key demand, The Times reported.

READ MORE: Holyrood does not express 'definitive view' on Scotland's future, David Mundell insists 

The signatories were said to include Gavin Williamson, who she sacked as defence secretary, as well as potential leadership contenders Boris Johnson and Dominic Raab.

BrexitSacked defence secretary Gavin Williamson is among the ex-ministers warning against a customs union (Peter Byrne/PA)

Their intervention comes as Mrs May prepared to brief senior ministers on the state of the talks – which began in April – at the weekly meeting of the Cabinet in Downing Street on Tuesday.

Meanwhile, the PM’s Brexit negotiator, Olly Robbins, was said to be travelling to Brussels for talks with EU officials.

The BBC reported that discussions will include whether the political declaration could be changed – and how quickly – if it allowed the Government and Labour to break the impasse.

The two sides’ negotiating teams, led by David Lidington, Mrs May’s de facto deputy, and shadow Brexit secretary Sir Keir Starmer, met again on Monday.

However a group of former ministers, who also include Sir Michael Fallon and Maria Miller, warned the Prime Minister stands to lose the support of more Tory MPs in the Commons than she gains Labour backers if she breaks her “solemn promise” not to enter a customs union.

READ MORE: Arlene Foster: Second Brexit referendum ‘places democracy at risk’ 

The group, who made the point they all backed the Withdrawal Agreement in the last Commons vote in March, said Mrs May could not bind her successor to a deal so any agreement with Labour was likely to be “at best temporary, at worst illusory”.

“We believe that a customs union-based deal with Labour will very likely lose the support of Conservative MPs, like us, who backed the Withdrawal Agreement in March … and you would be unlikely to gain as many Labour MPs to compensate,” they wrote, according to The Times.

“More fundamentally, you would have lost the loyal middle of the Conservative Party, split our party and with likely nothing to show for it.

“No leader can bind his or her successor so the deal would likely be at best temporary, at worst illusory.”

Their intervention comes amid growing frustration among Tory MPs, both with Mrs May’s leadership of the party and the continued Brexit deadlock.

European <a href=Parliament election">Nigel Farage fronts the Brexit Party (Danny Lawson/PA)

Many fear they are heading for a mauling at the hands of Nigel Farage’s Brexit Party in the European elections on May 23 having already suffered heavy losses in the English local council elections earlier in the month.

Following Monday’s talks with Labour, a No 10 spokesman said they were still trying to find an agreed way forward that would allow Britain to leave the EU with a deal.

“In preparation for an update to Cabinet, today’s meeting took stock across the range of issues discussed in talks over the last few weeks,” he said.

“We continue to seek to agree a way forward in order to secure our orderly withdrawal from the EU.”

Meanwhile, Jeremy Corbyn has come under fire from both pro-Remain and pro-Brexit MPs in his party amid complaints that confusion over Labour’s position cost them votes in the council elections.

At the weekly meeting on Monday of the Parliamentary Labour Party, the pro-Remain MP Peter Kyle told the Labour leader their message on a second referendum had been too complicated and had left voters perplexed.

“I urge you to simplify our policy so people realise we are speaking with absolute sincerity,” he told Mr Corbyn.

The pro-Brexit MP John Mann called on Mr Corbyn to give Labour MPs a free vote on Brexit issues to reflect the divisions within the party and the country.

“Labour voters are divided in a very big way,” he told him.

“If you don’t get this right you cannot be prime minister.”

Mr Corbyn acknowledged the MPs’ “frustrations” and said he understood the need to simplify their message, telling them: “I get that.”

A Labour source said: “Our message is about bringing the country together.

“That means people who voted Leave and people who voted for Remain.”

Earlier, Sir Keir Starmer and Labour deputy leader Tom Watson threw their weight behind calls for any Brexit deal to be put to a second referendum – something Mr Corbyn has resisted.