The Treasury will sell 6% of the bank to big institutions, cutting its stake to 32.7% from 38.7%, which could net millions of pounds of profit for the taxpayer.
Chancellor George Osborne kicked off Lloyds' re-privatisation almost five years after its disastrous acquisition of
Loading article content
Halifax Bank of Scotland left the bank needing a £20 billion bailout.
Shares in Lloyds have soared to three-year highs in recent days on anticipation of an imminent sale. They closed at 77.36p yesterday, above the 61p level at which the Government says it would break even on its bailout.
Based on the closing share price, the sale would recoup £3.31 billion for the taxpayer.
UK Financial Investments (UKFI), which manages taxpayers' banks stakes, did not say what price it will sell Lloyds shares for.
Earlier this year Mr Osborne said the mortgage lending giant was ready to begin its return to private ownership.
A Treasury spokesman said: "UK Financial Investments today advised the Chancellor it would be appropriate to begin the process to sell part of the Government's shareholding in the Lloyds Banking Group. The Chancellor agrees with that advice and has authorised the process to begin.
"We want to get the best value for the taxpayer, maximise support for the economy and restore them to private ownership. The Government will only conclude a sale if these objectives are met."
UKFI pulled the trigger on the sale of 4.28 billion shares after the stock market closed.
The public will not have a chance to buy Lloyds shares in the first wave, as UKFI will sell the shares to large institutions - although the public is expected to be able to participate in future share sales.