UK Financial Investments (UKFI), which manages the taxpayers' stake in the bank, said the Treasury's stake will fall to about 32.7% from 38.7%.
UKFI pulled the trigger on the sale of 4.28 billion shares after the stock market closed.
Based on today's closing share price, the sale would recoup £3.31 billion for the taxpayer after Lloyds' £20 billion bailout and part-nationalisation in 2008.
Shares in the bank closed at 77.36p, above the 63.1p level at which the Government says it would break even.
UKFI will sell the shares to large institutions, but did not say what price it will sell them for, or what the sale will make.
The sale was authorised by Chancellor George Osborne, who earlier this year said the mortgage lending giant was ready to begin its return to private ownership.
A Treasury spokesman said: "UK Financial Investments today advised the Chancellor it would be appropriate to begin the process to sell part of the Government's shareholding in the Lloyds Banking Group. The Chancellor agrees with that advice and has authorised the process to begin.
"The Chancellor set out the Government's objectives for its shareholdings in the banks at the Mansion House speech earlier this year.
"We want to get the best value for the taxpayer, maximise support for the economy and restore them to private ownership. The Government will only conclude a sale if these objectives are met."
UKFI added it will not sell more shares for a further three months after completing the share placing.