Mr Whyte and his then wife bought Castle Grant near Grantown-on-Spey for £720,000 in 2006 with a 110% mortgage. But Mrs Whyte moved out after their marriage fell apart and Mr Whyte failed to pay the £7000-a-month mortgage for two years.
The Bank of Scotland launched a civil action to recover its money early in 2013.
The case was called at Inverness Sheriff Court last week, but dismissed on a motion from the Bank of Scotland's lawyer, with no costs awarded to either side, after Mr Whyte agreed to hand the castle over to the bank.
Kevin Maley, of Strutt & Parker, the estate agents handling the sale, said "Castle Grant is a wonderful property which is absolutely ripe for development, which could lead to it being one of the premier private houses in the UK."
Mr Whyte has restored parts of the first and ground floors of the castle, including the ballroom, and installed new features including a cinema room.
Other features include four bedrooms, a billiard room, library and kitchen.
The ancestral home to Clan Grant had been in the control of Manchester-based creditors Excel Securities before Mr Whyte bought it, after previous owners Ian and Helen Bailey-Scudamore reportedly ran into financial difficulties.
Before Mr Whyte bought the castle, it had been on the market for offers over £1m for more than a year.
Mr Maley added: "It is clear that a number of improvements have been made, however a large part of the property remains undeveloped and a significant amount of investment will be required to finish this off."
The 16th-century castle was first extended in 1752 by Ludovic Grant, who employed architects John and Robert Adam to modernise the building with a new granite facade.
The housekeepers' wing was accidently burned down in 1948, with the whole building stripped of its assets and left abandoned in the 1950s. It lay in ruins for 30 years before being put up for sale and bought by David Ward in 1983.