The case was called at Inverness Sheriff Court yesterday but dismissed on a motion from the Bank of Scotland's lawyer, with no costs awarded to either side, after Whyte agreed to hand Castle Grant near Grantown-on-Spey over to the bank.
He and his then wife Kim bought the castle for £720,000 in 2006 with a 110% mortgage, which carried a monthly repayment charge of £7000 a month
However, their marriage broke up and he moved out of the castle. The bank launched its civil action to recover its money early in 2013.
A spokesman for the Bank of Scotland said Whyte was a customer and the bank would not comment on any customer's affairs.
People living nearby say removal staff have been arriving at the castle and taking items away in recent days, with many of its contents being taken to London.
A former employee at the castle was quoted saying: "If Whyte could dismantle the castle, he'd have that away as well. There's been a race against time. He knows a repossession company will be instructed by the end of the week to replace the locks and secure the grounds."