Marshall Ronald is seeking the payout from the Duke of Buccleuch following the recovery of the Leonardo Da Vinci masterpiece Madonna of the Yarnwinder.
Mr Ronald, who was cleared of criminal charges including conspiracy to extort money in 2010 in a trial at the High Court in Edinburgh, said he had acted in good faith and added: "I don't accept any illegal acts."
Mr Ronald, who is conducting his own case, told the Court of Session in Edinburgh: "Buyback is a lawful and commercial method of recovering paintings." Andrew Young QC, for the Duke, said: "Properly viewed, what the pursuer (Mr Ronald) is averring here is an attempt to extort a sum of money."
Mr Ronald, 57, of Upholland, Lancashire, wants his action to go to a hearing of evidence while lawyers acting for the Duke are seeking to have it dismissed.
The painting, worth an estimated £50 million, was stolen from Drumlanrig Castle, near Thornhill, in Dumfriesshire in 2003 and recovered four years later.
Mr Ronald claims that following discussions with an undercover police officer, John Craig, it was agreed he would be paid £4.25m for the return of the painting. However the Duke maintains there was no contract.
Lord Glennie reserved his decision in the case following the hearing.