For decades Jamaican-born Scots academic Sir Geoff Palmer has described how Margaret Thatcher's intellectual mentor told him to go home and "grow bananas" half a century ago.

Now the distinguished scientist is being asked to prove that Tory legend Sir Keith Joseph - the father of British monetarism - really did make the racist remark in 1964.

Sir Geoff, 75, repeated his claims about Sir Keith, who died in 1994, on a Radio 4 programme this summer.

The broadcast sparked fury from supporters of the arch-monetarist, including the think tank set up by Sir Keith and Mrs Thatcher, the Centre for Policy Studies (CPS).

The CPS has now complained to the BBC about the broadcast, on the highly respected show, A Life Scientific.

A prominent Tory columnist, Charles Moore of the Spectator, is demanding that Sir Geoff prove the allegation.

Mr Moore, who revealed the complaint to the BBC, said the claims were reminiscent of a recent "tendency to let dreadful things", such as paedophile allegations, be said about dead Tories who cannot answer back.

Specifically, Sir Geoff, professor emeritus in the School of Life Sciences at Heriot-Watt University in Edinburgh, said that Sir Keith told him to "go back where you come from and grow bananas" during a university panel interview in 1964. Sir Geoff - whose real first name is Godfrey, did not get the post he had applied for. He went on to become one of the leading experts on cereal crops.

Sir Geoff said he had a clear recollection of the interview, which took place in Reading, and that he recognised Sir Keith, then a well-known politician.

He said: "I have been telling this story for years. This was before the Race Relations Act. I only brought it up on A Life Scientific because this is a programme about how to overcome things. You can't let things let this stop you."

Sir Geoff said Sir Keith, then a minister, was representing Whitehall for the funded position at university.

Mr Moore and other critics have cast doubt on this.

Mr Moore has written that Sir Keith did not have a role in the agriculture ministry.

He wrote: "The Centre for Policy Studies has now complained to the BBC about the programme. Sir Geoff might reflect that his account of what Joseph said would, if Sir Keith were alive, require the legal defence of justification - provable truth - to avoid losing a libel action. As a learned scientist with a regard for evidence, Sir Geoff surely feels a moral if not a legal duty to produce his."

Sir Geoff responded he was absolutely sure he had been insulted by Sir Keith. He said: "His name was on his name tag. They are implying I can't read. If that's the case, that's what I call offensive."

In a letter published in The Spectator, Sir Geoff also said that Mr Moore had wrongly assumed that the interview on the "incorrect assumption" had taken place in Nottingham, not Reading, and that Sir Keith could not have attended.

The BBC said it was unaware of the complaint from the CPS. The CPS did not comment.