Students accused the controversial Respect MP of racism as he left the event, organised by Christ Church College.
Footage of the scene, uploaded on to the internet, shows that Mr Galloway's opponent, Eylon Aslan-Levy, was half-way through his remarks when the Bradford West MP interrupted him.
He asked the third-year student: "You said 'we'. Are you an Israeli?" Mr Aslan-Levy answered: "I am, yes."
Mr Galloway then stood up and replied: "I don't debate with Israelis, I've been misled, sorry."
To shock and then laughter from the audience, Mr Galloway walked out of the lecture room where the event was being held. Watch the video below.
He had been due to speak in favour of the motion "Israel should withdraw immediately from the West Bank".
Paul Charney, chairman of the Zionist Federation, which campaigns for a permanent homeland for the Jewish people, said: "George Galloway's refusal to debate with an Israeli student is absolutely disgraceful and clearly xenophobic. The incident shows a man consumed by hatred of Israel, who undermines freedom of speech as he fears his lies and demonisation of Israel being exposed."
In a statement, the British Board of Jewish Deputies said: "British universities are rightly cherished as places where debate and discussion can thrive. Yet it would seem that those intent on demonising the State of Israel would rather crush freedom of speech in pursuit of their own agenda of blind hatred.
"George Galloway's refusal to debate with an Israeli at Christ Church College, Oxford, and attempts by violent protesters to disrupt a talk by the deputy ambassador of Israel at the University of Essex, reveal the true nature of the anti-Israel movement in the UK."
The board's vice-president, Jonathan Arkush, added: "All universities should take steps to ensure that people are safe and secure to visit our campuses, irrespective of their race, faith or nationality."
Mr Galloway has long been known as an opponent of Israel, denouncing its policy towards the Palestinians as "apartheid".
After the debate, he said he had refused to engage with Mr Aslan-Levy because he was an Israeli. "The reason is simple: no recognition, no normalisation," he said.
"Just boycott, divestment and sanctions, until the apartheid state is defeated. I never debate with Israelis."