The Hollywood star spoke out after the London Mayor criticised him for suggesting that Britain should return the Elgin Marbles to Greece.
Accusing Clooney of losing his own marbles, Mr Johnson claimed he was "advocating nothing less than the Hitlerian agenda for London's cultural treasures".
But Clooney, whose new film The Monuments Men is inspired by the true story of a team of soldiers on a mission to rescue valuable artwork stolen by the Nazis during the invasion of Europe, has now hit back.
He told the Huffington Post: "I'm a great fan of the mayor, and I'm sure my right honourable friend had no real intention of comparing me to Hitler.
"I'd chalk it up to a little too much hyperbole washed down with a few whiskies. I've found myself in the same spot a time or two so I hold no ill will.
"When it comes to real facts, not imagined history, you need only to look at the Unesco rulings that have been agreed to by all parties.
"An occupying nation can't sell off the national heritage of the country it occupies.
"More relevant is the fact that the Parthenon Marbles were chipped away from the Parthenon by the occupying Turks and sold. It was a single monument broken into bits. It would be as if the statue of David's head were sold to England. His arm to the Vatican. And his torso to the Met."
The award-winning actor added: "There are many pieces in nearly every country that this conversation should take place. The best place to start would be at the most obvious object. When polled the British people are overwhelmingly in favour of their return.
"The rest of the world follows suit. If you want to deal in facts. Those are the facts. But maybe it's just easier to compare me to Hitler."
Clooney was responding to comments made by Mr Johnson in The Daily Telegraph.
"Someone urgently needs to restore George Clooney's marbles," the mayor said earlier this week.
"Here he is plugging a film about looted Nazi art without realising that Goering himself had plans to plunder the British Museum.
"And where were the Nazis going to send the Elgin marbles? To Athens! This Clooney is advocating nothing less than the Hitlerian agenda for London's cultural treasures. He should stuff the Hollywood script and stick to history."
The row began when Clooney was asked about the sculptures, taken to Britain from the Parthenon in the 19th century, during a press conference to promote the new film, which he wrote, directed and stars in - although then he mistakenly referred to it as the "Pantheon".
The British Museum has rejected repeated requests to send the marbles home, countering that it legally owns the collection and that it is displayed free of charge in an international cultural context.