Mr Kerry, on his first trip to a Muslim nation since taking office, is meeting Turkish leaders for talks meant to focus on Syria's civil war and bilateral interests from energy security to counter-terrorism.
But the comment by Turkish Prime Minister Tayyip Erdogan at a UN meeting in Vienna earlier this week was condemned by his Israeli counterpart, the White House and UN Secretary-General Ban Ki-moon, and has clouded Mr Kerry's trip.
"We not only disagree with it, we found it objectionable," Mr Kerry told a news conference with Turkish Foreign Minister Ahmet Davutoglu, saying he raised the issue directly with Mr Davutoglu and would do so with Mr Erdogan.
"That said, Turkey and Israel are both vital allies of the United States and we want to see them work together in order to be able to go beyond the rhetoric and begin to take concrete steps to change this relationship," Mr Kerry added.
Washington needs all the allies it can get as it navigates the political currents of the Middle East, and sees Turkey as the key player in supporting Syria's opposition and planning for the era after President Bashar al Assad.
However, the collapse of Ankara's ties with Israel has undermined US hopes that Turkey could play a role as a broker in the broader region.
Mr Erdogan told the UN Alliance of Civilisations meeting in Vienna on Wednesday: "Just as with Zionism, anti-Semitism and fascism, it has become necessary to view Islamophobia as a crime against humanity."
Mr Erdogan's caustic rhetoric on Israel has in the past won applause from conservative supporters at home, but raised increasing concern among Western allies.
Ties between Israel and mostly Muslim Turkey have been frosty since 2010, when Israeli marines killed nine Turks in fighting aboard a Palestinian aid ship that tried to breach Israel's blockade of the Gaza Strip.
"If Israel wants to hear positive statements from Turkey it needs to reconsider its attitude both towards us and towards the West Bank," Mr Davutoglu told the news conference.
Moshe Kantor, president of the European Jewish Congress, said: "Erdogan's comments about Israel have become progressively more worrying. This type of rhetoric is pushing Turkey further away from the West and Europe and closer to Iran."
Turkey's relations with the US have always been prickly, driven more by a mutual need for intelligence than any deep cultural affinity. And Mr Erdogan's populist rhetoric, sometimes at apparent odds with US interests, is aimed partly at a domestic audience wary of Washington's influence.
However, the two have strong common interests. Officials said Syria would top the agenda in Mr Kerry's meetings with Mr Erdogan and President Abdullah Gul, building on the discussions in Rome on Thursday between 11 European and Arab nations within the Friends of Syria group. After the meeting, Mr Kerry said the US would for the first time give non-lethal aid to the rebels.