The leaders met at Rev Mervyn Gibson's Westbourne Presbyterian Church in Belfast to try and forge a resolution following rioting and attacks on police over the past week.
Pro-Unionists are protesting over the decision to reduce the number of days the Union flag is flown over Belfast City Hall.
The Democratic Unionist Party's Robin Newton said a lack of engagement from protest organisers made it tough to see an end to the unrest.
He said: "We have to find a way out of this, but how we do it I don't know."
Mr Newton said there was confusion about what protesters wanted, amid calls for an end to devolution.
He said the British National Party and other far-right groups had been addressing rallies.
He said: "I think we need a bit of calm and reflection. We need to get wise heads together."
Michael Copeland, Ulster Unionist Party MLA, was pessimistic about a positive outcome to the talks, as there was a lack of leadership among protesters.
He said: "There doesn't seem to be any one person, or group of persons, that we can go to."