Only one thing is certain from last week's unexpected election result in Israel: the next Prime Minister will almost definitely be the previous incumbent, Binyamin Netanyahu, who now has a time limit of six weeks of political horse-trading to cut a deal with rivals and retake the premiership, albeit in a much weakened shape.

With only a tiny majority for the ruling Likud-Beiteinu party – 31 seats in place of 42 – he has been badly battered but due to the country's complicated electoral process he remains unbowed. Now the race is on to create a workable coalition and one that will take into account the shock success enjoyed by newcomer Yair Lapid, whose Yesh Atid party – which translates as There is a Future – is now the real power broker in the Israeli Knesset.

The outcome is one of the biggest surprises in recent Israeli politics. Not only is Lapid a new face but no-one really thought that he would emerge as the country's second most powerful politician. By winning 19 seats, Yesh Atid also rejigged the political scene in Jerusalem. Six months ago, Lapid and his party were written off as a fledgling and frivolous minority that would be crushed by the growing power of the conservative right, but now they can pretty well decide what happens next.

Yesh Atid wants to "normalise" Israeli politics and has a broad appeal among the young and the middle class. They put an emphasis on civic life – education, housing, health and transport and policing – rather than the Palestinian conflict. They aren't trying to placate the ultra-Orthodox religious lobby, and they say they are striving for peace according to an outline of "two states for two peoples", while still maintaining the large Israeli settlement blocs and ensuring the safety of Israel.

Faced with the rise of Yesh Atid, it is now clear that Netanyahu got this election badly wrong, by pandering to those on the right, by making bellicose statements about the nuclear threat in Iran, by ignoring the Palestinians and by refusing to listen to moderate opinion. Now he has to try to cobble together a centre-right coalition, which will mean entering into an accommodation with Lapid who might be offered the finance portfolio but could still hold out for foreign affairs.

As Israeli commentators such as Yael Paz-Melamed have noted, "King Bibi", as Netanyahu is known, has paid a hefty price for thinking that he was unique and irreplaceable. In the centrist Hebrew language newspaper Maariv, Paz-Melamed wrote: "The silent majority in Israel, the people who work, pay taxes, go to the army, serve in reserve duty, and especially those who chose to live here freely – they got off the couch, filled the ballot boxes and took back the power they deserve."

However, it is not a foregone conclusion that Lapid will throw in his lot with Netanyahu. He is also under pressure from his supporters to form a centre-left coalition with Labour Party leader, Shelly Yachimovich, who congratulated him on his success before offering her unconditional support. "I urge him not to join a Netanyahu-led government and not take part in the middle-class calamity which will happen the day after he is sworn in," she said. "Should he choose the other way – I'll stand by him and assist."

However, the most likely outcome for Yachimovich is that, with only 15 seats, she will head up the opposition, albeit a strong one. The new Knesset will be a more variegated place with less room for right-wing hawks such as Yisrael Beiteinu, the "Jewish Home" party led by Naftali Bennett who rejects the idea of a Palestinian state. As for Netanyahu he will have to trim his own philosophy, not just in foreign and defence matters but also on the domestic front to take into account the widespread support for Lapid's main manifesto arguments, such as provision of more housing, lower taxes and cheaper education.

Against that background voters rebelled against Netanyahu's election slogan, "a strong Prime Minister for a strong Israel" and turned instead to domestic issues that mean more to them, such as the economy, health and education. The shift in emphasis also means many ordinary Israelis are less concerned about the construction of settlements in the occupied territories, and more worried about their bank balances and the economy.

"If that is the case then it demonstrates a welcome end to a worrying drift towards the right in Israeli politics," says a British diplomatic source. "In the readjustment which is bound to follow it will also give the Europeans and the Americans a little more leverage to persuade the Israelis to make the discussions with the Palestinians work."

It is on that point that Netanyahu will have to make the greatest adjustment. Although he is unlikely to drop his hawkish stance he may no longer be able to count on support from the orthodox party Shas and Yisrael Beiteinu. None of this means Lapid was going to offer a soft line on the Palestinian question – he is committed to talks – but it does mean that any new coalition will have to think again about finding a workable solution.

It promises to be a bumpy ride. To get his majority Netanyahu will have to construct a stable administration but already fault lines are appearing. If he turns to the right and Yisrael Beiteinu he will almost certainly forfeit the support of Lapid, who has said he will not be used as a "fig leaf" for a continuation of Netanyahu's right-wing policies, while the right have let it be known that in turn they will not support Yesh Atid's proposals for insisting on the conscription of orthodox Israelis, who until now have controversially dodged the draft.