Abbott, a former boxer, Rhodes scholar and trainee priest, promised to restore political stability, cut taxes and crack down on asylum seekers arriving by boat.
"From today I declare that Australia is under new management and Australia is once more open for business," he told jubilant supporters in Sydney.
It was frustration with Labor's leadership turmoil that cost the government dearly at the polls.
Labor dumped Prime Minister Kevin Rudd in 2010 for Australia's first female prime minister Julia Gillard, only to reinstate Rudd as leader in June 2013 in a desperate bid to stay in power.
"It is for the people of Australia to determine the government and the prime minister of this country and you will punish anyone who takes you for granted," said Abbott.
Rudd was given a rousing welcome from dejected Labor party supporters in his hometown of Brisbane, conceding defeat and announcing he would step down as party leader.
"I know that Labor hearts are heavy across the nation tonight. I gave it my all. But it was not enough to win," said Rudd, supported by his wife and family.
Labor's overall vote was its worst since 2004, when then conservative prime minister John Howard won his fourth and final term, but was not as bad as the party had feared. Labor held on to all of its close seats in Rudd's home state of Queensland, as well as several marginal seats in western Sydney.
Election officials said with about 80% of the vote counted, Abbott's Liberal-National Party coalition had won about 52.6% of the national vote, and projected it would win at least 88 seats in the 150-seat parliament. Abbott could end up with a majority of around 30 seats, ending the country's first minority government since the Second World War. Labor had relied upon independent and Greens support for the past three years.
"This was an election that was lost by the government more than one that was won by the opposition," said former Labor prime minister Bob Hawke.
The election has been pitched as a choice on who is best to lead the country's economy as it adjusts to an end of a prolonged mining investment boom.
Abbott, 55, built up a strong opinion poll lead on the back of promises to rein in government spending, scrap an unpopular tax on carbon emissions and stop the flow of refugee boats arriving in Australia's northwest.
His campaign had support from media magnate Rupert Murdoch and his Australian newspapers, which have urged voters to reject Rudd's Labor government. Australia's other major newspaper group, Fairfax, also called for a change of government, saying Rudd had painted Abbott's planned spending cuts as dangerous European-style austerity and that the latter's government was best placed to manage an economy that is slowing but remains the envy of much of the developed world.
A record 1717 candidates contested the election, including colourful mining entrepreneur Clive Palmer and WikiLeaks founder Julian Assange, who is holed up in the Ecuadorian embassy in London.
With Abbott's convincing victory, much of the interest is focused on the Senate, where the Greens, independents and fringe parties might still hold the balance of power and frustrate Abbott's legislative agenda.
Final results in the Senate could take more than a week to determine, due to the complicated system of preferential voting and proportional representation.