The upstart Aam Aadmi (common man) Party is led by mild-mannered former civil servant Arvind Kejriwal, who has vowed to end the stranglehold of India's two largest parties in the capital and beyond, and clean up politics in the process.
Hundreds of activists wearing "Gandhi caps" bearing the slogan, "I am a common man", gathered at the party's headquarters, cheering and waving brooms to symbolise a clean-out of rotten politicians.
Kejriwal defeated the three-time chief minister of Delhi in her own constituency and his party came close to winning control of the city.
The challenge for him now is to grow his movement in time for national elections due by May - a task that could pit him against leading opposition candidate Narendra Modi.
"I'm fully confident that finally the country will win, people will win, democracy will come, and India will win," Kejriwal said to supporters after bringing to an end Sheila Dikshit's 15-year run in Delhi.
Delhi is a city-state with a growing population of around 16 million.
The remarkable rise of the bespectacled Kejriwal and his party from the ashes of a street protest movement two years ago has shaken national parties which, only days before yesterday's result, had dismissed the buzz around the new party as hype.
It is uncertain how much the Aam Aadmi Party can grow in time for the general election, especially since one internal survey ahead of the Delhi election found that about one third of the party's supporters wanted to see Modi as prime minister.