The anti-apartheid icon, held in the highest esteem across the world for leading his nation's transition from white-minority rule in the 1990s after 27 years in prison, had been receiving medical care for a lung infection at his home.
Mr Mandela spent his final months surrounded by family after being released from hospital following a three month stay and passed away peacefully, President Jacob Zuma announced in a televised address.
"Our nation has lost its greatest son," he said. "Our people have lost a father.
"What made Nelson Mandela great was precisely what made him human. We saw in him what we seek in ourselves."
A law student turned freedom fighter, his battle to end the system of apartheid that tore apart his nation led to his incarceration in a tiny cell in the infamous Robben Island jail.
Prime Minister David Cameron said "a great light has gone out in the world" while US president Barack Obama mourned the loss of "one of the most influential, courageous, and profoundly good human beings that any of us will share time with on this earth".
In 1993 Mr Mandela was awarded the Nobel Peace Prize in 1993 and was elected South Africa's first black president the following year.
American civil rights leader Jesse Jackson said Mr Mandela's "imprint" would be "everlasting".
"My heart weighs heavy," said Mr Jackson, who has been in the UK this week and spoke to students in Cambridge on Monday. "The imprint he left on our world is everlasting."
United Nations secretary general Ban Ki-moon said Mr Mandela was "a giant for justice" whose "selfless struggle for human dignity, equality and freedom" inspired many people around the world.
"No one did more in our time to advance the values and aspirations of the United Nations," he said.
President Zuma said the man known as Madiba would be afforded a state funeral and South African flags would be lowered to half-mast until the funeral.
"Our thoughts are with the South African people who today mourn the loss of the one person who more than any other came to embody their sense of a common nation," he said from Pretoria.
"Our thoughts are with the millions of people across the world who embraced Madiba as their own and who saw his cause as their cause."
The clearly emotional leader added: "As we gather to pay our last respects, let us conduct ourselves with the dignity and respect that Madiba personified.
"Let us be mindful of his wishes and the wishes of his family as we gather wherever we are in the country and wherever we are in the world.
"Let us recall the values for which Madiba fought.
"Let us reaffirm his vision of a society in which none is exploited, oppressed or dispossessed by another."
Flowers and a framed picture of Mr Mandela were laid at the base of his statue in Parliament Square in the early hours.
One tribute on a card read: "Thank you for the sacrifices you made for all of us." A second read: "May God shine light on your homecoming in heaven.
"Rest in Peace Mr Mandela."
Two heritage wardens stood near the statue as passers by stopped to pay their respects.
The flag at No 10 will be flown at half-mast in honour of the former leader, who was a "hero of our time", Mr Cameron announced.
In London Lungi Morrison, the granddaughter of fellow apartheid opponent Archbishop Desmond Tutu, sang after lighting a candle for Mr Mandela outside the South African High Commission, the BBC said.
And Ela Gandhi, granddaughter of former Indian leader Mahatma Gandhi, told the BBC: "It is difficult for me to speak right now because I am so overcome with grief and emptiness... For me (Mandela) was really a wonderful hero and idol that I really felt that humanity should emulate, should follow."
Mr Mandela's death has provoked tributes from around the world.
Former Prime Minister Sir John Major said: "Nelson Mandela left an indelible mark on his time that few have ever equalled.
"He showed the world that reconciliation is better than retaliation and - for that lesson and much more - he will be remembered for as long as history is written.
"As the world mourns an extraordinary man, who Norma and I had the privilege to know, our thoughts and condolences are with his family and his countrymen."
Hillary Clinton, the former US Secretary of State, said: "Nelson Mandela was a champion for justice and human dignity, with unmatched grace. I'll remember him as Madiba, truly an unconquerable soul."
Indian prime minister Manmohan Singh likened him to his own country's great freedom fighter, Mahatma Gandhi, saying: " A giant among men has passed away. This is as much India's loss as South Africa's. He was a true Gandhian."
Mr Mandela's body will lie in state for three days in Pretoria, the BBC said, before a funeral is held on Saturday in Qunu, the village in Eastern Cape where he was born.
Britain's first black Cabinet minister Lord Boateng, who served as high commissioner in South Africa, told BBC Radio 4's Today programme: "There was a focus and a discipline to his activism, always.
"But imbuing it all was this overpowering sense, one had when one met him and worked with him, of the power of love. That's not a word that politicians use much but he was a consummate politician.
"This was someone who knew how to win in a negotiation, this was someone who knew how to lead a political movement, an intensely political movement, to imbue his own comrades in prison with a very clear political agenda which he then spread beyond Robben Island.
"But always there was this sense of overwhelming love of his fellow human beings that came through in his smile, in his words, in his physical presence."
Former ANC MP Andrew Feinstein told Today: "I found myself as a white MP in the ANC looking across at the former government benches and feeling a constant sense of anger and outrage. And from Mandela, and those around him, only a feeling of reconciliation and of working together to build a non-racial nation.
"It's something extremely difficult to fathom - how a man goes into prison, ostensibly for life, and learns the language of his jailers so that he can communicate with them."
He added: "I think we can count on the fingers of one hand people who have had that ability throughout the centuries and he's certainly one of them. It manifested in so many ways after he came to power."