The US president spoke of the "heroic" life of the South African anti-apartheid hero describing him as the "last great liberator of the 20th century" as he addressed thousands gathered for the memorial service to Mr Mandela in the FNB Stadium in Soweto.
Mr Obama compared Mr Mandela's actions to those of Gandhi, Abraham Lincoln and US civil rights leader Martin Luther King but also warned against viewing Mr Mandela as detached from normal life.
"He was not a bust made of marble, he was a man made of flesh and blood," Mr Obama told the crowds in the stadium including leaders from more than 90 countries gathered at the stadium.
Mr Obama further singled out world leaders who have publicly welcomed gains made by Mr Mandela but resist reforms to tackle inequality and injustice.
"Around the world today we still see children suffering from hunger and disease and we still still see run down schools and we see young people without prospects for the future," he said.
"Around the world today men and women are still imprisoned for their political beliefs and are still persecuted for what they look like and how they worship and who they are, that is happening today.
"There are too many people who happily embrace Madiba's (Mandela's) legacy of racial reconciliation but passionately resist even modest reforms that would challenge chronic poverty and growing inequality.
"There are too many leaders who claim solidarity with Madiba's struggle for freedom but do not tolerate dissent from their own people.
"There are too many of us on the sidelines comfortable in complacency or cynicism."
Mr Obama warned against viewing Mr Mandela as "detached" from normal life.
"It was precisely because he could admit to imperfection, because he was so full of good humour, even mischief despite the heavy burdens that he carried, that we loved him so," he said.
"He was not a bust made of marble, he was a man of flesh and blood, a son and a husband, a father and a friend, that is why we have learned so much from him and that is why we can learn from him still.
"Nothing he achieved was inevitable, in the arc of his life, we see a man who earned his place in history through struggle and shrewdness and persistence and faith."
Mr Obama's speech came after he shook hands with Cuban president Raul Castro among other world leaders gathered at the service. The US president was cheered by the assembled crowds. Boos were heard for the current South African president Jacob Zuma.
Those attending included Prime Minister David Cameron and his three surviving predecessors, Sir John Major, Tony Blair and Gordon Brown.
Also seen arriving at the service were supermodel Naomi Campbell, rock star Bono, former South African president FW De Klerk, former US president Bill Clinton and his wife ex-US secretary of state Hillary Clinton, former French president Nicolas Sarkozy and the current French president Francois Hollande.
Mr Mandela, who died last Thursday aged 95, made his last public appearance at the stadium at the closing ceremony of the 2010 football World Cup.
The service, marked by heavy rain, heard from Andrew Mlangeni, a former prisoner on Robben Island with Mr Mandela, who spoke of the "outpouring of love" following his death.
"Madiba is looking down on us. There is no doubt he is smiling and he watches his beloved country, men and women, unite to celebrate his life and legacy," he said.
UN Secretary-General Ban Ki-moon told the service: "South Africa has lost a hero, we have lost a father and the world has lost a beloved friend and mentor.
"Nelson Mandela was more than one of the greatest leaders of our time, he was one of our greatest teachers. He taught by example, he sacrificed so much and was willing to give up everything for freedom, equality and justice.
"His compassion stands out most."
Mr Cameron paid tribute to the spirit of "forgiveness" shown by Mr Mandela after his release from prison.
He told the BBC Radio 4 Today programme: "I think the classic lesson is that there is never a wrong time to do the right thing and that his generosity, his forgiveness, I think, was so important in bringing people together and there are so many parts of the world, in fact all parts of the world, where we need that sort of attitude.
"But there will be a lot of leaders upstairs who will be watching this and I hope that that's the lesson we take away."
After the memorial service, Mr Mandela's body will lie in state at South Africa's seat of government, the Union Buildings in Pretoria, until he is laid to rest in a state funeral at his home town of Qunu in the Eastern Cape on Sunday.
The Prince of Wales will be among a smaller number of dignitaries travelling to the remote rural location for that service.
Mr Zuma, addressing the service, said: "Today Madiba is no more. He leaves behind a nation that loves him dearly. He leaves a continent that is truly proud to call him an African.
"He leaves the people of the world who embraced him as their own... icon. Most importantly, he leaves behind a deeply entrenched legacy of freedom, human rights and democracy in our country.
"In his honour we commit ourselves to continue building a nation based on the democratic values of human dignity, equality and freedom. United in our diversity we will continue to build a nation free of poverty, hunger, homelessness and inequality."
Speaking to the BBC, Mr Blair said the service was "extraordinary".
Asked for his memories of Mr Mandela, Mr Blair said: "What I remember was how easy he was to get on with. I have still got the pictures of my little boy Leo just after he was born sitting on his knee when Mandela came to the Labour Party conference in the year 2000.
"He was so good with everyone, so easy with everyone, he was very determined, not to turn into, as Barack Obama said earlier, a sort of 'marble bust' , a sort of remote figure. He was very human and great fun.
"He was always reaching out to people, always trying to reconcile them and bringing people together and he did it in a way where there was something quite transcendent about it," he said.
"It was uncanny, his ability to reach people, not just on an emotional but almost on a spiritual level."
Labour leader Ed Miliband, who was at the service, tweeted afterwards: "Leaving the Mandela memorial service. A unique occasion, inspiring, moving and uplifting."
The Liberal Democrat leader and Deputy Prime Minister Nick Clegg was also present at the event.