The US president spoke of the "heroic" life of the South African anti-apartheid leader, describing him as the "last great liberator of the 20th century" as he addressed tens of thousands of people at a memorial service to Mr Mandela in the FNB Stadium in Soweto.
Mr Obama compared Mandela's actions to those of Gandhi, Abraham Lincoln and Martin Luther King, but also warned against viewing Mr Mandela as detached from normal life.
"He was not a bust made of marble, he was a man made of flesh and blood," Mr Obama told the crowd, which included leaders from more than 90 countries.
Mr Obama singled out world leaders who have welcomed gains made by Mr Mandela but resist reforms to tackle inequality and injustice. He said: "Around the world today we still see children suffering from hunger and disease and we still see run-down schools and we see young people without prospects for the future.
"Around the world today men and women are still imprisoned for their political beliefs and are still persecuted for what they look like and how they worship and who they are, that is happening today.
"There are too many people who happily embrace Madiba's [Mandela's] legacy of racial reconciliation, but passionately resist even modest reforms that would challenge chronic poverty and growing inequality."
Mr Obama's speech came after he shook hands in the rain-soaked stadium with Cuban president Raul Castro, the first such greeting in public involving a president of the US since the Cuban revolution. Former US president Bill Clinton reportedly shook hands with Raul's brother Fidel behind closed doors at a UN lunch in 2000.
A White House aide said later the handshake was a spontaneous gesture of greeting and went no further than pleasantries.
During the service in the two-thirds filled stadium, boos were heard for current South African president Jacob Zuma and the former US president George W Bush. But there were cheers for Mr Obama and for Zimbabwean president Robert Mugabe.
Those attending included Prime Minister David Cameron and his three predecessors, Sir John Major, Tony Blair and Gordon Brown. Also seen at the service were supermodel Naomi Campbell, rock star Bono, former South African president FW De Klerk, Mr Clinton and his wife ex-US secretary of state Hillary Clinton, former French president Nicolas Sarkozy and his successor Francois Hollande.
Mr Mandela, once described as the world's most famous political prisoner, was released after 27 years in jail in 1990 and went on to become South Africa's first black president. He died on Thursday aged 95.
The service heard from Andrew Mlangeni, a former prisoner on Robben Island with Mr Mandela, who said: "Madiba is looking down on us. There is no doubt he is smiling and he watches his beloved country, men and women, unite to celebrate his life and legacy."
UN secretary-general Ban Ki-moon told the service: "Nelson Mandela was more than one of the greatest leaders of our time, he was one of our greatest teachers. He taught by example, he sacrificed so much and was willing to give up everything for freedom, equality and justice. His compassion stands out most."
Mr Zuma added: "Today Madiba is no more. He leaves behind a nation that loves him dearly. He leaves a continent that is truly proud to call him an African.
"He leaves the people of the world who embraced him as their own … icon. Most importantly, he leaves behind a deeply entrenched legacy of freedom, human rights and democracy in our country.
"In his honour we commit ourselves to continue building a nation based on the democratic values of human dignity, equality and freedom. United in our diversity we will continue to build a nation free of poverty, hunger, homelessness and inequality."
Mr Mandela's body will lie in state at South Africa's seat of government, the Union Buildings in Pretoria, until he is laid to rest in a state funeral at his home town of Qunu in the Eastern Cape on Sunday.
The Prince of Wales will be among a smaller number of dignitaries travelling to the remote rural location for the service.