Mr Obama last week banned eavesdropping on the leaders of allies and began reining in the vast collection of US citizens' phone data, seeking to to reassure Americans and foreigners the US would take into account privacy concerns highlighted by former NSA contractor Edward Snowden's revelations.
But Kenneth Roth, executive director of the New York-based group, said in Berlin Mr Obama had provided little more than "vague assurance" on the monitoring of communications.
He added: "In none of this has there been a recognition non-Americans outside the US have a right to the privacy of their communications, that everybody has a right to the privacy of their metadata and that everybody has a right not to have their electronic communications scooped up into a government computer."
Mr Roth said the US needed to stop gathering communications en masse, saying there was no proof such vast surveillance had made a difference to security.
HRW General Counsel Dinah PoKempner said: "In the end, there will be no safe haven if privacy is seen as a strictly domestic issue."