Mr Mursi said the decree halting court challenges to his decisions, which sparked eight days of protests and violence by Egyptians calling him a new dictator, was "for an exceptional stage" and he aimed to speed up the democratic transition.
Thousands took to the streets in Cairo and other places, such as Alexandria and cities on the Suez Canal and in the Nile Delta, responding to opposition calls for a big turnout.
It did not match the tens of thousands who demonstrated on Tuesday but the rallies were expected to gather pace later in the day.
Protesters said they would push for a 'no' vote in a referendum, which could happen as early as mid-December.
If approved, it would immediately cancel the president's decree.
"We fundamentally reject the referendum and constituent assembly because the assembly does not represent all sections of society," said Sayed el Erian, 43, a protester in Tahrir and member of a party set up by opposition figure Mohamed El Baradei.
In the Cairo mosque where Mursi said Friday prayers, some opponents chanted against him but backers quickly surrounded him shouting in support, journalists and a security source said.
Two people have been killed and hundreds injured in protests since the decree was announced on November 22, deepening the divide between the newly empowered Islamists and their critics.