It is the latest attempt to crush the movement which supports elected president Mohamed Mursi who was ousted by the army.
While short of a formal ban, the panel's advice to a court to remove the Brotherhood's NGO status threatens the million-member movement's future in politics.
An attack on a police station in central Cairo and plans for new mass protests by the Brotherhood today showed the stability - which the interim government says it took over to impose after two-and-a-half years of turmoil - is still elusive.
At least 900 people, most of them Islamist supporters of deposed President Mohamed Mursi, have been killed since the army takeover on July 3. The government has accused the Brotherhood of inciting violence and terrorism, and arrested its leaders.
Egypt's oldest political organisation, the Brotherhood won a series of elections after protesters forced out long-time ruler Hosni Mubarak in 2011, culminating in last year's presidential vote. It formally registered itself in March as a non-governmental organisation (NGO) to secure its legal status.
During Mr Mubarak's military-backed reign, the Brotherhood had for decades operated underground.
The judicial panel backed Brotherhood opponents who argued that the NGO registration was illegal because the Brotherhood-led government had effectively issued a licence to itself.
The panel's recommendation to the court due to rule on the case is not binding, judicial sources said. The court's next session, they said, would be on November 12.
It adds to a whole array of steps taken against the Brotherhood since the army stepped in after mass protests against economic mismanagement and attempts to entrench the movement's power during Mr Mursi's rule.
The Brotherhood formally operates in the political arena as the Freedom and Justice Party, which there has so far been no attempt to outlaw, but its NGO status was seen as a bulwark against legal attack.
Most of the group's leadership have been arrested on charges of inciting violence or murder. Mr Mursi was himself referred to trial on Sunday on those charges.
In addition, the government is shaping a new constitution to remove the Islamist additions that the Brotherhood introduced. It has tasked a 50-member constituent assembly, including only two Islamists, with reviewing a draft constitution that may allow members of Mubarak's government, banned from office after the 2011 revolution, to return.
The Brotherhood accuses the "putschist regime" of staging a coup against democracy and fabricating allegations of violence and terrorism to justify a drive to erase it from public life.
The National Coalition for Legitimacy, which includes the Brotherhood, called for a "million-person march" in all Egypt's squares today under the slogan The Coup is Terrorism. At least six people died during similar protests last week.
Although the Brotherhood says it is committed to peaceful resistance, fears have grown that attacks by Islamic radicals, such as those which have already hit lawless Northern Sinai, could develop into a wider insurgency.
Memories are still vivid of an Islamist insurgency in the 1990s, when bombs and shootings destabilised Egypt and ravaged tourism.
Three people on a motorcycle hurled a homemade hand grenade at a police station in a working class area of central Cairo yesterday.
Earlier, the army said three people had been arrested for firing machine guns at a ship passing through the Suez Canal on Saturday. A rocket-propelled grenade was also used in the attack.