By Angela Waters, Special for USA TODAY

Austria announced Tuesday it would take over and possibly destroy the house where Adolf Hitler was born to prevent right-wing extremists from using it as a pilgrimage site.

"The decision is necessary because Austria would like to prevent this house from becoming a 'cult site' for neo-Nazis in any way. It has been used repeatedly for this in the past, when people gathered there to shout (Nazi) slogans," Interior Minister Wolfgang Sobotka said.

READ MORE: No applause from SNP as David Cameron steps down as Prime Minister

"It is my vision to tear down the house," he added.

For years, the Austrian government rented the building in the western town of Braunau am Inn from a local retired woman for $5,332 a month, using it as a workshop for disabled people. The relationship between the landlord and the government deteriorated after she refused offers to sell the property and rejected ideas for future use, the government said.

On Tuesday, Austria's parliament authorized the seizure of the property from its current owner under the country's eminent domain laws. The owner now has no right to appeal the decision.

Although the German dictator’s family lived in the pale yellow building for only three years, the site has attracted right-wing extremists for decades paying homage to Hitler. Hitler's Nazi Germany initiated World War II and killed millions of people, mostly Jews, during the Holocaust. Hitler killed himself in 1945.

Some disagree with Austria's new plan.

READ MORE: In Pictures: Glasgow's George Square from 1919 to present

Vice Chancellor Reinhold Mitterlehner said the house had “educational value” and should converted into a museum to teach future generations about the dangers of extremism.

Although it has been almost 80 years since Germany annexed Austria in 1938, it is still debated whether Austria welcomed Hitler’s leadership, or whether Austria was truly occupied by the Nazis.

Meanwhile, the Austrian government appointed a 12-member committee to determine the future — or demise — of the building, which has been empty since 2011.

In Germany, many former landmarks relating to Hitler, his officers or the Nazi era have either been destroyed or converted to memorials. After a fight that lasted more than a decade, Germany in 2011 destroyed the Bavarian gravesite of Rudolf Hess, Hitler's No. 2 official who died in 1987. That followed an unsuccessful attempt in 2005 to ban annual pilgrimages by neo-Nazis on the anniversary of Hess' death accompanied by Nazi songs and salutes.

Many Germans believe other European governments have a responsibility to curb Hitler memorials, even as the German government has yet to comment on the fate of Hitler’s birth home. Others say such sites should be preserved as a reminder and as a lesson.

READ MORE: No applause from SNP as David Cameron steps down as Prime Minister

Josie Winter, 27, a yoga teacher in Berlin, sides with the Austrian vice chancellor and thinks the building should be put to a good use.

“If it's still in good condition, then there is no need to tear it down,” she said. “Just renovate and change its function, transfer the energy it holds into something useful and progressive, a place that helps people."