The names of more than 19,000 landowners and farmers who annually receive public subsidies totalling more than £500 million are to be kept secret as a result of a new ruling by the Scottish Information Commissioner, Rosemary Agnew.

Despite the fact that they have been named in previous years and are likely to be named again, the vast majority of those currently profiting from taxpayers' money are to be kept under wraps.

Agnew is this week publishing a decision that backs the Scottish Government's refusal to release the names of individuals in receipt of farming subsidies. It follows a ruling in November 2010 by the European Court of Justice in Luxembourg that publishing the information breached people's rights to privacy.

Agnew made clear that she thought that the farmers and landowners should be identified, but argued that her hands were tied by the court judgment. "My own view is that information of this nature concerning agricultural subsidies should generally be disclosed into the public domain," she said.

She said that the previous commissioner, Kevin Dunion, ordered similar information to be released in 2007. "However, in the circumstances of this particular request, I am required to take into account the decision of the Court of Justice of the European Union," she added.

Agnew concluded that Scottish ministers were entitled to keep the names secret. The identities of all those in receipt of farming subsidies in 2011 were originally requested by the Sunday Herald in July 2012.

The secrecy has been condemned by campaigners. They pointed out that a change in the EU rules should ensure that most subsidy recipients would be named from 2014 onwards.

Land rights campaigner, Andy Wightman, thought it was "understandable" that Agnew had felt unable to order the release of the information. "However, it is plainly wrong that the public is not allowed to know who receives over 80% of the £636m in farm subsidies simply because it is paid to individuals rather than to companies," he said.

The European court judgment prevented governments from identifying farmers who traded as individuals, rather than companies. According to the Scottish government, this meant that in 2012 only about 2700 out of 21,800 subsidy recipients were named.

Wightman said: "There is no fundamental reason why such information should be secret."

Jack Thurston from farmsubsidy.org, a group that campaigns for greater transparency on agricultural payments, described the European court ruling as "frankly bizarre".

The last time the subsidy information was released in 2010, the Sunday Herald revealed that four farmers in Scotland were given record handouts of more than £1m each. Other big landowners given hundreds of thousands of pounds included the Duke of Buccleuch, Lord Morton, the Earl of Moray, the Earl of Rosebery and the Earl of Seafield.

Landowner and farmer groups said that they had not sought the ban on naming individuals but highlighted the problems that full disclosure would bring. Many people would feel "uncomfortable" about revealing their total income, argued Douglas McAdam, chief executive of Scottish Land & Estates. "A requirement to disclose support payments could have a disproportionate impact on the privacy of those farmers who depend on agricultural support for business survival," he said.

According to Bob Carruth, spokesman for the National Farmers' Union in Scotland, it was "inappropriate" to publish the personal details. "The public can be reassured that every business that receives support is held to account by the legislators," he said.

The Scottish Government said it had a record of openness on farm subsidies. It welcomed EU moves to correct the "legal technicality" at the heart of the court ruling.

"This will allow us to return to a position where we proactively publish details of all farm subsidy beneficiaries," said a Government spokeswoman.