Around 80% of nitrogen emissions in the European Union come from agriculture, for example from fertiliser and manure, and up to 88% of the sector's nitrogen losses into the environment are linked to livestock production.
Nitrogen emissions in the form of ammonia, nitrates and nitrous oxide can cause air, water and soil pollution and account for around 10% of the warming caused by greenhouse gases, experts said.
If everyone in the EU became "demitarian" - halving the amount of meat and other animal products they consume - it could reduce greenhouse gases from agriculture by 25% to 40% and nitrogen emissions by 40%, the European Nitrogen Assessment's "nitrogen on the table" report said.
It would also bring European consumption of saturated fats down to within levels recommended by the World Health Organisation and reduce red meat and protein consumption, cutting the risk of heart disease and cancer.
Because around 80% of crops are fed to livestock, reducing meat and dairy consumption would free up land for other crops or for plants grown to produce bio-energy.
It would also slash soya bean imports to the EU by around three-quarters.
The report's lead author Henk Westhoek, from the Netherlands Environmental Assessment Agency, said such a change could see the EU become a major exported of food products such as soya beans.