Monitoring by the government watchdog has found traces of nonylphenols - chemicals known to disrupt hormones and cause sex change - in the Clyde, the Kelvin, the Water of Leith, the Tay, the Ness, the Don and 14 other rivers.
Although the production and use of the chemical compounds is now banned in Europe, they have recently been detected in clothes imported from China and other Asian countries, where there are no restrictions. When the clothes are washed, the chemicals escape and end up in rivers.
Nonylphenols are endocrine disruptors, which means they can affect hormones such as oestrogen in the body. Studies have found that they can cause the "feminisation" of fish, decreasing male fertility and shrinking testicles.
The chemicals are very toxic to wildlife, very persistent in the environment and build up in fish, birds and other animals, says Sepa. "It is therefore possible that the presence of nonylphenols in the environment poses a long-term threat to wildlife on both a local and global scale," the agency warns.
Some experts say that endocrine disrupters could affect humans, even in tiny doses equivalent to a teaspoonful in an Olympic-sized swimming pool. "Recent research has found associations between environmental exposures and adverse reproductive and neurological effects in humans," said Professor Andrew Watterson, head of the Occupational and Environmental Health Research Group at the University of Stirling.
He urged Scotland to follow Sweden and assume that even the tiniest concentrations of nonylphenols were unsafe. "It is worrying that elevated levels are still being recorded across the country under the current controls when less hazardous substitutes and toxic-use reduction strategies may be available," he said.
Friends of the Earth Scotland said that it had first raised concerns about gender-bending chemicals 20 years ago.
"It took many years for them to be banned but here they are turning up again in Scottish rivers," said the environmental group's director, Dr Richard Dixon.
"Finding these chemicals at all is bad news," he added. "We need to get a grip on their sources and make sure the levels are reduced as quickly as possible," he continued.
Nonylphenols are regarded as a "priority hazardous substance" in water under European law. Sepa is required to monitor rivers for the chemicals, with the aim of eliminating them.
Sepa's environmental chemistry manager, Ashley Roberts, pointed out that of the 20 rivers where elevated levels of nonylphenols were detected in 2011-2013, only the River Irvine breached Sepa's environmental quality standard.
"Sepa's environmental monitoring will continue to investigate the possible sources and potential mitigation measures needed to support the long-term goal of removing this chemical from the natural environment," she said.
"Nonylphenols are already heavily restricted under European legislation and Sepa, alongside other UK agencies, has recently provided monitoring data to inform the European Commission's considerations of whether further restrictions - specifically in relation to textiles - are required."
A study last year by Sepa's sister agency in England, the Environment Agency, found nonylphenols in 28 out of 96 samples of cotton underwear imported from Asia, half of which came from China. Tests showed that the chemicals came out in the wash.
The Chemical Industries Association (CIA), which represents chemical companies, stressed that the use of nonylphenols had been greatly reduced in Europe. Member companies had withdrawn them from UK supply chains, and switched to alternatives.
The chemicals were, however, still used in textile manufacturing outside Europe, and regulators were working with the UK retail industry to try to address the issue.
"Many clothing brands have committed to work with their suppliers around the world to ensure that they are no longer using nonylphenols in their manufacturing process by 2020," said the CIA's chemicals policy director, Dr Jo Lloyd. "This is an international initiative that the global chemical manufacturing sector is actively engaged with."
However, two US companies that are members of the CIA, Dow and Huntsman, still market nonylphenols on their websites.
Rivers contaminated by gender-bender chemical, and the River Nonylphenols detected (average
nanogams per litre 2011-13)
1 River Clyde, Glasgow 193
2 River Kelvin, Glasgow 36
3 Black Cart Water, Paisley 129
4 White Cart Water, Paisley 152
5 River Irvine, Irvine 341
6 Annick Water, Irvine 98
7 River Annan, Annan 237
8 River Esk, Canonbie 156
9 River Carron, Falkirk 132
10 River Esk, Musselburgh 80
11 River Almond, Edinburgh 104
12 Water of Leith, Edinburgh 100
13 River Leven, Methil 61
14 River Tay, Perth 115
15 Dighty Water,Broughty Ferry 112
16 River Don, Aberdeen 88
17 River Nairn, Nairn 186
18 River Lossie, Lossiemouth 70
19 River Ness, Inverness 84
20 River Brora, Sutherland 159