The new research found that men with unhealthy lifestyles produced as much swimming sperm as those living more sensibly.
Under current guidelines, doctors are supposed to warn men diagnosed with infertility of the dangers of alcohol, tobacco and recreational drugs. Infertile men are also urged to avoid being overweight and not to wear tight underwear. In some cases, fertility treatment is delayed to allow couples time to improve their lifestyles.
However, the new research suggests none of these factors has much impact on the number of swimming sperm a man produces.
Results from studies around the UK, reported in the journal Human Reproduction, showed that men with poor quality sperm were 2.5 times more likely to have had testicular surgery, and twice as likely to be of black ethnicity.
They were also 1.3 times more likely to do manual work, not wear loose boxer shorts, or not to have had a previous conception.
But men's use of tobacco, alcohol and recreational drugs made little difference, as did their weight as measured by body mass index.
Study leader Dr Andrew Povey, from the University of Manchester, said: "Delaying fertility treatment then for these couples so that they can make changes to their lifestyles, for which there is little evidence of effectiveness, is unlikely to improve their chances of a conception and, indeed, might be prejudicial for couples with little time left to lose."
The number of swimming sperm broadly correlates with how fertile a man is likely to be. It also often determines the type of fertility treatment that is offered.