Some 53% of women drank more than the upper limit of two units a week during the first trimester, figures show.
Middle class women were more likely than women from other classes to drink more than the recommended limits during pregnancy, researchers said.
They called for a revision of guidance on drinking in pregnancy after their study found even those adhering to the suggested limits were more likely to have problems with their babies than those who did not drink at all.
The study examined 1200 women in Leeds who filled out food frequency questionnaires, including information on alcohol consumption in the month before conception and the three trimesters of pregnancy.
On average, women who participated in the study drank 11.2 units a week before conception; four units a week during the first three months; and less than two a week during the last six months of pregnancy.
If a woman drank more than two units a week in the first three months of pregnancy they were twice as likely to have a baby who was born "small for gestational age" when compared to non-drinkers, the researchers found.
NHS guidance says women should avoid drinking alcohol if they are pregnant or trying to conceive. If they do drink they should have no more than "one to two units of alcohol once or twice a week".
The study's authors said that the results highlight the need for endorsing the "abstinence-only" message.
Professor Andrew Whitelaw, professor of neonatal medicine at the University of Bristol, added: "This is further evidence that even moderate amounts of alcohol are toxic to the growing foetus and direct toxicity is further worsened by the increased complications of premature birth."