The claim by scientists comes after 95% of eggs laid by the new strain of Anopheles gambiae mosquito in laboratory tests hatched into males.
Introducing the GM insects to normal mosquitoes led to the complete eradication of populations denied the chance to breed because of the lack of females.
Replicating the research in the field could wipe out large numbers of malaria-carrying mosquitoes, or even lead to their extinction.
Lead researcher Professor Andrea Crisanti, from the Department of Life Sciences at Imperial College London, said: "Malaria is debilitating and often fatal and we need to find new ways of tackling it. We think our innovative approach is a huge step forward. For the first time, we have been able to inhibit the production of female offspring in the laboratory, This provides a new means to eliminate the disease."
More than 3.4 billion people are at risk from malaria infection around the world and 627,000 die from the disease each year, according to the World Health Organisation. In Africa, the disease claims one young life a minute.