Diabaly, 220 miles north of Mali's capital, Bamako, had harboured the main cluster of insurgents south of the frontline towns of Mopti and Sevare.
Residents said some of the rebels had abandoned their flowing robes to blend in with the local population.
The charred and twisted wreckage of their pick-up trucks littered the sandy streets between mud-brick buildings.
"French and Malian forces have advanced to Diabaly and they will continue their mission of securing the town," said the deputy commander of Malian forces in nearby Niono, who gave his name only as Captain Samasa.
The French commander in the region warned of the risk of mines and booby traps in the insurgents' wake. The region around Diabaly has long been a hub for al Qaeda-linked cells believed to have camps in the Ouagadou forest near Mauritania's border.
France has deployed 2000 ground troops and its war planes have pounded rebel columns and bases in Mali. Its intervention turned back a column of Islamist rebels heading towards Bamako that threatened to topple Mali's government.
France now aims, with international support, to dislodge the Islamists from Mali's vast desert area before they use it to launch attacks on the West.
The Islamist alliance, grouping al Qaeda's North Africa wing AQIM and the home-grown Malian militant groups Ansar Dine and Mujwa, has imposed harsh sharia law in northern Mali, including amputations and the destruction of ancient shrines sacred to moderate Sufi Muslims.
A resident of Timbuktu said scores of pick-up trucks carrying Islamist fighters had arrived there since Saturday, as the rebels pull their forces towards their desert strongholds.