The pilot project will mean drivers will have to slow down on trunk roads in Maybole, Largs, Biggar, Langholm and Oban.
A consultation is now getting under way to establish detailed proposals for each of the different locations, and the lower limits could be in place by as early as next spring.
Mr Brown said the new pilot would help "establish the benefits of lowering speeds in towns and villages where it is reasonable to do so".
The five locations where the lower speed limits will be trialled have been chosen after considering accident rates, the type of traffic in the areas and the vehicle speeds.
Maybole, Largs, Biggar and Langholm all have high accident rates, while a high proportion of the traffic in Maybole, Biggar and Langholm is made up of HGVs.
Oban has a high proportion of accidents involving vulnerable road users - such as cyclists and elderly people. Maybole, Largs and Biggar also have a large number of accidents of this type.
The transport agency Transport Scotland believes the lower limits will be "largely self-enforcing" and that traffic calming measures will not be needed to cut drivers' speeds.
Mr Brown said: "The safety of the trunk road network is a priority for Transport Scotland and managing speed is an important part of our strategy. It is essential that speeds are appropriate to conditions and these pilot zones will help us establish the benefits of lowering speeds in towns and villages where it is reasonable to do so.
"The five trials are being proposed that will seek to improve road safety generally, but we expect them to bring specific benefits for vulnerable road users, such as older people and cyclists."
The Transport Minister, who announced the pilot on a visit to Biggar, added: "There have been a number of calls for lower speed limits and specifically 20mph limits and we have had to whittle these down to a number that will give us a meaningful overview of how this will work in a variety of locations.
"In doing so, we considered the number of accidents and other factors such as traffic volumes and speeds as well as HGV numbers and the characteristics of the location. I know there will be some communities that are disappointed that they missed out but we will be looking at how their specific concerns can be addressed as part of our wider approach to speed management. The proposed pilot is an important step in our work to reduce accidents and casualties on the trunk network.
"The proposed pilot areas should not require significant engineering or police enforcement to support their operation and we hope to begin the wider consultation processes early next year. If these are completed successfully, the 20mph zones may be in place by the spring or early summer."