Last week, campaign group Migration Watch UK warned that by the next general election in 2015 there could be more than one million such voters.
The organisation has now revealed the results of a YouGov poll that suggested 60% of adults think that nationals from the 54 Commonwealth countries who do not have UK citizenship should be banned from voting here.
Currently, citizens from those states, which include Australia, Canada and India, can register on the electoral roll upon arrival in the UK as long as they have an address.
Sir Andrew Green, chairman of Migration Watch UK, said "Public opinion is rightly very strong on this. When Parliament has considered it in the past MPs had no idea of the scale that this issue had reached. This hang-over from the Empire must be tackled and soon."