Over 600,000 current account holders have swapped providers in the six months since a new guarantee to make it easier to move was launched, according to the Payments Council.
But calculations by Which? found that even if the number of people switching continues to grow, only one in every 24 (4.2%) people will ditch their old provider and switch to a better deal this year.
It said that the Competition and Markets Authority (CMA) should refer the current account market for a full competition inquiry to take a "hard look" at how it operates.
Which? said there is a "worrying lack of innovation in products and service" and consumers often find it hard to calculate charges and compare accounts.
Which? executive director, Richard Lloyd, said: "The Competition and Markets Authority should refer the current account market for a full competition inquiry.
"Low switching levels and a high level of complaints show the unhealthy dominance of the biggest banks is set to continue and this market is not working in the best interests of consumers.
"Even if the number of people switching continues to grow at the same rate as the Payments Council suggests, we estimate only 4.2% of people will switch accounts this year.
"Unless we see an injection of much needed competition into the market, the major high street banks will continue to have a stranglehold."
Cathy Jamieson, shadow financial secretary to the Treasury, said: "Which? is right to highlight the lack of competition in our banking system.
"A more competitive banking sector would be good for both consumers and businesses."
Complaints about current accounts jumped to 303,110 in the second half of 2013, an 8% increase on the previous six months. Which? said tests of a competitive banking market that works for consumers would include looking at whether firms are competing genuinely with better-quality, innovative products, whether they are offering a choice of products and whether there is enough clear information to make easy comparisons.
It said the tests should also look at whether customers are able to get quick redress if something goes wrong and whether consumers have everything they need to drive competitive pressure.
The switching guarantee started last September and aims to speed up and simplify the process of switching banks.
Which? has been campaigning for banks to release better information about current account running costs, which could then be used to develop comparison tools so that people can find the most suitable deal for their own needs more easily, as charges can vary hugely depending on how you use your current account.
It is hoped that eventually people should be able to simply upload information about how they use their account for analysis and get an immediate answer to whether or not they would be able to save money by moving to a different account.