The Office of Fair Trading (OFT) said the £9.4 billion-a-year market was not working well for consumers amid concerns that insurers are focusing too much on pushing up rivals' costs.
It wants to see firms concentrate on the quality and value of the service they provide to customers.
The industry has come under fire for soaring motor insurance premiums, which rose by 12% between 2009 and 2010 and a further 9% in the first three quarters of 2011, at a time when consumers have been squeezed by high living costs, including petrol prices.
The OFT estimates the problems it has uncovered cost private motor insurers £225 million last year and mean consumers could be paying an extra £10 per motor insurance policy.
The commission, which has intervened before on issues such as the sale of payment protection insurance, has wide-ranging powers to restructure the market and make firms sign up to binding agreements.
The probe comes at a time when MPs have been looking at how to tackle spurious whiplash claims, which have been ramping up premiums.
Prime Minister David Cameron has vowed to take on the "compensation culture", including the £2bn-a-year costs associated with whiplash that are forcing up insurance costs.
Proposals include consulting on the feasibility of introducing independent medical panels.
The independent medical experts, who would have no direct links to either claimants or defendants, would replace the current assessment of whiplash injuries by either GPs or doctors employed by medical reporting organisations.
The private motor insurance market in the UK was estimated to be worth £9.4bn in 2010, with the cost of third-party property damage, including repairs and replacement vehicles to not-at-fault drivers, standing at around £1.4bn.
Under the law, drivers who are not at fault are entitled to be put back in the position they were before the accident happened, which could include receiving a replacement vehicle.
The OFT found insurers of the driver at fault have little control over how repairs and courtesy car services are carried out.
Insurers of the drivers not at fault take advantage of this to make cash through referral fees and rebates. These expenses ramp up the costs for other insurers, which end up being passed on to drivers as their premiums go up.
From the late 1980s, credit hire companies supplied replacement vehicles on credit terms, recovering the costs from the at-fault drivers' insurer.
While this has improved the service the not-at-fault driver receives, some insurers believe the cost of this is too high.
The OFT pinpointed practices which could be inflating the cost of replacement vehicles, including insurers of not-at-fault drivers referring them to credit hire organisations which tend to charge higher daily rates, in exchange for referral fees of up to £400.
It was also concerned that courtesy cars were being provided for longer than necessary.
The watchdog highlighted practices which may be increasing repair costs, including agreements between insurers and repairers to charge higher labour rates.
Motorists in Scotland typically pay smaller premiums than those south of the border, given the comparative lower population density.
Contextual targeting label: