The controversial policy has attracted criticism after the emergence of pictures of armed officers on routine patrols.
Mr Rennie accused Police Scotland and the Scottish Police Authority (SPA) of "stonewalling" calls by Highland Council for the policy to be scrapped in the area.
He also claimed Justice Secretary Kenny MacAskill had "abdicated any and all responsibility" for the policies of Police Scotland
He said: "Highland Council's united calls for the policy to be scrapped have been stonewalled until the next routine SPA risk meeting in September.
"That isn't a good enough response to the pressing concerns of these communities. Scottish Liberal Democrats want the SPA to undertake an immediate and full formal review of Police Scotland's policy on armed police. A separate review would allow swifter action on an increasingly controversial policy.
"Pending the outcome of that review I know many wish to see an end to the deployment of armed officers on routine duties. Questions also remain about the need for those armed officers to carry sidearms if they are to continue attending routine duties."
Police Scotland's refusal to act on the issue put them at odds with the Scottish Human Rights Commission, Justice Scotland and Highland Council, among others, he said.
Mr Rennie added: "It is time the growing chorus of concern was met with a more thoughtful response."
A Scottish Government spokeswoman said: "Armed police officers have been a long-standing feature of policing in Scotland and it is for the Chief Constable to make operational decisions about where and when to deploy resources.
"Police Scotland has emphasised that it regularly reviews the use of standing firearms authority.
"In the first year of Police Scotland, specialist firearms units attended more than 1,300 incidents across the whole of Scotland - including more than 100 in the Highlands.
"The approach taken by Police Scotland is an operational decision which allows officers to be deployed quickly in the event of any emergencies."
More than 98% of police are unarmed and the remaining 275 officers, from a 17,244-strong workforce, are spread across the whole of Scotland, according to the Scottish Government.
The spokeswoman added: "Armed officers are deployed on shift-pattern basis and, consequently, only a small number will be deployed across our communities at any one time."
In a statement the SPA said: "As the national oversight body for policing in Scotland, the SPA has made clear that there is a need for ongoing information, transparency and reassurance on this issue.
"In our consideration of the issue, SPA members stressed that public concerns, particularly in the north of Scotland, were real and an ongoing process of information and engagement was required to address them.
"Police Scotland has acknowledged the need for ongoing dialogue and has indicated that concerns raised, for example by Highland councillors, will be taken into account at the next review of the standing authority planned for September.
"The SPA will keep the issue under review - particularly around the areas of risk, health and safety, and complaints."
A spokeswoman for Police Scotland said: "The decision on a standing authority is an operational decision for the chief constable and is informed by a range of factors including evidence and intelligence from the Strategic Threat Assessment that ensures a proportionate response to the level of threat to Scotland's communities.
"The chief constable's role requires him to make these decisions free from political interference in order to take all appropriate measures to keep people safe.
"The operational independence of the chief constable is supported by the Cabinet Secretary for Justice, the Scottish Police Authority and HMICS (Her Majesty's Inspectorate of Constabulary in Scotland)."