In the unusual move, an official forum of inmates at Castle Huntly, Scotland's open prison for criminals being prepared for release, has written to Kenny MacAskill to ask him to reconsider the decision to abolish the visiting committees.
The Herald revealed in December that Mr MacAskill had decided – following a consultation – that the committees were no longer required.
However, last month, following cross-party pressure and the campaigning of the committees, he said he would conduct further consultations.
Established in 1871, the independent committees monitor and review prisons and prisoner conditions. They are made up of volunteers and the annual cost is £70,000 a year.
Mr MacAskill said the committees would be replaced with a professional advocacy service but questions were raised about how much this would cost and how it could be independent.
The letter states: "We are writing to you on behalf of the prisoners at Castle Huntly. While we are glad to hear the matter concerning the abolition of the visiting committee in Scotland is subject to further public consultation, a number of points do not seem to make any sense.
"The Chief Inspector supported retention of the visiting committee in his response to the consultation.
"Most of the time, prisons are well aware when [Chief Inspector] Brigadier Hugh Monro is to visit, thus giving them sufficient notice to get their house looking ship-shape.
"It is suggested personal officers currently provide a general advocacy service. How can a prison officer act on behalf of a prisoner against his employer?
"There are many support services for prisoners, yet these are very hard if not impossible to access for prisoners on remand.
"Also they do not hold any credibility with the majority of prisoners as they are seen to be part of the 'system'.
"Many prisoners are put off using the complaints system by the prison officers.
"Many prisoners cannot complete the appropriate paperwork due to writing difficulties and therefore need help from a member of the visiting committee.
"Prisoners are often apprehensive about submitting a complaint as it could involve a complaint against a member of staff and, as such, they look to the visiting committee to resolve the problem.
"From our understanding, there are three things a prison, by law, must conform to before opening.
"It must have a governor appointed. It must have a chaplain appointed. And finally, it must have a visiting committee appointed."
Both the Tory party and Greens have called for the retention of visiting committees.
A spokeswoman for the Scottish Government said: "We agree independent monitoring of prisons is an important function, however the consultation revealed that the current system for doing this is not as effective and efficient as it could be.
"We consulted on the independent monitoring of prisons earlier last year and that process involved the publication of a consultation paper, prisoner focus groups and interviews with key stakeholders.
"We have considered the evidence gathered and will shortly come forward with a set of proposals to establish a new independent prisoner advocacy and monitoring service.
"These proposals will be laid before the Scottish Parliament for consideration and public consultation."