The committees are made up of volunteers and are responsible for monitoring Scotland's jails.
On a routine visit to Dumfries Prison in January, committee members Stuart Clement and Dianne Stewart were approached by inmates who raised concerns about medical treatment, including alleged difficulties with some of the health care team.
Mr Clement called committee chairman Phil Hughes, a former prison governor in England, about the healthcare provision. Mr Hughes visited the prison and reported back that no prisoner had complained to him.
On February 23, Mr Clement and Ms Stewart wrote to the governor at Dumfries Prison to reiterate their concerns and offered to meet with him to share their information.
Three weeks later, the deputy governor replied to Mr Hughes, saying the tone of the letter was "disappointing". He added that, following an investigation by the health centre manager, they believed the complaints to be "unsubstantiated".
He asked whether they would "reconsider their assertion".
In March, Mr Clement and Ms Stewart received letters from the committee clerk saying they were both suspended pending a review by Dumfries and Galloway Council, which is responsible for recruiting committee members. The letter does not say why they have been suspended – a point the council has been unable to clarify.
Mr Clement said: "We were working as volunteers and yet we were suspended for raising the concerns of prisoners.
"Our complaints were not treated as seriously as they should have been. If this is what happens, what is the point of the visiting committees? It doesn't make sense. We were told by one committee member we shouldn't have spoken to the prisoners, but that is our job."
A source said: "It seems confusing. Some people feel they exceeded their remit."
Joan McAlpine, SNP MSP for South Scotland, contacted the Justice Secretary to raise her concerns about the matter.
Mr MacAskill wrote back saying he has asked senior Scottish Prison Service officials to set up a meeting with her and Mr Clement and Ms Stewart "in order that these concerns can be explored".
Ms McAlpine said: "I'm very pleased the Cabinet Secretary has asked senior officials to meet with Dianne Stewart and Stuart Clement. It is my belief an injustice has been done. They were doing their job in raising concerns.
"They are upstanding members of the community who were giving their time for free.
"We have heard a lot of talk about visiting committees recently, but in this case the committee seems confused about what due process is."
Mr Hughes declined to comment.
A spokesman for Dumfries Council said: "A complaint has been received by the council. We are investigating this complaint. It would, therefore, be inappropriate to comment at this stage. We expect the investigation to be completed within a few weeks."
A spokesman for the Scottish Prison Service said: "The decision to suspend the members was taken entirely by Dumfries Visiting Committee.
"SPS has no locus in the visiting committees."
The Herald revealed earlier this month that the 16 historic prison visiting committees will be scrapped and replaced with three independent monitors.