Moi Ali, the Judicial Complaints Reviewer (JCR), also said she was "really baffled" that the SNP Government had not embraced reform, and claimed the country was lagging behind England.
MSPs yesterday welcomed the intervention.
Judges are responsible for probing complaints against their colleagues under the model of self-regulation, overseen by the Judicial Office for Scotland (JOS).
The rules that govern the system are also drawn up by the Lord President, who is the head of the judiciary. Ali can step in if an individual believes a complaint has not been handled properly, but her powers do not include ordering re-investigations or imposing sanctions.
Her second annual report is published tomorrow and it reveals she found 20 breaches of the rules last year.
However, in an interview with the Sunday Herald, Ali, 50, backs an overhaul of self-regulation.
"Fundamentally the problem is the legislation ... it's judges judging judges' conduct.
"I'm presented as the independent element, but without the powers I can't be independent."
She added: "Without any proper, external, genuinely independent oversight, you're not going to have public faith and confidence."
Ali, who also sits on the boards of the Scottish Police Authority and the Scottish Ambulance Service, believes the limitations of the post are stark.
She said: "I've made some small differences and they are small ... But really it's difficult to make an impact within the constraints that I'm in at the moment. It's a bit like being in a straitjacket."
South of the Border, the equivalent ombudsman has staff, a budget of £500,000 and beefed-up powers.
Ali, by contrast, is on her own and has a budget of about £2000 a year.
"Citizens here have a lot less protection than they do in England and Wales," she said. "I think that Scotland is leading the way in all sorts of areas - healthcare and education - but here, this is probably one of the few areas where Scotland is playing catch-up."
Asked why the SNP Government was resistant to changing the complaints system, she said: "I have to say I don't know, I'm really baffled."
In retrospect, Ali believes the JCR post was not taken seriously by those who created it. "I'm sorry to say that I do think there was an element of window dressing.
"I think that for any professional group, whether it's the judiciary or any other powerful group of people, it's quite difficult to take them on."
On the subject of her tiny budget, Ali said she recognised there was no appetite for a "great big quango", but noted: "It seems to have gone too far the other way and there's been an attempt to create something on the cheap.
"I know people will be very unhappy with me using the term 'window dressing', but I think there is an element of that."
However, Ali has helped reform the way in which the JOS conducts the investigation process.
The Lord President has agreed to inform her of the final outcome of any referrals she makes to him, while a summary of the initial JOS investigation report will also be provided to complainers.
Both changes resulted from Ali's pressure. Even so, she is realistic about the capacity for meaningful change within the status quo.
"If I were asked to create something to deal fairly, effectively, efficiently [and] transparently, with complaints about the judiciary ... I would not invent this."
She is highly critical of the Lord President's rules that govern the investigation system: "They are legal rules written by lawyers, for other lawyers to use. To me, the perspective is completely wrong."
She added: "If you have a set of rules that you can pick up and not understand, then they can't be fit for purpose. They are not written in an understandable way."
She has contributed to the Lord President's consultation on changing the rules, but says the practice of judges investigating their colleagues is the bigger problem:
"All of the correspondence I've had, people feel that's not right, that it's not fair. Even if the Judicial Office act completely fairly, and apply the rules fairly, public perception is really important."
She does not regret taking up the post, but said her stint had been "enormously frustrating and difficult".
Scottish Liberal Democrat justice spokeswoman Alison McInnes MSP said: "Moi Ali has been admirable in her pursuit of transparency within the judicial system. The Scottish Government should treat her concerns with seriousness, as the current system of self-regulation is not as transparent as it could be. It is clear that there is more work to be done to ensure public confidence in the judicial system."
Scottish Conservative justice spokeswoman Margaret Mitchell said: "If the Judicial Complaints Reviewer believes her position is simply window dressing and that the current system is not fit for purpose, then the Scottish Government should look into these concerns."
A spokeswoman for the Judicial Office for Scotland said: "It would be inappropriate to comment in advance of the publication of the Judicial Complaints Reviewer's report on December 16."
A Scottish Government spokesman said: "The JCR has carried out only a small number of reviews since the post was created two years ago. It would be premature to review the powers of the role at this point in time."