The asylum seeker, known as AB, entered the UK illegally by lorry in 2008 and since then has spent a period of time in a psychiatric unit at Glasgow's Stobhill Hospital.
He exhausted his rights of appeal years ago but challenged a UK Border Agency (UKBA) ruling from last year in a judicial review at the Court of Session in Edinburgh.
The UKBA had rejected his claim based on sexual orientation, saying there was no realistic prospect of an immigration judge finding he had established he was gay or would be treated as such by potential persecutors in his homeland.
However, Lord Stewart said the decision was "technically flawed", and a new ruling should take into account mental health material that was omitted.
The judge said: "If our border control system is dysfunctional, it is not the job of the courts to make it more dysfunctional.
"The petitioner should have left the UK four years ago when he became rights-of-appeal exhausted: but we are where we are and the decision whether this petitioner is now to be removed or is now to stay should be properly made." The judge pointed out he was not making a decision on the 29-year-old man's sexuality or perceived sexuality.
When he arrived in Britain AB told the UKBA he did not consider himself to be gay but was considered as such by family and acquaintances.
Other members of his boxing club would not spar with him and a brother, who is an influential cleric, reportedly thought he was gay.
His lawyers made further submissions to the UKBA last year, supported by psychiatric and social work material and a letter from a male with whom AB claimed to be in an eight-month relationship.
He also claimed to have had four brief physical relationships with different men. He disclosed to social care services he is homosexual and struggles with religious and emotional issues linked to his sexuality.
The decision maker in his case questioned why he had waited until about a month after the Supreme Court ruling in 2010 before disclosing he had become "a practising homosexual".
He claimed he would be subjected to persecution and execution if he was sent back to Iran.
Lord Stewart said it was advanced on his behalf he has suffered a breakdown struggling to come to terms with his sexuality.
AB was diagnosed with schizophrenia and hears voices from an angel of God. He believes he is in contact with famous people, such as President Obama.
He was sectioned for almost three months following an assault on a UKBA employee, according to a medical account.
A psychiatric nurse reported the had interrupted a Catholic mass on a few occasions by walking on to the altar and declaring he was the son of God and could cure cancer and HIV.
Lord Stewart said mental health material referred to in the UKBA decision letter was on the face of it potentially relevant to the issue of whether he was homosexual or was at risk of being perceived as such if returned to Iran.
However, the judge said it seemed the decision maker had not brought that material into the evaluation of the prospect of success for AB's sexual asylum claim.
Lord Stewart said: "I have to conclude the decision maker did not bring the mental health material into her evaluation of the prospects of success for the petitioner's homosexuality claim and this was an omission."