Father Paul is a Catholic priest.

He is also a gay man. That may explain why he can only speak about his sexuality from under a cloak of anonymity. He comes to his door dressed in black, with a traditional dog collar at his neck. He's been a practising priest in Scotland for decades. Homosexuality and Catholicism have been the two cornerstones of his life for more than half a century. Yet to speak candidly about the realities of being a gay priest in Scotland, he has asked for his identity to be kept hidden.

"I don't know any priest in Scotland who is openly gay," he explains as we chat in his homely livingroom. "It's not amazing, it is because of the whole historical lack of openness. It takes somebody with courage.

"There's no real reason why you should come out, because you would be misunderstood. That person would find themselves in an awkward position if they had a difficult congregation."

These aren't hypotheticals – Father Paul has first-hand experience of bigotry. Years ago, back when he served as a full-time parish priest, the phone rang with an anonymous call.

"I remember distinctly a guy phoned up and he was horrified that I was gay. Although he had no reason to be, because I wasn't running around looking for boys, you know? He just said, 'it's horrible'. And I said, 'that's your problem, not mine'."

This week, the issue of homosexuality in the Catholic Church has found itself squarely in the media spotlight after the allegations of inappropriate behaviour by Cardinal Keith O'Brien.

The scandal follows hot on the heels of the church's open attack on gay marriage. Last month, more than 1000 Catholic priests signed a letter opposing planned legislation. "The natural complementarity between a man and a woman leads to marriage," it read, noting the centrality of childbirth. "That is why marriage is only possible between a man and a woman."

As these debates played out across the media, few thoughts were given to the gay priests who live and pray in Scotland and elsewhere. Their experiences, shaped at the frontline of a clash between the church they love and the lifestyle they live, remain largely untold. Out of choice or necessity, they stay hidden.

Father Paul knows he is not alone in the Scottish Catholic Church. He is reluctant to say how many gay priests he knows, not wishing to break confidence, but admits he couldn't name dozens.

"Most gay priests I have known have left the priesthood eventually, because they can't reconcile it," he says. "Some of them just want a partner. They couldn't reconcile the fact that they could be a priest and be active sexually."

He now acts as an unofficial guidance councillor for Catholic men struggling with their sexuality. Realising he was gay during training for the priesthood, the elderly priest has never seen a contradiction between his homosexuality and his religion. He tells this to those gay Catholic men – and sometimes their concerned mothers – who are directed his way for advice.

"If somebody comes to me in confession, for example, and tells me about them being homosexual then I just deal with that. I tell them what the church teaches, I affirm that the church doesn't teach being homosexual is a fault.

"Okay, it is tough if someone is homosexual and wants to act on their homosexuality. They have a problem. But there's nothing in church teaching that says being a homosexual is wrong."

He shies away from outright criticism of the church. While conceding individual points – there are pockets of homophobia; it gets harder to be gay the higher you rise; there won't be an openly gay priest in Scotland in his lifetime – he refuses to attack the institution he calls home.

No clearer is this than on gay marriage. Was the church's stance right? "The church as a whole has certainly come out against gay marriage," he says carefully. "That is just the facts of things, that's the official outlook. I can't give you my personal opinion, because that might be against what the church is teaching. Even being anonymous, it is not fair."

Being caught between what the church teaches and what the individual believes is a familiar topic for Ruby Almeida, chair of the gay Catholic support service, Quest. The charity runs five or six events every month across the UK and has about 600 concerned people calling its helpline every year.

"What you have to understand is that if you are a priest you are, in effect, an employee of the hierarchy of the Church," she says. "So it is very hard to be stand up and be counted.

"If you do say what you truly feel about homosexuality and same-sex marriage, you are probably going to be hounded out of the church. Unfortunately, their hands are tied, their lips are sealed and they are afraid to speak out very publicly."

One man who did speak out was Father Bernard Lynch. The 65-year-old Irishman is one of the few openly gay Catholic priests and has led the charge for liberalisation of the church. Last year, he published a book revealing he had been married to a man since 1998. The Vatican soon intervened.

"The Vatican does not deal directly with plebs like me," he says dryly. "They always try to come to me through my superiors or those who have charge over me. Of course, it was communicated to me that this was totally, utterly and absolutely unacceptable."

Lynch was thrown out of his order, the Society of African Missions, on grounds of a broken celibacy vow, but insists he remains a priest. "It felt very hurtful and extraordinary unjust," he says, while admitting a married straight man could be treated the same. "Of course, if any priest comes out as gay he loses his job automatically."

It was not the first time he had suffered due to his sexuality. He has written a book describing how, in the 1980s, his Aids campaigning in New York riled Rome and led to the FBI bringing trumped-up charges of child abuse. He explains how the judge dismissed the charges out of hand, turning him into a cause celebre for progressive Catholics.

The church's stance on homosexuality, Lynch says, is corrosive. He believes about 40% of priests are gay. "The thing that is the most harmful to LGBT people is to label them as disordered," he says. "Then it goes from being harmful to destructive when the Vatican calls the love shown by gay people in a marriage or relationship 'evil'. To me there couldn't be anything more destructive of the soul of somebody than to call love evil. In fact, it's a contradiction in terms."

But doesn't the church specifically teach that homosexuality is wrong? "It may be against the teachings of the Catholic Church, but the Catholic Church has taught slavery," says Lynch.

"The Catholic Church was against democracy, was against suffragettes, against the vote. So I don't look to the Catholic Church even as a Catholic. Ultimately we look to the Gospel. What does it say in the Gospel about homosexuality? It says nothing."

The historical context of the debate about gay priests is a curious one. Professor Mario Aguilar, who teaches divinity at St Andrews University, said these conversations simply weren't happening in the Catholic faith until recently. "When one looks at the history of the church, one must remember that not all the concerns we have today – for example, concerns about the disabled, gay marriage – were the concerns 30 years ago. There has been an evolution."

It was only the arrival of the Aids epidemic in the 1980s that got the church talking about homosexuality and it wasn't until the last decade that gay priests became a point of debate. "The discussion was pushed within Europe," Aguilar explains. "This was not a topic entertained in Latin America, Asia or Africa. And the European discussion came particularly when society evolved, gave gays rights and then, in the last 10 years, created civil partnerships."

Some might be heartened by the short history of the debate – progress has already been made in a brief period – but Lynch says problems remain internally. Indeed, he says it is, in fact, other gay priests, rather than straight ones, who get most angry with his open homosexuality, fearing it may lead to their exposure.

"My greatest friends would be priests who are straight and my most hurtful, harmful, destructive friends are priests who are gay, closeted and very often involved in sexual relationships themselves," he says. "I have been reported to bishops by gay priests saying that I shouldn't be able to do anything because I am so open and honest."

With the issue of homosexuality and the Catholic Church once again in the spotlight, it seems the reaction of some gay priests is just to slam the shutters. "They don't want the light," says Lynch with regret. "They want to have their cake and eat it."

Some liberal Catholics feel that perhaps the church needs a period of instability to shake itself into the 21st century. Quest's Ruby Almeida certainly thinks so. "The church is in absolute turmoil at the moment," she says. "It is lurching from one crisis to another. And in a way that doesn't bother me. It is important these things happen. The more you shake the trees the more the rotten apples are going to fall off."