Swivel partners

"There's something special about marriage," he said, a predictable enough line at a Tory conference, before adding: "And by the way, it means something whether you're a man and a woman, a woman and a woman or a man and another man." This qualification was delivered in such a way that invited applause, not just from the hall in general but from the Tory frontbench. Some of them didn't look too thrilled at being bounced into such an affirmation, but nevertheless they applauded their leader's sentiment.

This amused me, but I also recall being surprised by the man who would, four years later, become Prime Minister. For David Cameron – in many ways a throwback to the Old Etonian elite that had run the Conservative Party until the rise of the grammar school boys and girls – had not been compelled to include that line in his speech. Of course, it is clearer now what he was trying to do: not only detoxify the Tory brand (perhaps the central element of the Cameroons) but also soften up the rank and file for the legislative manifestation of that liberal sentiment.

It wasn't that the Tory Party was necessarily homophobic. As a young student activist in the late 1990s, I remember being struck by how relaxed the party, or at least its Scottish Conservative wing, was with an identifiable strand of gay activists. I was more surprised still when, during the 1999 Scottish Parliament election, I had cause to visit the Scottish party's HQ in Edinburgh, and found that a majority of its modest-sized staff were gay men.

Still, a party's attitude to such things ought to be judged by its deeds rather than its officials. On this, I considered the Conservative record mixed rather than the overtly anti-gay caricature peddled mainly by political opponents. Even Margaret Thatcher had supported decriminalisation in the late 1960s, while her first government belatedly (and admittedly rather grudgingly) brought Scotland into line with the rest of the UK. The fight against Aids in the 1980s, meanwhile, was sincere and effective.

And while Thatcher was not personally prejudiced, her capitulation to backbenchers over Section 28 (Section 2a in Scotland) remains the benchmark by which Tory attitudes to homosexuality are judged. This is understandable, for it was a mendacious piece of statute that, even if rarely given effect, was certainly nasty in intent. When the first devolved Scottish Executive sought to expunge it, the Scottish Tories said it wasn't a priority, thus reinforcing perceptions of what Theresa May called the "nasty party".

Which makes the Tory split over same-sex marriage intriguing. Ruth Davidson, openly gay, is Scottish Tory leader, and so relaxed is the political climate in Scotland that her sexuality was barely mentioned during the 2011 leadership campaign. (One Tory friend recently suggested to me, only half-joking, that many members simply didn't understand what a "lesbian" was.)

Davidson, naturally enough, supports same-sex marriage, and while it seems likely the Scottish Tory grass roots (such as they are) aren't as keen on the idea, there has been no English-style revolt. But then the SNP Scottish Government, itself split, has been dragging its feet over the formal introduction of legislation at Holyrood. Indeed, I suspect it will not happen this side of the independence referendum.

The Roman Catholic Church, in spite of its recent travails, also shapes SNP thinking in this respect. The First Minister recently went out of his way to praise Cardinal Keith O'Brien (who believed gay marriage to be a "grotesque subversion of a universally accepted human right") as a "considerate and thoughtful leader" of his church, "constructive in his approach" and responsible for "a lifetime of positive work".

I remember flinching at the lack of any qualification and, indeed, Salmond was conspicuous in being the only prominent public figure to defend O'Brien in this way, no doubt keen to reassure Catholic voters he was on their side even when no-one else would stick their neck out.

By contrast, the First Minister has had little involvement with Scotland's national Church, which last week grappled with its own modernisation agenda.

The Kirk is nowhere near recognising same-sex marriage, but after two years of fudge, delay and very real fears of a split, it last week decided to allow congregations to elect to ordain gay ministers who are in civil partnerships. I watched from the floor of the General Assembly as commissioners debated, in a rather technical and dry manner, what was clearly a momentous change for a body unused to making decisions under such public and media scrutiny.

But while on the one hand both the Kirk and the Conservative Party are looking to the future in adopting a more gay-friendly position, I also can't help feeling they're looking back, seeking to maintain – in an altered form – the stable (monogamous) relationship both organisations believe lies at the heart of a stable society.

Take the Prime Minister's famous quote: "I don't support gay marriage in spite of being a Conservative. I support gay marriage because I am a Conservative." There is a certain logic to this. If there truly is "something special" about marriage, then its proponents should want it extended as widely as possible. In other words, what better way to shore up marriage than by allowing more people to get married?

On this, perhaps paradoxically, I do not have strong views. The civil partnership legislation struck me as both necessary and right, but although I'm gay I am not religious, so I've never quite seen the compelling need for church-recognised unions. Nor does the traditional concept of marriage strike me as either necessary or desirable. That said, this is clearly a very personal view. What right have I or, indeed, has the state, to deprive gay couples of faith from formalising their union?

Thus the most persuasive case is that of basic equality. But such arguments do not appeal to traditional Conservatives and, indeed, the nub of the battle currently raging within the Conservative Party is values-based. David Cameron and George Osborne represent a more socially progressive strand in the party; relaxed, liberal and metropolitan – but it is one increasingly held in contempt by many Tory MPs, and perhaps to a greater extent by the party grass roots.

They almost regard Cameron et al as foppish usurpers, legislating for gay marriage and generally propagating the politically correct agenda of Blair and Brown. To them, it's baffling and unwelcome. Aren't Conservatives meant to conserve, rather than initiate radical social changes? Inevitably, more conservative forces are exploiting this; UKIP, for example, is conflating a general unhappiness with change – immigration, the European Union and gay marriage – with the resulting cry to "take our country back" both constitutionally and in terms of "values".

This clearly frustrates the Conservative Party leadership, which has hardly handled the resulting split deftly. Remarks attributed to party vice-chairman Lord Feldman that many activists were "swivel-eyed loons" simply served to confirm the disenfranchised fringe's worst suspicions: not only was a metropolitan Tory elite departing from true conservatism, but they were sneering at ordinary foot-soldiers. And it doesn't help, even with Tory activists, that they're mostly public schoolboys. Grass-roots Conservatives generally tolerate posh boys as long as they're competent, which presents the current crowd with an obvious problem.

I also find it difficult to dismiss out of hand sincerely-held views on any subject, and that includes opposition to gay marriage. This is where Cameron has played his hand very badly. His detoxification project was the right one, but poorly executed. Despite a long process of softening up his party (including the 2006 speech mentioned here), he has failed to win the argument with his own troops, swivel-eyed or not. Instead, he appears, in their eyes, both contemptuous and arrogant.

The result was played out on the floor of the House of Commons. In the second reading vote on the Marriage (Same Sex Couples) Bill, more than 100 Tory MPs voted against (they didn't technically "rebel", it was a free vote). They did so for a variety of reasons. Some, though not all, were genuinely opposed to same-sex marriage, others felt under pressure from their constituency associations, while a hardcore of Tory malcontents dislike their leader so much that it became simply another convenient stick with which to beat him.

But there is another way of looking at that vote and, indeed, last week's third reading. Although well over 120 Tory MPs voted against, the same number also actively voted for same-sex marriage – a split, certainly, but also a margin unthinkable even just a decade ago, when Commons divisions on lowering the age of consent routinely attracted no more than a dozen Conservative votes. It may not be obvious to the uninitiated, but the Tory Party has moved a long way.

Interesting, too, was the reaction, on Twitter and elsewhere, to Tory MP Sir Gerald Howarth's speech about "aggressive homosexuals" (by which he meant the gay rights lobby). It was considered eccentric, almost a blast from the Tory past, and jarred with the more nuanced and respectful opposition of his colleagues. Indeed, the second and third reading debates showed the much-maligned House of Commons at its best: passionate and tolerant of opposing views. The revolution, however, is clearly incomplete.

And it is worth remembering that this UK battle isn't taking place in isolation. In France, where massive pro and anti demonstrations filled the Paris streets, the vehemence of the disagreement makes an internal Tory Party dispute look like a minor fracas in an Edinburgh gay club. A few days ago, a prominent French historian placed a suicide note in Notre Dame before shooting himself dead in the cathedral in protest over his government's intention to legalise gay marriage. Mercifully, there has been nothing comparable on these shores.

AS of mid-May, 12 US states – including an uninterrupted swathe from New York to Maine – now allow gay and lesbian couples to marry. Minnesota recently joined the club, although constitutional bans remain in place in 31 states. The US Supreme Court, meanwhile, is expected to rule next month on the Defense of Marriage Act, under which the federal government only recognises heterosexual marriages. Equal rights campaigners are confident of success.

Wiser heads among Conservative critics detect this wind of change and are equally conscious that the dividing lines are generational. According to an ICM poll earlier this year, 37% of over-65s support the idea of gay marriage, compared with 72% of those below. Martin Vickers, the staunchly traditional Tory MP for Cleethorpes in North East Lincolnshire, even acknowledged as much in his speeches opposing the legislation. In some ways, this represents the Old Guard's final defence of a rapidly diminishing mindset.

David Cameron, meanwhile, was criticised for not being more visible during the two set-piece Commons debates, but then he has had little choice but to tread carefully. Speaking after last week's vote he was more frank, and also very moving, talking about how "young boys in schools today, who are gay, who are worried about being bullied, who are worried about what society thinks of them", could now see that the "highest parliament in the land" had deemed their love the same as everyone else's love.

"I think they'll stand that bit taller today," he added, "and I'm proud of the fact that has happened."

Cameron, I suspect, had hoped to draw a line under the gay marriage controversy with this bill, even if that meant relying – humiliatingly – on Labour votes (unlike the Conservatives, no-one questions their pink credentials). That might still be the case.

I've spoken to Conservative MPs – both for and against – who believe that at the next General Election, same-sex marriage simply won't be an issue.

By that point, there will have been two years of ceremonies, two years of gay and lesbian couples walking down the aisle. It will be the norm, and no serious party could possibly fight an election promising to reverse it – not even UKIP.

David Torrance is a journalist and author. His books include We In Scotland: Thatcherism In A Cold Climate and Whatever Happened To Tory Scotland?