Lex Brown shouts down my mobile as I reach the grassy car park at the Arisaig Highland Games, held at Traigh Farm. ''You can't miss me,'' he booms, ''I'm wearing a brown kilt and beret. I'm in the Clanranald tent.'' I park the car and hoof it past some dancers and kilted behemoths competing in various tough man challenges. Finally, I catch sight of him and find him in a welcoming mood. Brown, owner of Castle Tioram, the 13th-century Highland home of Clanranald, on Moidart, has been variously described to me by individuals opposed to his ambitious (pounds) 4.5m restoration plans for the medieval fortress as someone ''overwhelmed by his own sense of majesty'', ''a debunker and spoofer'', a man who ''has risen without trace'' and a ''complete enigma''. Since his synthesis of commerce and soldiering skills (he is a decorated ex-RAF pilot) led him to purchase the building once acclaimed by Winston

Churchill as the second most beautiful place on earth - surpassed only by the great lakes of Africa - Brown has been plagued by unpleasant rumour and innuendo.

Described by one of his former colleagues as someone ''who lives life in the fast lane as if he is still flying a helicopter at 180 mph'', in the flesh he is so effortlessly charismatic that any previous criticism instantly diminishes. He could charm the birds from the trees. Today Brown has, by his own admission, been partaking rather generously of the hospitality. There's a case to be made for joining him but, given that I have another hour's drive ahead of me to the castle that has been at the centre of a seven-year dispute between Brown and the heritage body Historic Scotland, I decline. Meanwhile Brown is holding court, looking somewhat resplendent alongside his wife, Frances, 37, almost 20 years his junior. A lawyer from London, she married him six days previously, following an engagement lasting more than 15 months.

After a little chinwagging with a few tartan- festooned Americans and Ranald Macdonald of Clanranald, the 25th chief and captain of Clanranald, who is head of five million clan members worldwide, Brown sighs and laments the ''bureaucratic idiocy'' that has seen his project stall in the seven years since he purchased the castle, with its spectacular views across Loch Moidart, for (pounds) 300,000. It was put up for sale in 1997 for (pounds) 100,000 by its former owner, Wiseman Macdonald from California, who bought it in the 1920s. So far Brown has spent around (pounds) 2m on his medieval money pit - on lawyers, architects and surveyors' fees - despite being unable to do any work on the castle, which has deteriorated significantly in the meantime.

His plans to meticulously restore the building with virtually complete fidelity to its past were backed by Highland Council, but Historic Scotland has objected to the granting of scheduled monument consent, essential to the project going ahead. The agency argued the proposals were detrimental to both the historic fabric of the monument and its cultural significance. A public inquiry lasting 30 days was held and, in February 2002, the Scottish Executive backed the decision to refuse permission. Anta Estates Ltd, Brown's company, submitted revised proposals in an effort to address these concerns. Historic Scotland, however, said the new plan was ''essentially the same'', and that it would be more appropriate to preserve the castle as a historic ruin. The body has offered to pay Brown 85 per cent of the consolidation costs to prevent its collapse.

''You must go to the castle,'' he enthuses, while we watch some teenage dancers performing to the Sailor's Hornpipe. ''It's wonderful. But the bloody thing will probably fall down before you get there.'' Since 1998, public access to the interior has been prevented for safety reasons, after Highland Council closed it. ''In seven years it's got worse. Every time I go there there is something else that is deteriorating. It's a tragedy, an absolute tragedy. Laughable, tragic, risible, pathetic, disgraceful,'' he says.

We chat a little longer before he is accosted by some more of the tartan-clad diaspora around him. I go for a walk past Ewan, a young piper, playing for the dancers, past a hot dog stall, someone selling jeans, and a collection of young men the size of stags, throwing 15lb hammers. At the bouncy castle I meet up with Brown and Frances again and we watch a group of muddy youngsters competing in the hill race. In the spirit of helping the local community, Brown suggests I stay at the Loch Shiel Hotel in Acharacle, the next village to the castle, and we agree to meet in the morning.

It is the stuff of a Boys' Own story, an intriguing tale of the divisive influence of money, whispers behind closed doors and an international man of mystery and action, not to mention dark revelations and accusations: all set against the brooding history of one of Scotland's finest castles. The saga of Lex Brown and Castle Tioram has left opposing sides fighting like vultures over their carrion. It has attracted a mixture of both widespread support and opprobrium over whether a wealthy owner should be able to turn part of the nation's heritage into his own modern home - ''a McBarratt house'' according to one detractor - or, in a perfect world, have it left to the nation as a romantic ruin with access for all.

Castle Tioram (pronounced ''cheerum'') is built on a small rocky island. Signs on the causeway warn visitors of the risk of falling masonry. For most of its existence the castle belonged to the Lord of the Isles and is the ancestral home of the Macdonalds of Clanranald. In the 1715 rebellion, after capturing their old castle from the Hanovarians, legend has it that the Macdonalds set fire to the building rather than let it fall back into enemy hands. It has stood as a romantic monument to the Jacobite uprising since.

When Brown, who lives in London, bought it in 1997 he set out to convert part of the fortress into a residential flat, while restoring the rest to include a clan museum to be opened to the public at a cost of around (pounds) 4.5m - with funding provided entirely by himself.

Near the castle we scramble across the mud. Alongside us is Alistair, who keeps the keys while the 56-year-old millionaire is away on business. ''The accountability is unbelievable,'' says Brown, whose interest in the castle has turned into an obsession. ''Who are they?'' he asks of Historic Scotland. ''How do they make their decisions? There were doubts expressed by Highland Council at the planning permission stage but they sat down and our application was assessed with intelligence. We came to an absolute compromise. The most critical aspect was public access. We worked it out.''

For Historic Scotland, however, public access to the castle is the sticking point. Brown wants public access 49 days a year, split between local and tourist visiting days. Historic Scotland wants year-round access.

Alistair fiddles with a padlock. ''There are a few dissenters,'' he tells me drolly, ''but they're the incomers who don't want bloody anything here to change, they're the ones who want to preserve us in aspic.'' The building has the sad air of a structure overtaken by history, politics and the elements. The north-west wall appears in danger of collapse. ''People like the Society for Protection of Ancient Buildings try to pin the closing-down on me,'' says Brown. ''But I didn't close it down. Highland Council closed it down.''

Without urgent repairs, he believes, the castle could start falling into the sea within as little as five years. How does he react to claims that, if his plans are allowed, Castle Tioram has the potential of setting an immensely damaging precedent for the care of Scotland's ruins? ''I think Historic Scotland are setting the damaging precedent,'' he replies. ''Their stewardship is horrendous. They won't even allow me to put a roof on it. It had a roof before. We found out that the tiles came from Norway and it would be restored to its original condition.''

According to Brown a number of prominent bodies, including the local community council, the Royal Fine Arts Commission, the Scottish Environmental Protection Agency and Scottish Natural Heritage, have supported his plans. He has nothing but contempt for Historic Scotland. ''Their traditional method of 'drag it out and you'll run out of money' has not worked with me. They are an unaccountable clique serving their own interests. If you think they are playing fair and impartial, you believe in fluffy bunny rabbits under your bed. They want access 365 days of the year. If we don't do something no-one will have any access at all. It's not going to exist.''

After a brief tour Alistair locks up before we head back to Acharacle for lunch. A couple of tourists ask to look around. Brown explains that it is too dangerous. Then he sees their

disappointment. ''Jump in,'' he says, turning on the charm.

According to Historic Scotland, the application for scheduled monument consent was rejected on the recommendation of an independent inquiry reporter, following a full and open public local inquiry held in the area at which all parties had the opportunity to be heard. ''Ministers cannot safely reverse or amend the decisions taken on that previous application,'' says a spokesperson. ''The minister for tourism, culture and sport has asked for a report on the current condition of the castle and for a 'statement of cultural significance' to be prepared, which addresses some gaps in our understanding of the castle that were identified by the reporter. Any decisions that require to be made in future by Scottish ministers about the castle will be informed by the information these provide and will also take account of the statutory, international convention and policy obligations that are upon Scottish

ministers.''

The first time Lex Brown saw Castle Tioram he ''very nearly demolished it''. He was an RAF helicopter pilot at the time and there was a blizzard when he was flying over it. ''I had 20 years flying helicopters,'' he says, ''during the course of which we always used to love coming to the north-west coast of Scotland because it was such a beautiful area to fly in. Once, I was looking for somewhere to land and this building came up in front of me and I quickly veered off. It was my first real introduction.''

Brown was born in Maybole, in Ayrshire, and spent most of his formative years growing up in Egypt and Ghana with his parents, both Scots. Like a lot of his generation, his father, a civil engineer, sought work overseas. ''There wasn't too much at home.'' His mother, he tells me, used to sing songs in Gaelic celebrating Tioram's beauty. ''But it never really registered.''

On top of his years as an RAF pilot, there are rumours Brown was once a member of the SAS. When I ask, he just shrugs, smiles and answers cryptically: ''It was a lot of fun, and sometimes more serious than fun.''

Colonel Clive Fairweather, a former deputy commander of the SAS now working for a public affairs and security firm, is an old comrade-in-arms. ''I think he [Brown] was probably the most senior flight lieutenant in the Royal Air Force,'' he tells me. ''He was one of the RAF's best helicopter pilots, but always teetering between possible court-martial or gallantry medal. He ended up with lots of medals. And he was always dodging the next trail of women as well as dodging any posting that involved normal duties. He didn't like normal stuff. He never got beyond the rank of lieutenant for these reasons.''

According to Fairweather, Brown flew ''all over the place'' in many of the world's trouble spots, including numerous tours of Northern Ireland. ''That's where the SAS bit comes in. He was never in the SAS, but he would be involved. That's how I knew him. He was the sort of guy who fished you out of whichever trouble spot you were in in the dead of night.'' He adds: ''He's as clear as a bell, an absolute charmer, very amusing. He's quite a private individual and extremely determined. We always knew that. We always knew that if he was coming in a helicopter to get us out, he would.''

After leaving the RAF, Brown ''farmed Loch Lomondside for a while'', then went into business. ''I got offered a job advising on aircraft down in London. I just rocked and rolled. Air cargo. I'm a specialist in aviation and, with my military background and the fact I'd had a lot of friends around the world, a lot of governments, I provided specialist information and consultancy, offshore work, oil exploration, whatever. That was all quite successful. It paid well.'' He sits upright in his chair. ''It's who you know, not what you know, more than anything else. Trust me, 20 years of a military background, it cuts out all the CVs if you are known and trusted. I was very well connected at government level around the world and that has continued.'' Asked what exactly he means, he just smiles. ''I got to the stage where I hated London. Tioram came up.'' No sooner have I dipped into his personal

history than I am being shown the door.

There has been a lingering question mark hanging over whether Brown actually owns Tioram, and this is a matter he is eager to explain. After he bought the castle, there was a suggestion that he was acting on behalf of Bronson Conrad, a Canadian millionaire linked to the goldmining industry in war-torn Sierra Leone. In 1999 a newspaper report suggested Brown was simply a middleman for Conrad (who died in 2001 with Brown at his bedside), the then chief executive of a Canadian company that mined diamonds and gold in Sierra Leone. At the time, according to Conrad's lawyer, Conrad had intentionally duped the local castle community. ''We deliberately kept things as opaque as we legally could,'' the lawyer is reported to have said. Conrad allegedly outbid the community for the castle because he felt that what they wanted to do to it was inappropriate. His lawyer maintained the bid was an action

creating no profit.

It was alleged that, while Conrad put forward money to buy and later fund the castle's restoration and upkeep, Brown would handle the day-to-day running of the ruin. In a subsequent newspaper story, also in 1999, Conrad was quoted as saying he wanted to ensure Castle Tioram would last another

900 years.

Brown, who admits he was friendly with Conrad, says he bought Tioram through Anta Estates Ltd, which is registered in Charlestown, Nevis, West Indies, for the purposes of fiscal efficiency. ''I set up a trust, and Bronson Conrad was one of the protectors in this trust. He was an old military Canadian friend of mine. I'd known him a very long time. He left the military when I left the military and went into business. He was one of the main trustees. It was a trust set up so that upon my death the castle would be owned by the trust and then whoever wanted to buy it had to buy it off the trust. It stops it being decimated by tax, an efficient safeguard for the ongoing future of the castle.''

According to Brown, funds for the venture were provided by a non-UK benefactor ''as an absolute gift.'' He did not confirm the identity of this benefactor.

The conservation of the castle is being undertaken by the Highland Stronghold Trust (HST), according to Brown a charitable body that owns Anta Estates. ''Anta Estates is my company. I was going to buy more properties through the HST. There are a number of other little pockets around here I own. Bronson helped set it up and was part of that trust tax structure. The ownership is 100 per cent mine.''

Conrad was chairman and chief executive of a Canadian mineral firm, and was described on its website as a financier-entrepreneur with extensive international interests. The firm ceased trading around 2001 and was delisted in April 2004 in Canada. The project in Sierra Leone was listed as Conrad's major interest.

According to a report into the diamond trade in Sierra Leone, published in 2000 by a Canadian NGO called Partnership Africa Canada (PAC), the company for which Conrad worked was one of three Canadian operations that provided military supplies and acted as a go-between for mercenaries in the seven-year civil war in Sierra Leone. They were, PAC claimed, ''willing to go to almost any lengths, including the provision of arms and mercenary armies, to get diamonds''. The allegations were denied, and there is nothing to suggest Brown was aware of any of this throughout his relationship with Conrad.

A close friend of Brown's, Colin Orr-Ewing - son of the late Tory minister Ian Orr-Ewing, and currently executive chairman of River Diamonds plc, a UK-based international diamond-mining company - also knew Conrad. Orr-Ewing says that, while Conrad was often mentioned as owner of the castle, he ''always felt that he might have been a 'nominee' for Lex. Lex is a friend of mine and it has always seemed strange to me that Historic Scotland have opposed the restoration of the castle.''

Rhoda Grant, the former Highland and Islands Labour list MSP, is only too aware how the case of Castle Tioram has driven many of the locals of Acharacle and various conservation bodies to tear at each others' throats. Having been an outspoken supporter of Brown as an MSP before losing her seat, she is now paid by the former military man to give ''political advice'' regarding the castle. ''If I had a problem, it would be very difficult for me to do that,'' she says.

Asked if there is some kind of agenda against Brown, she only serves to deepen the mystery. ''He won't be able to come out and ever publicly say what he did because that could cause problems all over the place.'' There's a bit of the James Bond about it? ''That's maybe not too far off,'' she jokes.

Earlier this year, David Breeze, the chief inspector of ancient monuments, and Sheenagh Adams, Historic Scotland's director of heritage policy, visited the castle. ''But nothing has effectively happened,'' says Brown, emphatically. ''Historic Scotland must put their house in order and come to a sensible compromise. There should be a fair and open method of challenging decisions and a transparent process of decision-making. At the moment there isn't.''

Not everyone agrees with his assessment. Denis Mollison, trustee of Clanranald Castle Tioram Trust, insists Brown is ''throwing money around, trying to get his way''. He also insists that ''local opinion is not entirely for him.'' Then he adds: ''We're not anti-Mr Brown. The trust's view is that we were in favour of consolidation as a ruin but if Mr Brown legitimately gets permission to turn it into his house, then we would be happy to collaborate with him for the good of the local community. Meanwhile it [the castle] should be opened up again.''

Hugh Donaldson, a local crofter, accepts that there are two sides to the argument going on in the village of Acharacle, but believes Brown has offered the best solution. ''We should be biting his [Brown's] arm off for it. It's the Brigadoon version they [Historic Scotland] are after. The kind of attitude they have is sepia-bloody-toned Scotland.''

Historic Scotland, meanwhile, still argue that there is no firm evidence to support Brown's ''speculative'' plans for the castle. ''Historic Scotland met with the owner in a genuine effort to find a solution,'' says a spokesperson. ''Historic Scotland's only motivation has always been to safeguard the future of Castle Tioram and see public access restored.''

However, Ranald Macdonald of Clanranald is in no less a forgiving mood than Brown. ''I think Historic Scotland's argument is complete and utter restrictive rubbish,'' he says. ''The fact it has been a free-for-all ruin for the past 100 years doesn't mean it has to remain that way. It will benefit Clanranald and everyone else who wants to come and see it. You've got to use common sense. Do things. Don't stop things.''

Ben Tindall, one of Scotland's leading conservation architects, disagrees. ''The Highlands has had enough of millionaries using our heritage as their plaything,'' he says.

There have been times, says Brown, over the past seven years when he has questioned whether it was worth carrying on. He got very depressed about the state of Scotland more than anything else. ''It's a Scottish problem, not just a local problem, actually,'' he says. ''I was persuaded by various people that it was too big an issue to drop now, it was bigger than me, much bigger than me and my interests. And I've got the means to do it. I will not stop. My intentions are so far beyond self-interest now.

''I think Historic Scotland certainly have got an arrogance and power and they are used to yielding it without question, and I think there is a grudging admiration for myself that I've stayed the course,'' he adds. ''I think they would like me to walk away but I think they've realised now it's not going to happen. I've invested so much time, money and effort and heart and soul into this project that it isn't going to happen. End of story. And they don't know how to get themselves out of this mess.''

For now, though, the door to Castle Tioram remains locked. There are no workmen nor visitors. For how long is anybody's guess. n