The north-east of the tiny Pacific archipelago reminded the explorer of Scotland, although it was the French who eventually colonised the islands in 1853, turning them into a penal colony. Later this year, though, Nouvelle-Calédonie, just like Old Caledonia, will begin the process that determines whether it will retain its ties with the French Republic or become an independent nation.
In May, provincial elections will select a new 54-member Territorial Congress which will then have to decide - by a three-fifths majority - if it wants to organise a referendum on independence between now and 2018. If that doesn't happen, then France will initiate its own ballot in four years' time. This will mark the final phase of the "Noumea Accord", signed in 1998 between pro and anti-independence campaigners and the Élysée Palace - New Caledonia's version of the Edinburgh Agreement, which paved the way for this year's referendum.
The Accord has gradually devolved autonomy to the island under the watchful eye of the United Nations, which in 1986 added New Caledonia to its list of overseas territories to be decolonised.
But the process is not an unambiguous choice between splendid isolation and continued French control, producing striking parallels with Scotland, and between independence debates in two very different parts of the world.
New Caledonia's shiny new airport was likely built with French money: Paris sends €1.6 billion or £1.32 billion to the island each year. Driving from it to the capital Noumea it is easy to see why the dramatic mountainous landscape reminded Captain Cook of Scotland.
But unlike other French "overseas departments", for example Guadeloupe and French Guiana, New Caledonia is not considered part of the European Union, despite its inhabitants being European citizens and therefore voting in May's elections to the European Parliament. But if it opts for independence in a few years' time, those ties with Brussels will be severed; unlike Scotland, "independence in Europe" isn't really an option.
Andy, a transport consultant of French Polynesian descent, says he has an "open mind" but is inclined to vote No in any referendum - that's no to continued links with France. "The local government does an okay job of running things," he says, "so why not?"
But Andy also wants to keep his EU passport - he likes the ease with which he can travel in Europe - and is also concerned about currency as the Pacific Franc is linked to the euro. If there was a way of retaining both these things and still being independent, then he'd be an enthusiastic proponent of full autonomy.
Sound familiar? New Caledonia's Kanak or indigenous leaders, like the SNP, long ago reconciled themselves to the realities of independence in a globalised world: thus they envisage the islands retaining France's co-operation when it comes to defence, justice and policing even after independence. As Jean-Marie Tjibaou, a Kanak leader assassinated in 1989 put it: "Independence is the capacity of deciding on your own your interdependences."
Or, as a local métro or French-born migrant put commented: "I don't believe that one day we will cut all ties with France, for historical and linguistic reasons. In my opinion, New Caledonia will become a kind of 'associated state' to France, a little bit like the Cook Islands is to New Zealand."
Perhaps unsurprisingly, there is a divide between indigenous New Caledonian Kanaks, who make up two-fifths of the local population and of whom around 90% of support independence, and Caldoches, those who have ancestral ties to the early French settlers, who are generally against it.
Support for full autonomy sits at around 35%, with two-thirds opposed. Only those who have been resident in New Caledonia since the Noumea Accord was agreed in 1998 will be able to vote in the referendum, a once-controversial qualification that is now generally accepted. Obviously this disenfranchises anyone who has settled on the island in the past 15 years, mainly ex-pat French citizens.
Many of those migrants are very wealthy, as evidenced by the luxury yachts moored in Noumea's beautifully landscaped bays. New Caledonia raises about as much in taxes as it receives annually from France, and together with grants from the European Union this makes it one of the Pacific region's largest economies.
As a Noumea-based journalist said with some understatement, it would be "difficult" to plug this gap if the island becomes independent.
That said, the economic argument, as in Scotland, cuts both ways: those in favour of independence claim the island's wealth as the world's fourth-largest producer of nickel demonstrates New Caledonia could easily stand on its own two feet, while those opposed hint that independence would jeopardise the wealth currently enjoyed by a majority of islanders.
But inequality is also a major talking point in the debate, with stark gaps between the life chances of Kanaks and the European population.
Whatever happens by, at the latest, 2018, New Caledonia has steadily been accumulating the trappings of nationhood, having chosen a national anthem, motto and new design for its banknotes over the past decade-and-a-half.
In July 2010, it became one of the world's only countries or territories to possess two official flags (the Kanak emblem and the French tricolour); the Noumea Accord even allows for the island to adopt a completely new name.
Meanwhile, France, keen to make a better fist of this independence process than it did in, say, Algeria, has been - in the words of a local journalist - "watchful and co-operative".
"It is up to Caledonians," she says, "and France will never go against the choice of the population here." Paris realises the situation could become violent, as it did in the 1980s when fighting broke out between French settlers and the Kanaks, leading to dozens of casualties.
Today that outbreak of violence remains a distant yet traumatic memory. New Caledonia feels remarkably French, with Noumea bringing to mind Nice or Cannes on the Mediterranean coast.
But beneath the surface, perhaps this sun-drenched corner of the Fifth Republic is getting ready to follow many of its Pacific neighbours in charting its own post-colonial path.