Approaching Berwick-upon-Tweed, my emotions tend to fluctuate depending on the direction of travel.

Heading south, I get a sinking feeling when the train leaves the station and inches over Robert Stephenson's great bridge. Though it does not mark the border between Scotland and England it has always been for me the point of no return. I first crossed it when I was 18 and was heading, like so many Scots, for London. Even now, some 40 and more years hence, I remember the pang of regret I felt, wondering when, if ever, I'd get back.

But that was mingled with a sense of excitement and escape that for the moment at least I had, with the arrogance of youth, sucked my homeland dry. I needed new experiences, fresh perspectives. Leaving Berwick, which I then thought was in Scotland (such was my ignorance of history), I was in England, foreign terrain, where things were done differently, where folk called you Jock and said "Och aye!" in response to an enquiry.

Travelling north induces an altogether different sensation. Then, as Norman MacCaig once wrote, I can feel Scotland rushing towards me, like a burn in spate. Durham, Newcastle, Alnmouth: they all come and go in a blur. I long to see the coast and a solitary fishing boat ploughing through the North Sea because when I do I know I am nearly home. Then, that bridge over the Tweed - a conduit to something indefinable - is a link between past and present, between belonging and not belonging, between where I was brought up and who I am now. Once over it, I breathe more easily. Were it necessary I could do the rest of the journey on foot.

On this occasion, however, I arrive in Berwick from the north. It is quite early on a Monday morning and the town feels newly awake and not entirely conscious. The idea is to start my travels here in a place that is geographically in England but which draws its population of 13,400 from a hinterland that includes Scotland. It is cold but dry, for which everyone I meet counts their blessings. Elsewhere, as every front page attests, flooding is widespread and rain is unrelenting, causing havoc and misery in equal measure.

My first impressions are not good. Like so many towns Berwick is blighted by empty shops, which gives its centre a hangdog air. And more closures are in the offing. Even more worrying, though, is the demeanour of the people. They have a look of defeat about them, as they queue at Greggs or rummage for clothes in charity shops. I eavesdrop on two men who at first seem to be discussing the suicide of friend. "He was only 46," says one, "why would he kill himself?" It transpires they are talking about the actor Philip Seymour Hoffman.

"Are you Scots or English?" It is the question asked of everyone from Berwick at some juncture. To which the usual answer is: "Half and half." That it suffers from schizophrenia is hardly surprising. In times past, it changed hands frequently until, in 1482, it was finally recognised as English. But it wears its Englishness with ambiguity. The football team, Berwick Rangers, plays in the Scottish league and its rugby teams either play against Scottish or English opposition. Moreover, it has two newspapers, the Berwickshire News and the Berwick Advertiser, the former, explains their editor Phil Johnson, geared towards a Scottish readership, the latter an English one.

It's a unique position to be in, and Johnson has to be careful to observe local nuances. Reprobate Scots, for example, are dealt with at Duns Sheriff Court while their English counterparts come under jurisdiction at Berwick. Should independence become a reality that won't change. Indeed, little will, reckons Johnson, who lives in Coldstream, in Scotland, with his wife and two-year-old twin girls. "People," he says, "are more concerned about where decisions are made in Northumberland County Council." At present, the council is headquartered in Morpeth, some 50 miles to the south.

That Berwick is not getting the best of deals is a sentiment shared by Isabel Hunter, the town mayor. It is largely an honorific position and one which Hunter, who runs a haulage business with her son, combines with that of LibDem councillor. Detecting a sense of "negativity" among some folk, she is also aware that Berwick is in something of a Catch-22 situation, especially when it comes to funding, be it the development of the football stadium, widening broadband access or the dualling of the A1. Would Berwick do better if it were to throw its lot in with Scotland? Hunter is uncertain. As it is, she says, "We don't see that border." She adds jokily that if Scotland and England were separate a couple of her drivers who live in Scotland would need to acquire passports to get to work. Back in the day, she reminds me, there was a passport office in Berwick, which could be reinstated.

There is a bus leaving for Melrose just after three, going via Norham, Cornhill, Coldstream, Kelso, St Boswell's and Melrose, where I've elected to stay for the night. I am looking forward to a gentle excursion through the countryside, over which generations of Borderers engaged in tribal warfare not too dissimilar to that practised by Sicilian families. Passengers are few: mainly pensioners taking advantage of their free passes. Hereabouts the land is flat and fertile but rather featureless, necessitating an injection of imagination to see it as other than benign. The driver drives like a barman shaking a cocktail. On the outskirts of Duns we draw into Berwickshire High School, a modern, characterless building quite out of keeping with its surroundings. Teenagers pour on, two of whom sit behind me. One had been given a 15-minute detention, unfairly by his lights. "So what are you going to do now?" asks his companion, airily. "Go home, pour yourself a glass of red wine and sink into a hot bath?"

At Kelso, there is a short stop to allow the timetable to be met and a man in a bunnet bemoans the inconsistency of the buses' arrivals and departures. It is getting near five when we pull into Melrose. Like other Border towns it closes early and abruptly. I enter the wine shop-cum-cafe in the square and order a coffee but am informed that it has stopped serving. I stroll down the empty High Street. Everywhere is either bolted or making preparations to that end. The light is fading and as I pass the abbey you can feel history come alive. It is raining hard and the Tweed, spied from the old iron suspension bridge which connects Gattonside, where I am staying, and Melrose, looks swollen and threatening to encroach on its banks.

Later, I am driven back into Melrose by my landlady, Sheila Robson, chatelaine of Fauhope, a superior B&B. Swapping roles, she wants to know what I think about the independence referendum. Keen to assert my impartiality, I find a fence and sit on it. Robson is rather flustered by the thought of it all, though she does not say whether she is for or against it, only that her brothers' votes would leave the status quo intact, one being pro, the other against. I had hoped to find out what other locals thought but Burt's Hotel is virtually deserted, February being, in its owner's view, "a doldrum month". Melrose, he says, springs into life a couple of times a year, during the rugby sevens and the Borders Book Festival, both bringing much needed visitors. Otherwise, it is dependent for its obvious prosperity on wealthy retirees who dine before eight and head home before they're tempted to indulge in bacchanalia. The bar is tended by an Australian woman who arrived only a few weeks previously and will soon be moving on. Apart from me, there are two other customers, a husband and wife from England and their dog, about which they talk as if it were an astrophysicist. They've spent part of the day in Hawick where they managed not to spend £49 on a dog scratcher, the kind of information which leaves one bereft of comment.

The next morning I seek out Christopher Harvie, an academic and former SNP MSP. Harvie lives with his aged father on the edge of Melrose, near the Borders General Hospital and 15 minutes by foot to Tweedbank, the southern terminus of the Waverley line when it reopens next year. He is in the garden when I call, cutting logs with a crocodile-toothed saw and dressed, as ever, in tweed plus fours and bushwacker hat. He has long been in favour of the Waverley line, believing it will bring the Borders into closer contact with the rest of Scotland, Edinburgh in particular. Peppering his conversation with words which would require the employment of asterisks were one to repeat them, he is unstinting in his contempt for those landed and moneyed interests in the neighbourhood who've fought like the reivers of yore to thwart the reintroduction of trains after they were decommissioned by Beeching in the sixties.

This year, Harvie reminds me, also marks the bicentenary of Sir Walter Scott's novel, Waverley. Does Scott, whose house Abbotsford is nearby, still matter? "I think he has enormous potency," says Harvie, who, one senses, has more affinity with the writer's idea of the Borders than he does with them in actuality. Things like rugby and the Common Ridings, he reckons, are "really private affairs", and for those who like such things that is the kind of thing they like. In a way, though, they help mask the gravity of the current situation, which is that of depopulation, unemployment and inaccessibility. As the disappearance of shipbuilding has impacted on the west of Scotland so, too, has the demise of the textile industry affected the Borders. When I was a boy, travelling to Innerleithen to visit relatives, you couldn't see the grass for sheep. The valleys were loud with baa-ing and towns buzzed with an activity and energy that's difficult to replace by attracting ramblers and mountain bikers.

Galashiels was once the most productive of the mill towns and as distinctive from Hawick, its rival, as Edinburgh is from Glasgow. Today it's the hub through which all buses pass. I spend an hour there and feel I have exhausted it. I kill time by studying the notices for clubs and societies posted on shop windows. Were one inclined one could join the Scottish Borders Ukulele Club ("Have you been banished to a room to sit alone and fume? It could be worse, you could be learning the spoons"), have a dog micro-chipped for free or take part in the Firewalk Challenge ("40 fearless fundraisers will walk 20 feet over embers burning at 12,000 degrees Fahrenheit …").

I take the next bus to Selkirk, where there is a statue to Scott and in whose Forest Hotel - long gone - Burns once stayed. On the memorial to the battle of Flodden three words are inscribed: "O Flodden Field." There is a bookshop, The Forest, which Allan and Anne Harkness opened in 2006 as, in Allan's words, "a cultural catalyst of sorts", and in defiance of Amazon and co. An idealist and a nationalist, Harkness's vision of the Borders is for "more creative involvement in making a good society", his splendid shop less of a business and more of a cause.

As its inhabitants never cease to remind us, a day spent outside of Hawick is a day wasted. Currently, the town calls itself the "Home of Cashmere". I stumble across the Borders Textile Towerhouse which tells "the knitwear and tweed story", which is a lot more compelling than it may sound. "There was a time," the writer Allan Massie says when I bump into him in Selkirk, "when Hawick was the best-dressed town in the UK." I don't doubt it. That was the era when the height of fashion for women was the twinset, especially when worn by the likes of Marilyn Monroe, Grace Kelly and Audrey Hepburn. The equivalent for men was the V-necked sweater, beloved of golfers and film stars such as Bob Hope and Bing Crosby.

In one of the Towerhouse's rooms I appear to have happened on a demonstration of weaving but closer investigation reveals it is a first-aid class in CPR and that the body on the floor is a dummy. There is nowhere in Hawick's main street where you can buy premium knitwear. There are, however, plenty of places for sale at knockdown prices, a one-bedroom flat - albeit in need of a sledgehammer - in the town centre on the market at around £22,000. Or, if you so desire, a pub with a three-bedroom apartment could be yours for as little as £100,000.

Enticing as such offers may be, I have a bus to catch to Langholm, where this particular leg of my travels is scheduled to end. I am one of two passengers, the other being an elderly woman who appears to be wearing a red bell tent and who ignores all attempts to engage in small talk.

"Neil Armstrong of the local clan," Chris Harvie has said, "was the first Langholm man on the Moon." I know it better as the birthplace of Christopher Grieve, also known as Hugh MacDiarmid, a monument to whom overlooks the "muckle toon". Grieve and Langholm were not, it's perhaps fair to say, simpatico. Langholm, situated 20 miles from Carlisle and 70 from Edinburgh, feels out on a limb. Bureaucratically, it belongs not to the Borders but to Dumfries and Galloway, which displeases some of its inhabitants. Here the divisive issues are the mooted fracking at Canonbie, six miles to the south, and wind farms, not the forthcoming referendum. "Fifty years ago," a local man recalls as we stand together waiting at a bus stop, "the last train stopped in Langholm. What more can I say?" n

Next month: Alan Taylor visits Edinburgh.