But some time in the early 1960s a white South African named Cecil Williams embarked on a journey that took him from one such prison to the unlikely destination of Anniesland.
Newly released from a jail sentence he had served for opposing his country's racist regime, Williams, like many political outcasts, was now an exile, a stranger in a foreign land, and he sought out friends among locals who shared his ideals.
Those he befriended included a Glaswegian couple called Phil and Cathy Filling. Both were activists in the city's burgeoning Labour and communist movement, and listened along with others as Williams told of the terrible conditions under which so many black people in South Africa lived.
The exiled man also described the dangers faced by his comrades in the African National Congress (ANC). Williams spoke of one resistance leader in particular, who headed the movement's armed wing, known as Umkhonto We Sizwe or Spear Of The Nation. He had been nicknamed the "black pimpernel" because of his ability to evade the apartheid security services and was high on the apartheid regime's wanted list.
Williams told the Fillings how he had worked as the pimpernel's driver, but that the pair would sometimes have to swap roles because a white man spotted driving a black man around would have been sure to attract attention.
The pimpernel's real name was Nelson Mandela.
Mandela lived on the run for 17 months until, in 1962, he was tracked down and arrested by the authorities. Appearing in the dock alongside other prominent ANC activists in the now infamous Rivonia trials of 1964, he was found guilty of the capital crime of sabotage, which was the equivalent of treason.
For most of the next 27 years, Nelson Mandela was incarcerated in a cell on the notorious Robben Island. In that time he became the world's most famous political prisoner, and the living embodiment of the sacrifices made to rid South Africa of apartheid.
Not long after his release in 1990 Mandela, by then a household name throughout the world, went on an extraordinary pilgrimage. Just like his white comrade Williams who, all those years before, had found himself turning up in Scotland, Mandela was now bound for Glasgow.
In part, Mandela's visit was a gesture of thanks to the Glaswegian people for honouring him with the freedom of the city years earlier while he still languished in jail. But in many ways it was much more than that. For the story of Nelson Mandela's special relationship with Scotland is also a tribute to the role the nation's anti-apartheid campaigners played in contributing to the demise of such a repugnant regime. For that reason it is a story of which Scotland and Glasgow can be justly proud, not least this week, when Glasgow joins the world in mourning the death of its most famous freeman.
Few who met or saw Mandela in Glasgow over the course of a few wet and windy days in October 1993 were immune to his spell. In a scene that was to be repeated across the city, he first arrived at Glasgow's Hilton Hotel from the airport, bringing the place to a virtual standstill. Waiters and bar staff stopped serving, while guests rose from their tables in dining rooms to come to the foyer and give him a standing ovation.
The following day it continued, as a carnival atmosphere gripped George Square where tens of thousand gathered to see him despite a deluge of rain that was torrential, even by Scottish standards.
"Whilst we were physically denied our freedom in the country of our birth, a city, 6000 miles away, and as renowned as Glasgow, refused to accept the legitimacy of the apartheid system and declared us to be free," Mandela reminded those who listened, enraptured, to his speech. "You, the people of Glasgow, pledged that you would not relax until I was free to receive this honour in person. I am deeply grateful to you and the anti-apartheid movement in Scotland for all your efforts to this end," he intoned.
Moments later, to the delight of the crowd, the "black pimpernel" was shimmying to a song by the South African singer Mara Louw, who had joined him onstage in George Square. Perhaps this was the man's greatest gift: a human touch that could make kings and commoners alike feel uplifted in his presence.
For years leading up to that special moment, I too had been active in the anti-apartheid movement, working as a freelance journalist and photographer covering many of its demonstrations, rallies, and events that were tirelessly pulled together by devoted campaigners. But few I met were more committed than Brian Filling, the son of the Anniesland couple who had befriended Mandela's driver and comrade in arms, Cecil Williams, all those years before.
I first met Brian Filling in the late 1970s in Glasgow's Star Folk Club, which in those days met at the old Communist Party premises in Calton Place on the banks of the River Clyde. It was a regular watering hole and music venue frequented by trades unionists and left-wing activists.
"My mother was in the Communist Party and my father chairman of the local Labour Party constituency and they were very much internationalists," Filling recalls.
Soon he, too, was active in the Communist Party. As a student in Glasgow University's debating society, Filling made his maiden speech against the then Labour government sending arms to South Africa, as the world still reeled from the images of black people being gunned down in the Sharpeville Massacre of 1960.
In 1970, Filling was arrested at Murrayfield Stadium in Edinburgh during a demonstration as part of the Stop The 70 Tour to disrupt the all-white Springbok rugby visit.
From there Filling went to London where he edited the Young Communist League (YCL) newspaper, and began to meet important contacts among exiled ANC activists.
"Among then was Essop Pahad, later to become South Africa's minister in the presidency and Aziz Pahad, who went on to hold office as deputy minister for foreign affairs," remembers Filling of time spent with these influential ANC men who were to help him in the creation of a formal anti-apartheid movement in Scotland.
Indeed in the years to come, it was Essop Pahad who would suggest the idea of a campaign to rename what was then St George's Place in the centre of Glasgow, and home to the South African consulate, as Nelson Mandela Place.
By 1974, Filling was back on his home turf where he found a disparate collection of political pressure groups challenging apartheid.
Around this time, he also met another key activist and campaigner in Scotland called John Nelson. "John was involved in a group opposing apartheid that met in a Quaker house in Wilton Street in the west end," says Filling, describing what in essence was the kernel of the official anti-apartheid movement in Scotland.
This was 1976, the time of the Soweto uprising. Conviction was mounting both inside South Africa and around the globe that pressure must be intensified on Pretoria.
In the years ahead I would often join Filling and our fellow campaigners outside the South African consulate in what had become a weekly picket of their premises in Glasgow and London. It was there, too, on Friday night that old comrades and many members of the public came together one more time to honour the life and work of Nelson Mandela.
Back in those activist days of the 1970s and 80s it was not uncommon to see dozens of people, young and old, on the streets of St George's Place, placards in hand, or selling copies of anti-apartheid news, harassing any officials coming and going from the consulate offices. In my capacity as a reporter covering these activities for the movement's newspaper, I often had the chance to interview, or at least meet, some of the major figures in the battle against apartheid.
Until my encounter with Mandela himself, however, perhaps the most memorable individual I met was his fellow ANC activist Denis Goldberg, the only white member of Umkhonto We Sizwe to be arrested and sentenced in the Rivonia trial to four terms of life imprisonment.
It was a sentence which, as a white, he was to endure in isolation from his black comrades.
When we met in a bar in Glasgow's Sauchiehall Street, rather ironically named The Colonial, Goldberg had just been released after serving 22 years in a Pretoria prison almost completely cut off from the outside world.
Rather nervously I asked him whether he felt he could ever fully recover from his prison experience. Goldberg looked at me. "Oh yes, of course," he said, before adding matter of factly, "but sometimes I just have to get up and go outside into the open air to remind myself I can now do what I want."
Rarely before or since have I been struck so profoundly by the modesty and courage of another human being.
It was only much more recently, however, as Mandela's condition worsened and he began slowly to slip away, that Dennis confirmed what I had heard previously of the crucial role played by Scottish activists within a covert movement that was part of the anti-apartheid struggle.
Certainly, Scots had thrown themselves behind the anti-apartheid cause on a mainstream level of boycotts, pickets, rallies, concerts and calls for sanctions. Much less widely known, however, was the role some played within a clandestine or underground movement.
Ordinary Scots acted as couriers, procured documents, established safe houses in Glasgow for those ANC activists in exile; some were even involved with arms smuggling to the ANC in South Africa.
So pivotal were these Scots during this period that agents of the apartheid regime's secret service, the South African Bureau for State Security (BOSS) were deployed in Glasgow and elsewhere, monitoring and tracking activists.
"These were Scots, quite heroic young people who went in and out of South Africa delivering documentation, delivering money ... they played a role which showed the essential humanism of human beings who are going to help others," recalled Goldberg, when we met in Glasgow.
It was around this time too that Brian Filling was becoming increasingly aware that singling out Mandela to highlight the plight of all South Africa's political prisoners was becoming official ANC policy.
In Glasgow, the campaign was given added impetus when, in 1979, the then-Lord Provost of Glasgow, David Hodge, invited the South African ambassador to the City Chambers.
The event angered many people and generated even more publicity for anti-apartheid activities. The Labour whip was withdrawn from Hodge and, by the time Michael Kelly became provost in 1980, Hodge was virtually ostracised. Kelly was then able to secure Mandela's freedom of the city in 1981 almost by way of compensation for the anger and ill-feeling the Hodge controversy had generated.
Despite the groundswell of support for Mandela in Glasgow, it was not until 1993 that the man himself finally came to receive the honours bestowed on him by the city fathers.
Mandela's visit to Glasgow was fleeting, yet everyone lucky enough to have met him during those three days cherish their own memories of the encounter. For me, what stood out was an intimate insight into his character gained during a late-night function at the Hilton Hotel, where he met businessmen who, like Margaret Thatcher, had been sceptical or downright antagonistic towards the ANC and its calls for implementing economic and other sanctions against the apartheid government.
Throughout the evening, Mandela never faltered in his efforts to win over the critics among the guests, but occasionally would excuse himself to go off and speak with one of his aides. It was only after a number of such consultations that I realised these discussions had nothing to do with politics.
Rather, Mandela was getting updates on a televised world-title boxing match showing in the other room, in which one of his South African countrymen was fighting. There I witnessed that relaxed, natural lightness of touch that most politicians seem to lack. Mandela had an innate understanding of a world beyond politics, a world inhabited by ordinary people for whom such things as sport sometimes matter almost as much.
Brian Filling, by then chairperson of the Scottish Anti-Apartheid Movement, accompanied Mandela for most of his Glasgow visit. For him, it was the old campaigner's self-discipline and sense of organisation that created the most lasting impression. "One minute he might be talking and then he would be brought an urgent message to read, and he would say, 'Excuse me, I need to do this now'," says Filling.
"Some people who are important, and believe they are talking to someone less so, seem to lose eye contact as they look around for other important people worthy of their attention. With Mandela that eye contact is never lost and you are the centre of his attention."
Shortly before the ANC leader's departure from Glasgow, Filling asked him to sign a copy of a book on Scottish-South African writing he had edited. Mandela agreed, penning: "To one of South Africa's dependable friends" inside the fly sheet, before asking for a copy of the book and announcing that one day he'd like to return to Scotland, but without the media attention. Asked why he wanted to come back, Mandela simply replied that he would like to go salmon fishing.
So dependable a friend was Brian Filling to South Africa's fight against apartheid that he was a guest at the inauguration of Nelson Mandela as president in Pretoria in 1994 and was subsequently awarded the National Order of the Companions of OR Tambo in 2012 for his outstanding solidarity work. This is the highest honour which can be bestowed upon any non-South African by the Republic of South Africa.
Today, as the world salutes Nelson Mandela, it is perhaps worth taking a moment to consider the significance of Scotland's place in the battle against apartheid and help in securing Mandela's freedom. It is also worth pausing to take note of what that participation means in terms of our own national identity.
"One of the things it almost certainly contributed to was Scotland winning the bid to host the next Commonwealth Games," says Filling. He believes there is a feeling among Scotland's Games bidders that Glasgow's honouring of Mandela all those years earlier paid off in winning over African nations to the idea of the city as a deserving host. Filling also believes that the Mandela connection confirmed Glasgow's great tradition of political internationalism.
Perhaps Scotland's greatest reward for allying itself with Mandela's battle against apartheid is the collective sense of national decency such a positive role engendered. In opposing apartheid, it revealed itself as a nation capable of displaying the qualities needed in laying claim to be a tolerant, caring and multicultural society.
With the sad news of Nelson Mandela's death, Scotland can at least console itself with the knowledge that we stood alongside him when it mattered.
In looking forward as nations both Scotland and South Africa could also do worse than heed the words of Mandela himself: "I have walked that long road to freedom. I have tried not to falter … but I can rest only for a moment, for with freedom come responsibilities, and I dare not linger, for my long walk is not ended."