I was touring southern England recently, and being an aficionado of the weird, my eye was caught by a recurring theme. Every gift shop I visited was filled with supernatural paraphernalia.

I was expecting Union Jack teddy bears and tat emblazoned with "Dad went to Dorset and all I got was this lousy t-shirt", instead there were ouija boards, tarot cards, and shelves groaning with books about local ghosts and creepy legends.

How apt, I thought. Souvenir shops are culture’s bottom rung. Tourist trinkets may seem meaningless but the gifts we offer foreign visitors reflect who we are as a society, in the most crude way obviously. The symbols of memorabilia make a point, if we care to look. When you buy that Guinness key-ring in Dublin, you’re being sold a metaphor of a nation which knows its reputation for throwing a party, right?

Read more: Sinister Scotland is the natural home of horror

Then there’s the "Haunted Tourism" trend: trips to ghostly castles and sites of paranormal activity.

This all fits into a cultural jigsaw that’s taking shape. We’re a society obsessed with the occult. Unsurprisingly, ghost sightings are rising.

Let’s move off culture’s lowest rung, and look at mass entertainment. Two big commercial and critical hits of late have been the sitcom Ghosts, and the podcast Uncanny, now a TV series.

Both are marvellous. Ghosts is probably the most elegant sitcom of its generation. I’ll be lynched for this in Ireland, where I’m originally from, but I think Ghosts beats Derry Girls for the best sitcom of the 21st century. It was a classic from the first episode, set in a haunted manor where Stone Age, Tudor, Regency and Victorian spooks hang out with the shades of an executed witch, a dead Second World War officer and a sex-shame Tory MP. It makes you laugh out loud, and can put a sentimental tear in your eye. A minor masterpiece.

Then we’ve Uncanny. It began life as a BBC podcast exploring "real-life" hauntings. I listen to it on twilight walks in the park. It has me glancing over my shoulder and quickening my steps homeward before sundown. It’s so successful, it’s just transferred to TV.

These are just two biggies in a firmament of shows, films, and even computer games obsessing on the supernatural. Culture always reflects society’s concerns: our hopes, fears, and dreams.

At the higher end of culture, the supernatural is increasingly at the heart of art. There was the recent Horror Show! exhibition in London, where artists and filmmakers explicitly asked the question: why is there so much dread in the world?

We need to lend a thought to history when it comes to this supernatural trend, as we’ve been here before. Between the two world wars, Britain was gripped with ghost fever. This was the period when the most hauntings were reported. Cultural historians put that down to a number of factors which resonate today.

Between 1918 and 1939, our ancestors lived in the long shadow of conflict and death. After the trenches, Spanish Flu shredded mortality. War and disease: sound familiar? When times get frightening, minds turn to frightening themes.

Read more: Why horror is good for you

It’s that old notion of catharsis: nothing makes you feel better than a good scare. The Ancient Greeks did it with tragedy. When was the heyday of the horror movie? The early 1930s when the likes of Dracula, Frankenstein and the Wolfman gripped the imagination of the west.

New technology and new scientific theories are also seen as contributing to this interwar spook boom. Radio, TV and the telephone changed how our ancestors viewed the world. Einstein opened up the universe to dizzying possibilities, and Freud did the same with our inner world. Science and technology put "reality" into flux.

Right now, we’re still coming to terms with the internet. Neuroscience, quantum physics and theories like the multiverse, with its endless array of universes, make us question the very fabric of our existence.

Life is grim, too. The Roaring Twenties only roared for a few, most had it tough. There were soup kitchens and hunger marches. Food banks didn’t exist in Britain until around 2010. Hard times make people look for unusual sources of comfort or explanation.

Perhaps that’s why so many young people are turning to paganism. New Age faiths are rapidly growing in numbers year on year. Maybe the pandemic prompted the shift, with more folk seeking solace in nature. Paganism quite literally deifies the natural world, after all.

Nor does Paganism have the divisiveness of organised religion, yet it still offers some sense of spirituality in an increasingly atomised world where "God is Dead", as Nietzsche said.

Anthropologists have noted that urbanisation leads to an increase in claims of hauntings, and speculate that this is because in our city-dominated lives we’ve abandoned that sense of extended family, and so feel greater guilt at the loss of loved ones. Guilt may trigger claims of ghost sightings.

Now let’s look at the theory. It’s called "Hauntology". Cultural theorists propose that humans are obsessed with ghosts as they’re symbols of a society’s unresolved past traumas.

Read more: Dracula's undying legacy

Ghosts aren’t real; I certainly don’t believe in the supernatural, although it fascinates me from a psychological-sociological perspective. Rather the cultural theory of Hauntology suggests that ghosts are society’s undigested problems, repeating on us: a visual, subconscious metaphor for a society looking back on its past, trying to work out what went wrong; ghosts represent a society trapped in nostalgia’s deadly embrace.

I hate to get political, but surely Brexit is the perfect example of a society looking backward? Think also of the specific periods which so many of our ghosts seem to come from: Jacobite and English Civil War battlefields dominate. Hauntologists would say that these two bloody, fratricidal rebellions represent unresolved national traumas which play out in our cultural imagination as apparitions.

And clearly, once some folk in the years after Culloden start seeing battlefield ghosts - perhaps literally "haunted" by the memory of what happened there - an infection of sorts begins. One sighting leads to another, and centuries later folk are still seeing spectral clansmen.

Here’s the most chilling notion: put all this together and the answer seems to be that ghosts are us, our regrets, our fears, our guilt.