The coldest day of the year was not perhaps the best time to be wandering around the Canongate Kirk's melancholy graveyard.

Nevertheless, that's where I found myself, on a morning so frozen that it seemed the air might shatter and sprinkle ice on the path.

The grave that drew me was that of the poet Robert Fergusson, whose spritely statue graces the entrance to the Kirk. Fergusson's memorial keeps company with soldiers, surgeons, candlemakers and brewers, and, on the far side of the Kirk, the unfortunate David Riccio, or so some say. The young poet was no less unlucky than the murdered courtier, though it was insanity, possibly brought on by syphilis, that killed one of the most influential literary figures who ever walked the Old Town. If, like me, you are of a fanciful turn of mind, then on a heartless January morning you can still sense him languishing in the city's asylum, not far from the Kirk.

Though he died in abject poverty, and had a pauper's burial, Fergusson's grave is stately, fenced by iron railings, and planted with rose bushes. A toppled jar of red roses brought a splash of colour the day I visited. It was Robert Burns who probably paid for the headstone, and wrote its tribute, though it was not erected until after his death.The inscription, in a stonemason's flowery hand, reads: "Here lies Robert Fergusson Poet, Born September 5th 1751 [in fact, he was born in 1750], Died October 16th, 1774. No scupturd Marble here nor pompous lay/No storied Urn nor animated Bust/This simple Stone directs Pale Scotia's way/To pour her Sorrows o'er her Poets Dust."

Whoever left the red roses is, perhaps intentionally, also paying tribute to Burns, whose work would have been less rich and vigorous without the example Fergusson set. As Robert Crawford writes in The Bard, his biography of Burns, when the younger aspiring poet first read Fergusson - Burns was 15 when he died - he discovered a soul mate. Reading Fergusson, writes Crawford, brought on a "step-change in his own verse". Though they were from entirely different backgrounds - Fergusson was highly educated and well connected - they shared a similar outlook on life, and a passion for the verve and melancholy of Scots that makes their poetry memorable. They were also drawn to the stews and hotspots of a teeming city, a predilection that made it almost certain they would find themselves in trouble.

Copying Fergusson, Burns took to calling himself a 'Bardie'. He identified closely with the poet he called "my elder brother in Misfortune,/By far my elder brother in the Muse", and was drawn to Fergusson's miserable end, a fate he perhaps feared for himself.

Leaving the chill Canongate for the warmth of the Music Department of Edinburgh Central Library, I found further evidence of the place Fergusson held in Burns's heart. In a display case is Burns's copy of Fergusson's poems. On the fly-leaf he has written his own version of a passage from Jeremiah, which begins: "Oh Woe is me, my Mother dear! A man of strife ye've born me; For sair contention I maun bear, They hate revile and Scorn me..."

This echoes a similar exercise by Fergusson, who paraphrased a chapter of Job. The two were thus inspired by miserable passages of the Bible, morbidly in thrall to the dark side and penury of the versifier's life. At around the time he wrote his jeremiad, Burns was struggling to cope with rejection by Jean Armour. It was only the hope of making it as a poet, he said, that prevented him from killing himself. Thankfully, he did not, but in his emotionally overwrought state, one catches a glimpse of the despair he, like Fergusson, suffered. Both, however, were equally able to be light-hearted and humourful, and it is their unquenchable spirit, as well as their profundity, that endeared them to readers generations after they were gone.

Without his early influence, who knows how long it might have taken for Burns to find his voice. On the eve of the Bardie's birthday, Fergusson deserves a toast. And a fresh bunch of roses.