Hayes, whose beautifully minimalist, trance-like approach to Irish music has earned him a worldwide following, was at the time a young lad learning his instrument and Potts was one of many musicians who used to call in to have a tune with his father, a founder member of the revered Tulla Ceili Band.
"I would always get asked to play a tune for these visitors so they could see what they thought," says Hayes. "And I got quite a lot of what they thought, I can tell you. Not in a nasty way; it was all good humoured. But I remember playing a tune for Tommy Potts and you know how in traditional music there's this thing that says, 'This is how it is?'
"Well, Tommy Potts' view on that was "not necessarily" and he proceeded to show me variation after variation and I was spellbound. He was a solo fiddler, rarely played accompanied, and he was all about making new pieces out of old pieces, and I have taken that idea on myself, I suppose, not in exactly the same way but I think there is a world of possibility in this music."
For Hayes, music should live in the moment and it has been one of the great thrills in listening to his partnership with the Chicago-born guitarist Dennis Cahill over the past 20 or so years that you never hear the same gig twice. They might play the same tunes night after night, but in Hayes' musical language it is not about the notes you play, it is the feeling you put into them and the feeling can change by degrees every time. Although they have played in other formations - and will be working with English folk band The Unthanks shortly - Hayes and Cahill together make such a complete sound that it is difficult to imagine what they might want to add to it.
Enter The Gloaming's three other components: sean-nos singer Iarla O Lionaird from the Afro Celts; New York based pianist Thomas Bartlett, who numbers David Byrne and Yoko Ono as clients; and the man with possibly the least enviable job, Hayes' fiddle partner Caoimhim O Raghallaigh, whose philosophy, like Hayes', is about under- rather than over-playing.
Indeed, Hayes regards all five as such reflective players that he had a horrible thought as they prepared for their first gig together, at the National Concert Hall in Dublin, no less.
"I had no idea what we were going to sound like together," he says, "and to begin with, we did not have a name, we did not have an album, we had not played a note together. But we had a tour and the first night of that tour was at this fairly prestigious venue. I remember thinking, 'I hope this turns into something and it doesn't become one embarrassing night'. We all like fairly ponderous music but I hoped it had a bit of balls, and fortunately it did."
Behind this worry he must have trusted his own judgement, however, as he felt the time was right to go from a solo - or duo - career into a band.
All the participants in the Gloaming have been known to Hayes for many years, since they were both young lads at music festivals in O Lionaird's case. Hayes and Cahill go back to around the late 1980s. O Raghallaigh was a Hayes hero-worshipper who befriended his hero as a teenager and the best story belongs to Vermont-born Bartlett, who became besotted with Hayes's first album as a 10-year-old.
On a trip to Ireland that same year Bartlett persuaded his parents to follow Hayes on tour for a week, during which Hayes spotted his "stalker" and introduced himself. A year later Bartlett promoted a concert by Hayes in Vermont and Hayes' agent only discovered that the impresario was a schoolboy when they spoke on the phone. Hayes stayed with the Bartletts that night and after he and young Tom shared a few tunes, Hayes kept in touch and knew immediately who to bring into The Gloaming on piano.
"I chose the people rather than the musicians," says Hayes. "I mean, with Caoimhim, people might have said, 'Why another fiddle?', but I felt that the tunes would need more than myself playing them to carry them over guitar and piano and I knew Caoimhim would bring the right personality.
"I feel we have just scratched the surface so far in concert and on the album and that there are lots of possibilities in terms of different sounds. I am certainly looking at the longer term and looking forward to see how we develop as a band because everyone is key to what we are doing. If anyone left it would be hard to replace them and the sound would change forever."
The Gloaming appear at City Halls, Glasgow, tonight. The Gloaming album is released on Real World Records.