If you are a fan of TV panel shows, it's likely you've come across Romesh Ranganathan on the box this year.

After a breakthrough 2013, the former teacher from Crawley has become a well-known face, bandying clever and topical quips alongside the likes of Seann Walsh, Sean Lock and Russell Howard.

He thinks his comic career trajectory is thanks to being in the right place at the right time, and says Walsh in particular took on a mentoring role, offering him gig recommendations and taking him on as his tour support. "He just generally looked out for me," Ranganathan explains. "And a couple of lucky breaks then followed.

"I had never entertained the possibility of stand-up comedy being a job, I just didn't think it was an option. It was just something fun on my bucket list. But when I was entering - and getting to the final of - competitions like So You Think You're Funny, it actually became viable. In 2010, when Dan Antopolski came up to me and said I could do it for a living, that's when I began to take it seriously."

Ranganathan already had a successful career as a high school maths teacher, and had achieved a promoted position in a comprehensive school. "I taught for seven years, working up to assistant head and head of year," he reminisces. "At that level you're kind of like the Gestapo, though, always interrogating kids and trying to get them to own up to things.

"My discipline style was never to raise my voice though, quieter was always better. And nine times out of 10 the kids got it. I rarely had any major problems, as I'd stare them down in silence. Pretty impressive stuff, I suppose, as my last school was like Michelle Pfeiffer's in Dangerous Minds."

I suggest that this death stare is something he has brought to his new career, as I experienced it first hand in Edinburgh last year. When I let him know that he stared me down (as I shuffled into the front row after a particularly arduous Fringe queue) he is clearly mortified. "I didn't… I'm so sorry. Was I touching my glasses too? My wife says I do that all the time; one time she came to my gig and she counted how many times I did it."

His dead-pan expression is not reserved for latecomers, however, and can sometimes get him into amusing situations. He tells me of an incident in Brighton when Jill Edwards, who runs stand-up comedy courses attended by then up-and-coming acts such as Seann Walsh, proclaimed to the other aspiring comics: "Don't be put off, don't be put off by Romesh's face!"

As much as he can't help his face, his facial expressions are - for me - part of the winning combination that has secured his success. Last year's show, Rom-Com, received pretty solid critical acclaim and he was shortlisted for the Foster's Best Newcomer Award.

"Last year's show was kind of inward looking: I talked about the people in my house, my wife and kids, and my mum. This year I'm more outward looking. I might even venture into soft-play territory."

Family is clearly important to Ranganathan and he admits to being a complete "homebody". The opening gambit of our phone conversation was that his four-year-old son's sports day had just been cancelled and I couldn't work out who was more gutted, him or son Theo. This year, his family will be with him in Edinburgh for the duration, as being apart from them was his biggest gripe from last year's Fringe (that and finding vegan food in the early hours of the morning).

His funny bones, he says, are thanks to his dad, who sadly passed away two days after Ranganathan's decision to leave the teaching profession.

"My dad was very funny, he made me love comedy. My mum's funny too, but in a different way," he says.

"My first taste of the stage was when I was eight years old at a Pontins holiday camp - I memorised lots of jokes and decided it would be hilarious to deliver them in a Sri Lankan accent. I don't really know why I thought that would be funny. My act is definitely more honed - and less offensive - now."

The art of self-improvement is the main thrust of his new show, Rom Wasn't Built in a Day.

"It's a sequel to Rom-Com. I'm just trying to make myself more interesting with some self-examination. In three words? Grumpy. Sardonic. No, scrap sardonic. Hopefully. Funny."

I point out he has "improved" his three words before they've even gone to print. "I just want the show to be good, well, better than last year," he says. Again, this self-improvement. "Much like teaching, some days you think you're really good at it, and some days it should be illegal. It's not easy, it can be really hard work. Putting a show together is hard."

Setting himself yet another Festival challenge, Ranganathan's (drama teacher) wife is due to give birth in early September - and has previously gone before her due date with both of their children. Perhaps the Fringe baby could be a whole new angle for self-improvement?

"In three words, this year is going to be amazing, exhausting and a buzz. You'll mainly find me at kids' shows, although I don't really have the face for an audience member, I've been told."

Romesh Ranganathan plays the Pleasance Courtyard (Beneath) at 8.15pm, from July 30 to August 24 (not 11th).