j ack Whitehall's face hasn't prompted quite as much debate as the Turin Shroud but he has certainly polarised opinion.

It's not a bad-looking face, as it happens. Set underneath a crop of Dennis The Menace hair and punctuated by piercing blue eyes, it's well supported by a strong jaw line (visible when the beard goes) that wouldn't have looked out of place in The Dambusters.

Many will want to kiss this face, because it's the face of success. Whitehall's standing as a stand-up comedian is such that he can fill arenas (up soon, Glasgow's SSE Hydro) and his TV fame is almost sky high. He stars in the excellent Bad Education on BBC3, a chat show, Backchat, with his father, what seems like countless panel series, and there's his role in Channel 4's student comedy drama, Fresh Meat.

However, there's a section of the country which it seems would never tire of slapping the pale cheeks of this cheeky young comic. Many clearly begrudge the 25-year-old his TV triumphs, his £3 million house in Notting Hill and cosy nights in with actress girlfriend, Gemma Chan.

But why such vituperation? Whitehall himself accepts the public school accent plays its part. Brits loves to pillory posh people, (unless they are royalty, and then deference kicks in).

In a frustrated voice, he says journalists, for example, won't separate him from JP, the character he plays in Fresh Meat. "They often write the line 'Jack Whitehall is very good at playing JP - but is he really just playing himself?' I read this and I think it belittles entirely anything I've done in the show. It suggests none of the acting counts and because JP is such an arse, it hurts to think anyone could think I was anything like him."

Whitehall can indeed act. JP, with a skin thicker than three copies of Debrett's, works because the actor has imbued him with vulnerability. But does the young comic get critics' backs up because he's all too self-assured? Does Jack Peter Benedict Whitehall, like JP, give the impression he's Superiorman?

Or perhaps it's because Whitehall has the temerity to cross over from the likes of League of Their Own to clever clogs' shows such as Have I Got News and QI?

We're chatting in the basement of his PR company office in the west end of London, and Whitehall doesn't come across as cocky at all. He's polite and relaxed with the shiny stage suit wardrobed for a pair of jeans and a casual jumper. He's taller than imagined, but with the body of a sociology student who would struggle to bench press a wild flower. With the height comes a slight awkwardness, which makes him seem even younger than his years.

There's certainly no sign of any JP-like arrogance, evidenced when he talks about landing the Fresh Meat role. "I was totally surprised to be offered the part," he admits, grinning. "You see, in the script, JP was written as big rugby lad. And this physicality was important because there's one scene where he beats up Howard [played by Greg McHugh], picks him up and puts him in a bag and hangs him from a door. My first thought was 'Would anyone believe I'm likely to be able to beat up Greg? Probably not.' Thankfully, they adapted the character to fit me."

While Whitehall's voice may be redolent of the Raj, his family didn't fit the middle-class identikit drawing.

His dad, Michael, was a top showbiz agent; and baby Jack was delivered in the Portland private hospital in central London by an obstetrician in a dinner jacket on the way to a gynaecological bash, with godfather Nigel Havers, the actor, in attendance. And it was "All rather Downton Abbey."

Yet, the Whitehalls were more downstairs than upstairs.

"My granddad came from a lower-middle class family in Beckingham and he sold dresses out of the back of a van," says Whitehall. "Life was very much Hyacinth Bouquet, keeping up appearances, being socially mobile. My dad then worked incredibly hard all his life and every penny he earned was spent on sending me and my brother and sister to these f****** expensive schools. (Marlborough College, Kate Middleton's alma mater).

"The reality was we never had a flash lifestyle. We went on camping holidays because all the money went to the schools. We were taught to be frugal."

He adds, smiling: "I'm not saying I come from the street or anything, and I play up to the posh bit. But I won't inherit anything. I don't have any land or a title."

The education bought the posh voice, but not acceptance into the school's elite groups. Like his comedy hero Norman Wisdom, the young Jack Whitehall played the Outsider.

"I have a lot of character flaws and I was quite an awkward child," he admits."I wasn't part of the cool group at school and even now I'm not friends with them. I didn't excel at anything. In fact, I felt humour was the only thing I had going for me."

To get laughs he became Jack the Lad. Bored rigid in class, he'd be disruptive or overly talkative. In school plays or comedy sketches he'd take the stupid role to gain attention. Was he ever diagnosed with attention deficit disorder or labelled a problem child?

"Yes," he says, grinning. "I was once taken to see an educational psychologist, but she admitted she couldn't make head nor tail of me. They never gave me Ritalin, but I suspect had Ritalin been around then I'd have been a guinea pig for it."

Perhaps Whitehall was an outsider because he didn't come from the same economic background as his peers? "Perhaps. Perhaps I just needed recognition. You know, people say it must be hard being a stand-up but the honest truth is it's not hard for me because it's the only thing I can do. Any other job I've had I've been shit."

He adds, with a wry grin: "I can draw a bit. But how many portrait artists make a steady living?"

Indeed. On leaving school, Whitehall trod the route to conventionality by heading off to Manchester to study History of Art. But after stints in revue and comedy clubs, so many offers of comedy work came in that art was consigned to history.

Initially, on stage, he tried to contain his accent, but soon realised it was his USP. "I've come from a background of privilege and I do like playing on it. But at the same time, I have to let the audience know I'm in on the joke."

He does it cleverly, shown recently when he fronted Have I Got News For You, talking on the subject of Old Etonians in government.

"My personal belief is that it's ridiculous" he says. "But the best way I felt, to mock that, wasn't in saying 'Oh, we shouldn't have so many public schoolboys . . .' That would be disingenuous and stupid. So I said 'Oh, not all of the Cabinet are that posh. Michael Gove's actually from quite a rough background. He's only been skiing, like, twice.'"

The gag had the studio audience in stitches but Whitehall admits he worried whether people would get it, whether they'd think he thought Gove was in fact from a rough background. "When I do Have I Got News and shows like QI, I feel the viewers are not necessarily fans of me. In fact, I get more abuse when I do QI than any other show. It's watched by nice people, who then savage me on Twitter. The thing is I'm trying not to piss them off, although I quite like going on and subverting it a bit."

Sensitive to criticism but not beyond a little subversion. Admirable qualities. But perhaps some critics can't take to him simply because he's never off the telly. Is he wildly ambitious? "I always wanted to write a sitcom, to tour. And it's happened. I am quite ambitious but... (he pauses and thinks) I don't think I'm consumed by it all. The success is an overwhelming thing. I wake up every day and think 'Wow, this is amazing!' But what does consume me are the worries and the insecurities of keeping it all going. I never feel pleased with myself. I never feel I've done enough. I feel consumed with feeling empty."

He admits the insecurity pressures him to work hard at being funny. But can he enjoy the wealth? "It's nice," he admits. "But I can't drive, for example. So I don't see myself buying a Ferrari any time soon."

As chat runs on, what's apparent is Whitehall is certainly not Lord Snooty. And unlike many comedians, he's naturally funny off stage. What's also refreshing is he doesn't play it safe; opinions come at you hard and fast.

"Did you see Godfrey Bloom?" he says of the Ukip spokesman's appearance on HIGNFY. "What a complete f***wit, completely devoid of self-awareness, without the filter people normally have.

"And Sally Bercow? She brought extra cringiness to the show because she went to the same school as me. What role models!"

Whitehall has a fearless critical voice, but the media often bat it right back at him. When the comedian told a fairly anodyne gag about the Queen, the newspaper reaction suggested he'd drop-kicked one of the corgis over the palace wall.

"There was a story about Prince Harry in the first episode of Fresh Meat, where JP claims Harry was supposed to have got off with a girl who worked in Abercrombie and Fitch.

"The tabloid heading ran: 'Channel 4 defend disgusting Harry slur.' And it had a pic of me. But who believed I was reporting fact? I play a fictional character. And here's the thing; Harry has a notorious sense of humour. Like he'd have given a shit."

More recently while hosting the British Fashion Awards, Whitehall had fun with Nigella Lawson's face powder of choice, which had some newspaper writers snorting with derision.

It was especially bold, given he'd once been on the receiving end of a story, when he was pictured with a white powder on his lap that wasn't sherbert.

"I've been clear on this," he says of his travails. "I'm not a saint or looking to set a moral code. Don't guide your life by me."

But does the sharp lens focus make him trepidatious about going out on the town?

"You get better at it (avoiding headlines) but you don't want to be a hermit or a weird alien. And you do behave better. Although there's no one in my line of business who isn't glad the News of the World has gone."

Whitehall's girlfriend was also pictured at the end of last year with One Direction's one-man love machine, Harry Styles, with a caption suggesting she and little Harry were an item.

Did this grate? "It was me who took that photo," he says, laughing. "If you're going to let your girlfriend have one off the wish list, it might as well be Harry."

Pragmatic and good fun. And when he goes on to talk about his friendship with Kevin Bridges, (he's stayed with the Scot in Clydebank) you realise he's far from the toff sensibility. "When I meet Kev, he comes along with some of his mates from school. And I do the same, with blokes I've grown up with. It's not about joining up with famous people you've only just met in a TV studio."

He adds: "It may all change. When we're 50 we may find ourselves hanging out with celebrities, but I hope not."

What's refreshing about Whitehall is he's more than prepared to put himself down. And it's not a performance.

"When you're younger you feel you're funny because your friends laugh at you. Then you go up on stage in front of strangers and no one laughs. Suddenly you become more self-aware.

"Here's the thing; to go into stand-up as a teenager there has to be a slightly deranged element to your personality. I certainly had that. I wanted people to laugh at me.

"But getting up on stage helps you deal with that need. You become a slightly more normal person and you don't feel the need to entertain people all the time."

Yet, he worries a lot. He worries he comes across as a prat in interviews. He worries that his new show won't hit the mark. But he has nothing to worry about. The sky's the limit for someone with a sharp, dark sense of humour, who is always self-deprecating, and clearly intelligent.

"I think my dad would beg to differ," he says, smiling, in his best David Niven accent.

Jack Whitehall, SSE Hydro, Glasgow, March 7