Three years ago, on the eve of appearing in the third series of Downton Abbey as the sainted Matthew, husband to Lady Mary and all-round good, middle-class egg, he made a prediction.
"There's still some rebelling in me to be done," he said. "For my next part I should choose something completely out of character to what everyone expects. I don't know what, but you should always expect the unexpected."
The Guest, Stevens's new film, is about as out of the blue as a shower of grand pianos. At a friends and family screening this week, someone described his character, a soldier recently returned from Iraq, as "Captain America gone very, very wrong", and the film as The Night Of The Hunter meets Universal Soldier with a touch of The Terminator. Cousin Matthew as Arnie? Are there enough smelling salts in the world to rouse the Dowager Countess from such a shock?
"She'll love it," laughs Stevens.
The DC would not be alone. Part thriller, part horror, wholly tongue in cheek, The Guest is inviting the sort of reviews that would lead a 31-year-old actor to think it wasn't such a bad decision after all to leave a globally successful television show to run away to that other screen circus, the movies.
Stevens did not immediately see himself in the role of David, the vet who turns up on the doorstep of a fallen colleague to offer his condolences, and more. He loved the script's darkly comic bent, though, and knew he had to have the part.
"I certainly wasn't a shoo-in. I had to go and meet the director, Adam Wingard. As soon as we met it was very quickly established that we had the same kind of twisted sense of humour and that we'd watched and loved a lot of the same films growing up, and that we had the same kind of vocabulary."
With Wingard's mother being a Downton Abbey fan, there was a fair amount of teasing going on amid the bonding. "I think he was very amused by the prospect of taking this nice young chap from Downton Abbey and turning him into a bad ass."
When he met Wingard, Stevens had just finished playing a scrawny drug trafficker in A Walk Among The Tombstones, the eagerly awaited adaptation of the Matt Scudder mystery with Liam Neeson as the private eye. Stevens had to go from pipe cleaner to a pumped-up muscle merchant, necessitating a training regime that involved four hours in the gym each day, every day.
"I really threw myself into the physical side of things. It's something I hadn't really had a chance to do much before. I wasn't the sportiest kid growing up but I've since learned to enjoy looking after myself a bit. I certainly enjoy watching sport but that doesn't tend to get you in great shape."
He also had to train in martial arts and learn how to dismantle a Glock. All a world away from Downton, where the most dangerous thing he was on the other end of was one of Lady Mary's moods. Stevens was in the show from 2010-2012. Like the rest of the cast, he was pleasantly surprised at how it took off, winning Baftas, Golden Globes and Emmys along the way.
"From the first episode it had got very healthy figures, it was nicely reviewed, and people were very enthusiastic about it. That snowballed throughout the first series in the UK, then it aired in the States and it went crazy. Then it just kept getting bigger and bigger."
The new series starts on September 21. As regular viewers will know, Matthew is now conspicuous by his absence, having died in a car crash at the end of series three. Stevens had made it known that he wanted to leave, but he did not know the manner of his exit. Viewers were similarly caught by surprise when the episode aired on Christmas Day. Middle Britain was enraged, and took to Twitter to have a go at the programme makers. Some were also furious at Stevens.
"It's amazing how the supposed anonymity of something like Twitter gives people a licence to feel that they can say things I'm almost certain they wouldn't say to me face to face," he says.
But Stevens, who is married to the singer Susie Hariet, believed he was doing the right thing. "It was a very tough decision. My wife was pregnant with our second child at the time, and I didn't know what I was going on to, honestly. But something just told me that I'd had a great three years, time to move on."
Before Downton, the Croydon-born Stevens had been best known for television adaptations of Sense And Sensibility and The Line Of Beauty, though he had also done a lot of theatre, principally Shakespeare with Sir Peter Hall. Moving into films was a leap that his friends, chief among them Benedict Cumberbatch, had made successfully.
Stevens is part of that Cumberbatch-Hiddleston-Redmayne generation of British actors, all privately educated, all terribly pukka, all making it big in America.
Well, he is and he is not. Stevens went to boarding school, but he did so on a scholarship, and with the encouragement of his adopted parents, both teachers. From there he made it to Cambridge and a degree in English Literature.
He was reminded of his university days when he was asked to be a judge for the Man Booker prize in 2012.
The task of reading 148 novels in a matter of months was "heaven and hell combined".
"I found the judging experience very inspiring but actually the feat of having to do all that reading was like a thousand essay crises rolled into one."
After The Guest and A Walk Among The Tombstones, he will be seen next in the latest Night At The Museum movie, out at Christmas, in which Robin Williams also appears.
The shock of Williams's death remains raw. "It's still very recent and I'm still reeling from that. Not really ready to talk about it yet."
Another film, The Cobbler, premieres at the Toronto International Film Festival this month. Starring Adam Sandler and directed by Tom McCarthy (The Visitor), the fantasy comedy drama about a shoe repairer who can step into his customers' shoes in every sense will introduce Stevens further to American audiences.
It is an audience he should be starting to know well, having been an Englishman in New York (Brooklyn to be precise) for a couple of years now.
He adores the neighbourhood where he lives with his wife, his two-year-old son and four-year-old daughter, knowing it was once home to Norman Mailer, Truman Capote, Arthur Miller, Walt Whitman and, one of his favourite poets, Hart Crane.
"I grew up reading all these people and reading about some of the streets I'm now walking.
"I didn't go there till I was in my early twenties but I'd read about this place. It had lived in my imagination for a long time."
Once again, the imagination that took Stevens from television to the movies, from Downton to The Guest, has served him well.
The Guest is in cinemas now.