"One of the kids at school rather cruelly told me that and, obviously, I believed it and so I used to lie awake every night," she said. "I couldn't stop sucking my thumb but at the same time I was utterly terrified of what kind of contraption an orthodontist would have to use on my mouth."
Fisher is promoting her latest film, Rise of the Guardians, in which, somewhat aptly, she lends her voice to the Tooth Fairy.
Based on the books by William Joyce, the animated film follows a band of heroes, led by Santa and including the Easter Bunny and Jack Frost, as they attempt to thwart a plan by an evil spirit named Pitch Black to rob children of their dreams and replace them with nightmares.
For Fisher, accepting the role was a no-brainer given her previous experience with animated hits Rango and Horton Hears A Who. But it wasn't all plain sailing.
"I specialise in silly voices, so at my first meeting I had 20 options of ridiculous voices for the movie and they said: "Oh, we want you to do you, but with an American accent." So, I felt quite vulnerable just doing my voice: "Is this enough? Am I enough?" But they're happy with it, so ..."
Given the nature of the film's themes and characters, questions inevitably turn to Fisher's own family (she's married to actor-comedian Sacha Baron Cohen) and whether her two daughters believe in Santa, or if she's yet to play a real-life Tooth Fairy.
"No-one's lost a tooth, so I don't have to think about it. But I haven't lied to my kids yet. I've managed to divert if questions have got hairy. But I don't know how I'd feel about it yet. I'm not ready to make that decision."
Being a mother is something Fisher holds dear, which is why she has taken the decision not to pursue leading roles. "While my munchkins are really small, I don't want to miss anything," she states.
When she does look for work, however, it's driven by gut instinct and the notion that "if something really terrifies me and I think 'oh my God, why would they offer that to me, I'm totally going to ruin it', then I should do it".
This was particularly true when accepting a role in forthcoming heist thriller Now You See Me, in which she plays against type as a "sexy, bad-ass" escapologist who moonlights as a bank robber.
"Why would I be offered that role?" she ponders. "I don't see myself as particularly courageous or sexy or any of those things. So, I took it, because that's just an interesting thing to do."
It also enabled her to study and learn magic, as well as giving her the chance to work alongside Michael Caine. "For me, that's a huge honour," she gushes excitedly.
But it also led to a near-death experience when an underwater stunt almost backfired.
"I don't know what happened to me because I'm a really fearful person in real life. But I did all of the stunts. And there was one stage where the chain got caught underneath one of the bars at the bottom, where even though I could release one part, I couldn't get the other out and I did think: 'Oh, is this how I'm going to go? I never saw it coming!'"
Another role about which Fisher talks excitedly is in Baz Luhrmann's forthcoming remake of The Great Gatsby, which enabled her to go back home to Australia.
"It's one of those jobs that I auditioned for and you never even let your mind wander to the hope that you could get the gig," she said. "You just think, this is a great opportunity to even be in a room with him [Baz Luhrmann]. And then when I got the job I think I screamed down the phone and deafened my poor agent."
So, how was the experience of filming? "It was just wonderful. They recreated Long Island in the 1920s in Sydney. It was surreal. And it was exactly how I'd imagined it from the book. It was fun to work in Australia too. It's just nice to go home."
Fisher may refer to Australia as home but she was born to Scottish parents in Oman and currently lives in England. It's an inevitable temptation to ask what living with Cohen is like.
"You have no idea," she laughs. "Particularly at social functions, my husband's faux pas are really fabulous. I remember him asking Cate Blanchett at the Golden Globes what she did for a living."
Rise of the Guardians opens in cinemas tomorrow.