The TV judge spoke out after fans of the hit BBC2 show pointed to an on-screen flirtation between the pair.
Hollywood, 47, who judges Bake Off with Mary Berry, dismissed as "rubbish" the suggestion that former model Tandoh's appearance had improved her chances on the show.
"Personally I think (fellow contestant) Kimberley's far prettier. With all the love in the world, Ruby's not my type," he told the Radio Times.
"But she's a great baker, one of the most talented bakers who has walked in that tent."
The judge, who left Alexandra, 49, his wife of 15 years, amid an attempt to launch his career in the US and reports of a relationship with his US co-star Marcela Valladolid, 35, added: "My job is to judge what goes on the plate. Always has been, always will be.
"I don't judge the person - Mary will back me up on this - because it's not one judge doing this. It's two. And we both say the same thing."
Hollywood said that it was coincidence that the fourth series of the hit show will have no men in the final.
"We had an all-male final last year, and an all-women final the year before," he said.
"There's no difference between male and female bakers. All I'm thinking of are the bakes. It's by the by what sex the bakers are."
Meanwhile, Tandoh, who is studying for a philosophy degree, said that she was not bothered by accusations of favouritism.
"It's easy to brush it off because I know how it was in the tent - and there was no favouritism. It was all very fair so that doesn't get to me very much," she said.
Publishers are said to be preparing to approach the Essex-born student to write a cookery book, which could lead to her own TV show.
Asked whether she thought Hollywood was a sex symbol, Tandoh told student magazine The Tab: "Not for me! Not my thing but, you know, for other people maybe that's what they go for."
Meanwhile, fellow contestant Frances Quinn, 31, said that the bakers were taking Rescue Remedy, a natural flower solution, to get them through the show.
"I feel bad because I'm the one that started the trend for it. I became Ruby's feeder! You're only meant to have a few drops. We'd jokingly go, 'A few drops, or a few shots?'," she told the Radio Times.
Meanwhile, it has been revealed that the hit show is to be moved to BBC1 next year as a result of its rising ratings.
The contest, which reaches its climax with the final of the fourth series next week, has had BBC2's highest viewing figures so far this year with an average audience of seven million.
BBC1 controller Charlotte Moore announced the move of channels today and it will begin for the fifth batch of shows.
She said: "It's been fantastic to watch it flourish on BBC2 and I can assure viewers I will continue to cherish it on BBC1."
The audience has grown from an average of 2.8 million for the first series in 2010, peaking with 7.8 million a fortnight ago - more than even last year's final, which drew a total of 7.2 million
The show - featuring experts Mary Berry and Paul Hollywood - has been credited with giving the nation a new-found interest in baking, and the format of the show has been sold to more than a dozen countries.
Moore, who originally commissioned the show when she was in another role in 2009, said she thought the time was right to bring the show to "an even broader audience".
"I've watched the series grow over the last few years and earn a special place in the nation's hearts. It's inspiring and warm storytelling at its best which celebrates the huge talent of enthusiastic and passionate bakers from all over the country," she said.
Executive producer of the show Anna Beattie, who is chief executive of Love Productions which makes it, said: "We're extremely proud that the show has captivated so many families and inspired so many people across the country and we aim to do everything we can to ensure its continuing authenticity and integrity."