The much-loved movie – starring Tim Robbins and Morgan Freeman – was nominated for the 1995 Academy Awards but missed out on the prize to Forrest Gump.
It was ranked the greatest film to slip through the net in a survey to launch a new dedicated Sky Movies Oscars HD channel.
Shawshank was followed in the list by The Green Mile, which was made by the same director, Frank Darabont. Both films were based on stories by novelist Stephen King.
Quentin Tarantino's Pulp Fiction, in joint eighth position, also lost out to Forrest Gump in 1995.
James Cameron's sci-fi film Avatar – which lost out to The Hurt Locker in 2010 – was ranked in third place.
Three films dating back to the 1960s made the list –Mary Poppins, Doctor Zhivago and To Kill A Mockingbird.
Director of Sky Movies, Ian Lewis, said: "Arguably the most prestigious accolade in the film industry, competition is fierce when it comes to the Academy Award for best picture – and, as we've found, even some of the nation's best-loved movies of all time were pipped to the post in this category."