Just 54% of adults like the dessert, making it less popular than Brussels sprouts, favoured by 64%, the survey for BBC Good Food magazine found.
Instead, 12% will be serving trifle, 12% will present a chocolate cake and 8% will make some other form of cake.
The survey also found 16% of respondents plan to spend less on food this year and just 9% will spend more, while 23% will spend less on presents and 8% will spend more.
It revealed 62% will sit down to turkey, although 39% say the main reason is that they have always had it at Christmas and 14% say there would be a "family uproar" if they changed the menu.
A tenth of adults (12%) favour beef and 11% are planning a vegetarian meal.
The Christmas meal is the best part of the day for 32%, followed by opening presents (21%), and preparing the main meal (6%).