Baroness Warsi, resigned last week, over the Government's policy on Gaza, said Tory bosses were ignoring "electoral reality" by relying on white voters. She also suggested Mr Cameron's inner circle did not understand those who had not gone to public school.
"I will be out there, vocally fighting for an outright Conservative majority," Lady Warsi said. "But the electoral reality is that we will not win outright Conservative majorities until we start attracting more of the ethnic vote."
Lady Warsi said that when she was one of Cameron's earliest supporters in 2005 she thought: "This is a guy who gets today's Britain." But she added: "I think the party has shifted since then. I think over time it will be a regressive move because we have to appeal to all of Britain, not just because it's morally the right thing to do ... but because it is an electoral reality.
"We've probably left it a little too late to take this part of the electorate seriously."
Former Foreign Office minister Lady Warsi called on the Government to "recognise Palestine as a state" and impose an arms embargo on Israel.
And she attacked Chancellor George Osborne and Chief Whip Michael Gove for failing to use their "very, very close" relations with the Israeli Government to defuse the crisis.
"What is the point of having that strong relationship if you can't use it to move them to a position which is in their interests and our interests?" she said.
Dismissing Mr Osborne's claim that her resignation had been "unnecessary", she said: "My actions would not have been necessary if he had done what he should have done, which is pick up the phone to people he is incredibly close to and say: 'It's unnecessary for you to meet your ends by taking out power stations, taking out homes, taking out schools and killing kids on beaches'."'