The call comes after figures showed councils have recorded more than 2000 incidents of children being disciplined in relation to phone use over the past three years.
The true figure is likely to be significantly higher, with many local authorities failing to collect information on the issue.
The Conservatives said taking phones away from pupils rather than asking them to switch them off, as is currently the case, would be a more proportionate response rather than an outright ban.
The party believes the move would go some way to addressing cheating in the classroom, unnecessary distractions and the issue of cyber bullying.
Mary Scanlon, the party's education spokeswoman, said: "In the classroom, the priority for everyone has to be the lesson in hand and it is simply too easy for a pupil to access a mobile phone in their pocket, distracting themselves and others in the process.
"This is something parents and teachers want to see us get tough on - it should very much be the norm that mobile phones are handed over at the classroom door.
"An outright ban on phones from school premises would be excessive, particularly as children may need to contact parents at home time, but we have to learn to use technology responsibly."
The Conservatives also called for better recording of incidents with some councils unable to provide figures.