Erratic bedtimes can cause a similar effect to jet lag and the longer youngsters go without regular bedtimes, the greater the impact on their behaviour, the researchers claim.
They conclude going to bed at different times could disrupt body rhythms and cause sleep deprivation. In turn, this undermines the way the brain matures and the ability to regulate some kinds of behaviour.
However, they also found the effect is reversible: parents who started putting their children to bed at consistent times noticed an improvement in behaviour, as did teachers.
The study said to have been of more than 10,000 children was carried out at University College London.
The information was collected via the UK Millennium Cohort Study, with bedtimes noted at age three, five and seven, and information on behaviour collected from parents and teachers.
Irregular bedtimes were most common at the age of three, when around one in five children went to bed at varying times.
However, by the age of seven, more than half of children went to bed regularly between 7.30pm and 8.30pm and just 9% went after 9pm.
The researchers found those youngsters who experienced erratic bedtimes throughout childhood displayed progressively worse behaviour. But those children who went from varying bedtimes aged three or five to a regular bedtime by age seven displayed a improvement in behaviour.
The research is published in the journal Pediatrics.