Iain Duncan Smith, the Work and Pensions Secretary, said there would be a "further tightening up" of the rules, including the possibility of strengthening those on residency so that individuals would need to show they had a lease on a home "of nine-months to a year rather than just a matter of months" before being able to claim benefits.
Last week, David Cameron made clear the UK Government was determined to crack down on migrants seeking to "take advantage" of Britain's welfare system.
The Prime Minister will host a policy summit on immigration at Chequers on Thursday with, among others, Lynton Crosby, his election strategist.
Concerns about access to benefits have been raised as from 2014 the restrictions on Bulgarians and Romanians seeking work in Britain will be lifted.
Mr Duncan Smith, outlining a "big battle" with Brussels on the subject, said the European Commission had to understand "people shouldn't use the free movement rules just to travel around, looking for the best benefits they can get".
He said a number of northern European countries were in tune with the UK Government's thinking on this. The Secretary of State noted: "We will be able to tighten up and make those regulations much tougher for people coming in just to take advantage of our benefits system."
But Mr Miliband said the Coalition should concentrate on action against rogue employers who exploited cheap labour, rather than "windy rhetoric" about challenging EU rules.
He said: "One very specific thing the Government should be doing before Romanian and Bulgarian accession in terms of the openness of borders is to say: 'Let's make sure we clamp down on rogue employers; rogue employers who bring people in and pay less than the minimum wage."
Meanwhile, Home Secretary Theresa May accused judges of subverting British democracy and making the streets of the UK more dangerous by ignoring new rules aimed at deporting more foreign criminals.