Michael Russell said some universities had done very poorly and needed to widen access.
Earlier this month, The Herald revealed some universities had been recruiting tiny numbers of students from the poorest backgrounds.
St Andrews University – where Prince William studied – recruited only 13 students from the most deprived backgrounds in Scotland in 2010/11. The second lowest proportion was at Aberdeen University, with 51, followed by Edinburgh University, with 91.
The Scottish Government has already announced plans to give universities binding targets, with the threat of financial penalties for those that fail.
Mr Russell told the Scottish Parliament: "I believe we have to put into statute the issue of widening access agreements with the higher education sector and that is what we will do.
"That will allow us to ensure there are carrots and sticks. I am more in favour of carrots in this matter than sticks, but we must make sure we continue to drive up access and we get it moving faster than we have done.
"Some universities have done exceptionally well, some have done very poorly and I want to make sure everyone is doing exceptionally well."
Mr Russell spoke out after making a statement on the next stage of reforms to post-16 education, which include merging colleges in new regions.
He said each region would have one over-arching governing board which would be appointed once the mergers are finalised.
In the meantime, key figures from the education sector have been asked to lead regional planning in each part of the country, including former First Minister Henry McLeish, who will oversee changes in Glasgow. Mr Russell also announced an advisory board for the university sector and said the committee of Scottish Chairs of Higher Education Institutions will lead work to develop a code of good governance.
Opposition politicians said colleges were facing an unprecedented level of interference.
Hugh Henry, Scottish Labour education spokesman, accused the SNP of a "power grab" as he hit out at "shotgun marriage" mergers of some colleges.
He said: "They have introduced an unprecedented level of ministerial control and interference in our colleges. New chairs and boards will be appointed by ministers and can be removed by ministers."
Tory education spokeswoman Liz Smith asked what evidence there was to suggest the current reforms were "in some way insufficient" and required Scottish Government direction.
The University and College Union, which represents lecturers, welcomed Mr Russell's commitment to governance, but said there had to be real engagement with staff and students.
The Educational Institute of Scotland union and Scotland's Colleges, which represents principals, said more clarity was needed over how the new regional boards would interact with current college boards.
In a separate development, Edinburgh University denied reports fee-paying students from overseas were allowed in with lower entry grades.
A Chinese student recruitment agency said students could study economics at Edinburgh without the same academic requirements, but a spokesman said there was no contract with the company and all applicants had to meet entry requirements.