Alex Salmond clashed with his Labour rival Johann Lamont on the issue, insisting that student numbers are at record levels.
The equivalent of 133,199 full-time students enrolled at Scotland's colleges in 2011-12, up from 129,343 in 2006-07, he said.
"That seems to be a substantial improvement and substantial achievement over these financially difficult years," the First Minister said.
But the total number of college students fell by 16% between 2010-11 and 2011-12, to 257,913.
Despite being pressed by Ms Lamont on the overall number of students, the First Minister did not give this figure.
The Labour leader repeatedly asked Mr Salmond if student numbers have gone up or down since he came to power in 2007.
"Simply tell us, has the number of people going to college gone up or down?"
When the First Minister did not provide the total number of students, Ms Lamont criticised his "complete failure" to answer the question.
The SNP administration has cut the number of hours in full-time college courses from 720 a year to 640 a year.
"That's what this Government has done: cut the hours of teaching, cut the definition of a full-time course and graduated with honours in cooking the books," she said.
Mr Salmond insisted: "It is a matter of great pride in Scotland that this year we have a record number of students in higher education across the university and college sector of Scotland."
But after he cited the full-time equivalent number of students, Ms Lamont called him a "part-time equivalent" First Minister.
The SNP leader, in turn, questioned whether the Labour leader, a former teacher, is "an enormous gain to Scottish politics".
He said: "I don't think so because she ignores the fact I have now mentioned several times that having a record number of full-time students in higher education in Scotland seems to me a very substantial achievement."
Ms Lamont, speaking during First Minister's Questions, recalled that Mr Salmond previously pledged to re-industrialise Scotland.
With overall student numbers down, however, she said: "This First Minister isn't re-industrialising Scotland, he is de-skilling Scotland."
Labour cited figures from the Scottish Funding council which show that the number of women in college fell by 77,210 to 140,494 over the period 2007-08 to 2011-12.
At the same time the number of 25 to 29-year-olds going to college fell by 64,045 to 108,78, while the number in this age group who had no qualifications when they started college dropped by 57,845 to 66,853.
"Let me tell the First Minister some of the facts," Ms Lamont said.
"Since 2007 the number of women going to college has fallen by 77,000. The number of adults who left school with no qualifications but got a second chance at college has almost halved, nearly 60,000 people denied that second chance to learn.
"Even the number of 16 to 18-year-olds going to college has fallen by 7,000."
Mr Salmond said the Scottish Government "prioritised" full-time college courses, telling MSPs: "The reason that we have prioritised full-time courses in the colleges of Scotland is we believe that is the courses that lead into employment, it gives people the skills that are required.
"That is perhaps one reason why unemployment, although far, far too high in Scotland, is better than it is south of the border and there has been a significant improvement over the last year."
He also highlighted the Scottish Government's record on providing free education.
"It does also strike me that the fact that none of these students, Scottish students in full-time education in Scotland, are paying tuition fees is another substantial achievement that should be welcomed," he said.
"Perhaps that's why we have a record number of students in full-time higher education in Scotland, a record number this year when numbers south of the border over the last few years have been falling like a stone."