The gunmen, thought to be members of rebel sect Boko Haram, attacked one hostel, shot some students while they slept, took some outside before killing them and shot others trying to flee.
One survivor of yesterday morning's shooting, a student who would only give his name as Idris, said: "They started gathering students into groups outside, then they opened fire and killed one group and then moved on to the next group and killed them. It was so terrible."
Ahmed Gujunba, a taxi driver who lives near the college, said: "They came with guns around 1am and went directly to the male hostel and opened fire on them.
"The college is in the bush so the other students were running around helplessly as guns went off and some of them were shot down."
Molima Idi Mato, of the Yobe State College of Agriculture, said the gunmen also torched classrooms in the attack which forced about 1000 students to flee the scene.
Boko Haram, which wants an Islamic state in northern Nigeria, has intensified attacks on civilians in recent weeks in revenge for a military offensive against its insurgency.
Several schools, seen as the focus of western-style education and culture, have been targeted.
Boko Haram and spin-off Islamist groups like the al Qaeda-linked Ansaru have become the biggest security threat in Africa's second largest economy and top oil exporter.
Western governments are increasingly worried about the threat posed by Islamist groups across Africa, from Mali and Algeria in the Sahara to Kenya in the east, where Somalia's al-Shabaab fighters killed at least 67 people in an attack on a Nairobi shopping mall a week ago.
After yesterday's attack, bodies were recovered from dormitories, classrooms and outside in the undergrowth by members of staff at the college.
Dozens of corpses were later piled on the floor at the main hospital in Yobe state capital Damaturu, mostly of young men believed to be students.
The bodies were brought from the college, which is in Gujba, a rural area 30 miles south of Damaturu and around 130 miles from the borders with Cameroon and Niger.
State police commissioner Sanusi Rufai said he suspected Boko Haram was behind the attack.
Thousands have been killed since Boko Haram launched its uprising in 2009, turning itself from a clerical movement opposed to western culture into an armed militia with growing links to al Qaeda's west African wing.
President Goodluck Jonathan declared a state of emergency in three north-eastern states in May, including Yobe, and ordered a military offensive to crush Boko Haram's insurgency.
There was an initial lull in the violence as Islamists fled bases in cities, forests and mountains before the militants began revenge attacks on schools, security forces and civilians believed to be helping them.
In July, suspected Boko Haram militants killed 27 students and a teacher at a school in Potiskum, a town about 30 miles from the site of yesterday's attack.
Several hundred people have died in assaults over the past few weeks. Some observers say the army offensive has only succeeded in pushing attacks away from well-guarded large towns and cities into vulnerable rural areas.
Boko Haram's insurgency is also putting pressure on the economy of Africa's most populous nation.
Nigeria's security spending has risen to more than £4 billion per year, or around 20% of the federal budget.