Yemen is dogged by internal conflicts and chronic poverty and the attack highlights the lack of stability in the Western-allied nation, which shares a border with top global oil exporter Saudi Arabia and lies next to major shipping lanes.
The Yemeni interior minister suspended senior security officials in the eastern province of Hadramout where the attack took place and ordered an immediate investigation, official news agency Saba reported.
A local official said the troops, who belonged to a paramilitary unit, were mostly asleep when the raiders attacked the checkpoint, some 75 miles east of the provincial capital al-Mukalla.
There was no immediate claim of responsibility for the attack, in which one person was also wounded.
A security source in the capital Sanaa said two members of the security forces had also been seized and taken away by the militants.