The three adults and four children were seized in Cameroon near the Nigerian border last week. In a video, the gunmen threatened to kill them unless authorities in Nigeria and Cameroon released Muslim militants held there.
Mr Le Drian said: "We do not negotiate on that kind of basis, with these kind of groups.
"We will use all (other) possible means to ensure these and other (French) hostages are freed."
"We do not play this bidding game because that's terrorism," he said, deploring the fact children were involved.