He said mediators had indicated time was slipping away to get the only US soldier held captive in Afghanistan out safely.
Mr Hagel, the first Obama administration official to testify publicly about the controversial deal, told the House Armed Services Committee Qatari officials warned in the days before the exchange that a leak would sabotage the deal.
The transfer of five detainees from Guantanamo Bay in Cuba, to Qatar was legal and advanced national interests, he added.
Republicans and some Democrats have criticised the administration for not informing Congress in advance, with some accusing the president of breaking a law requiring 30-day notification of any Guantanamo prisoner release.
Other questions centre on whether Sgt Bergdahl deserted and whether America gave up too much for his freedom. Administration officials have told Congress four of the five Taliban officials will likely rejoin the fight.
"We could have done a better job of keeping you informed," Mr Hagel said.
However, he said the "extraordinary situation" combined time-sensitive concerns over Sgt Bergdahl's health and safety and persistent fears the Taliban may have been negotiating in bad faith.
Mr Hagel added: "We grew increasingly concerned any delay, or any leaks, could derail the deal and further endanger Sgt. Bergdahl. We were told by the Qataris a leak would end the negotiations for Bergdahl's release.
"We also knew he would be extremely vulnerable during any movement, and our military personnel conducting the hand-off would be exposed to a possible ambush or other deadly scenarios in very dangerous territory."
Howard McKeon, the committee's Republican chairman, said the agreement with the Taliban could fuel further kidnappings of US personnel.