Attorney General Muhammad Isaaq Aloko met members of a Taliban peace negotiation team in Dubai, despite being told by the Presidential Palace not to attend the meeting, a senior Afghan official said.
A prominent member of parliament also said Mr Aloko had been sacked.
Peace talks between the Karzai administration and the Taliban are seen as crucial to averting another round of war as the Nato- led force in Afghanistan prepares to end its military mission in the country by the end of next year.
Talks with the Taliban began in 2010, but they have been marked by a series of false starts, delays and allegations of plotting and interference.
The senior Afghan official said some senior cabinet members were trying to persuade Mr Karzai to reverse his decision to dismiss Mr Aloko.
The meeting, held early this month, also involved other prominent Afghans.