The Kirkcaldy and Cowdenbeath Labour MP used the term as he was interviewed at an education conference in the Gulf State of Qatar.
He was reminded by the chairman of the session, BBC presenter Mishal Husain, that he was "still an MP, so still a politician". Since he left Downing Street in 2010, Mr Brown has faced a stream of criticism for his infrequent appearances in the Commons chamber.
But his office insists that he combines working hard for his constituents with his international commitments as United Nations Special Envoy for Global Education.
Friends of Mr Brown defended his comments last night, saying they were in answer to a question and he was trying to point out he was no longer in government and no longer in charge of policy decisions.
But SNP Westminster Leader Angus Robertson accused the former Labour leader of forgetting about those who voted for him.
He said: "It would seem Gordon Brown has forgotten about his constituents, though it is unsurprising given we have hardly seen him in parliament over the last year. It is astounding he is still claiming a full MP's salary."
Tory MP Henry Smith also claimed that Mr Brown's "foreign jaunt" showed he "can't even be bothered to turn up to work" for his constituents.
Mr Smith said: "Crisis talks averted the loss of hundreds of jobs at Grangemouth last week, just a few miles from Gordon Brown's constituency. But where was the Labour ex-leader? He was sunning himself in Mexico, at a beachside hotel in Acapulco.
"Today, Gordon Brown joked that he is an 'ex-politician', while on another foreign jaunt. But he is still paid more than £65,000 a year to be an MP. Hard-pressed taxpayers are footing the bill."
In recent months Mr Brown has taken part in campaigns over the clean-up of Dalgety Bay and to save the Remploy factory in his constituency.