Former miners' leader Arthur Scargill has lost his fight to have the National Union of Mineworkers (NUM) continue to meet the costs of his London flat for his lifetime.

The NUM had asked Mr Justice Underhill at London's High Court to declare it has no such continuing obligation to 74-year-old Mr Scargill, who was its president for 20 years until July 2002.

The union also successfully disputed Mr Scargill's fuel allowance at his Barnsley home and payment for the preparation of his annual tax return – but not the cost of his security system in Yorkshire.

After the ruling, NUM general secretary Chris Kitchen said it was "regrettable" it had to bring the case.

Mr Scargill said the judgment was "perverse" and flew in the face of all the evidence.

He said: "There can be no doubt that 30 years ago I was given an entitlement to a property by the union and that entitlement continued during my retirement, as it had done for all my predecessors including Lord Gormley and, after he died, his widow Lady Gormley."

On the question of an appeal, he said: "I shall talk to my lawyers about what the best course of action will be, but I think any independent observer will regard this as yet another judgment with the anti-Scargill feeling about it."

Mr Scargill has occupied the Barbican apartment, rented from the Corporation of London, since June 1982 – and the rent and associated expenses were paid by the union until 2011, except for a period between 1985 and 1991 when he met them.

The case was based on documents dating back 30 years and the identification of what terms were agreed between Mr Scargill and NUM representatives.

The judge rejected Mr Scargill's argument that having the union pay the rent on the flat was a replacement for the benefit given to his predecessors – who had houses bought for them in or near London that they were entitled to occupy before and after retirement at a very low rent or very reduced purchased price – and therefore was also a lifetime benefit. He said the original minutes of the national executive committee (NEC) from 1982 did not say this, and neither did the original contract Mr Scargill signed.

When, in 1991, Mr Scargill decided he did not want to take up the benefit of having a house bought for him but instead wanted the union to resume paying the flat rental, he should have gone back to the NEC and sought its authority, he added.

Instead, said the judge, Mr Scargill obtained on his own initiative, and without involving the NEC, legal advice that supported his right to have the union pay for the Barbican flat and told the finance department to resume payments.

"Although Mr Scargill claims otherwise, the process of drafting the contracts and obtaining the legal advice, which should have been arm's-length, was done on his initiative and under his control. None of this was disclosed to the NEC. The legal advice was based on information provided by him which did not give the full picture; and anyway it could not make up for the absence of NEC authorisation.

"Therefore, when Mr Scargill retired in 2002, the contract which appeared to give him the right to have the union continue to pay the rent on the Barbican flat was not effective because the NEC had never approved his having any such right."