LIKE all the best mysteries, it started with a noise in the night.

It came from 33 Horseferry Road in Vauxhall, central London, headquarters of the Department for Transport (DfT), which put out a startling announcement around midnight on Tuesday.

It said that, in view of certain irregularities with the process for renewing the rail franchise for the West Coast mainline between London and Glasgow, the choice of Aberdeen-based transport company FirstGroup was revoked. Instead, the DfT was going to set up two inquiries, one into the West Coast process and one into the whole system of rail franchising.

The DfT proposed to pay £40 million to the shortlisted bidders for the costs of preparing bids – the others being Virgin Trains, Abellio and Keolis – and that it would no longer contest its impending judicial review battle with Virgin over the award.

It said it was also putting the brakes on the next three franchise renewals, for Great Western, Essex Thameside and London Thameslink, in what is expected to lead to delays of at least a year in awarding them. The ScotRail franchise, due to be awarded by Transport Scotland to start in 2014, appears unaffected. Westminster Transport Secretary Patrick McLoughlin – only weeks in the job following the Cabinet reshuffle – called the matter "deeply regrettable".

Early on Wednesday there came an update on some of the characters in this Cluedo-esque drama. Three civil servants who had been involved with the bids had been suspended. Their identities were not revealed, though two were understood to be fairly senior.

Back on August 15, the DfT announced that, in line with leaked reports, FirstGroup had been successful in its attempt to take over the West Coast line.

Sir Richard Branson was enraged. Virgin Trains had run the franchise for 14 years, doubling passenger numbers to around 30 million and clocking up some of the highest approval ratings in the industry. He had lost three rail franchise bids in the past – and this time, Branson decided to go on the offensive. He claimed the First bid had been excessively optimistic and that the DfT had not followed its own rules over the bid process – a charge strenuously denied by then transport secretary Justine Greening and former rail minister, Theresa Villiers.

Branson claimed that Government had been taken in by the £13 billion premiums that First was proposing to pay over the maximum 15-year life of the franchise, compared with Virgin's £11bn. But the First bid was not the lucrative payday for the taxpayer it seemed, Branson insisted. Like GNER and then National Express on the East Coast line, and like First itself on the Great Western line in the south of England, Branson said the Aberdeen group would miss its revenue targets and end up handing back the franchise early.

Most people thought this was sour grapes. Eyes rolled when Branson confirmed towards the end of August that Virgin would be seeking a judicial review, having been rebuffed by the Government in its requests for full details of the rationale for the decision. When he appeared before the Transport Select Committee in early September with three executives from Virgin Trains and Perth-based Stagecoach (which owns 49% of the Virgin Trains business), calling the FirstGroup bid "preposterous" and "taking the system for a ride", eyes rolled some more.

The row boils down to the money – the subordinated loan facility, or SLF – which the chosen operator has to put up as a bond, to be forfeited if it walks away early from the franchise. This is determined by how risky the DfT thinks the bid is.

The officials look at each bid and cut the size of the proposed premium in proportion to how sceptical they are about the bid's assumptions about the cost of running the business and the money it will generate. Then they use a formula to get from this risk-adjusted figure to the level of the SLF. The first part is a judgment call, where it is possible for different assessors to reach different conclusions; the second part is more fixed.

Basically, the system is geared to choosing the highest bid, so long as it doesn't look so risky that it will end up with the franchise being returned early. At the end of the process, the DfT asks its preferred bidder if they are willing to sign the contract based on the level of the SLF. If they decline, the next-best applicant receives an offer instead.

In the case of the West Coast franchise, the department calculated that the SLF for FirstGroup should be £190m, around five times higher than what Virgin would have been required to pay had it been successful. This obviously meant the DfT viewed First's bid as much riskier than Virgin's – but Branson and his people argued before the select committee that £600m would have been the correct figure based on the rules of the process; a figure that heavily-indebted First could probably not afford to commit to. Branson's team said that First would not achieve its 10.4%-per-year revenue growth forecast, which required raising passenger numbers to about 66 million. Consequently it would walk away early, depriving the taxpayer of hefty payments since it was proposing to pay almost half the premiums in the final three years of the contract.

Virgin conceded it had achieved 10.2% growth a year since 1997, but said that was on the back of a £9bn one-off upgrade programme that could not be repeated. In its own bid, Virgin had proposed revenue growth of 8.5% a year, to about 50 million passengers. Virgin also offered an £11bn premium but proposed to pay much more than First in the first few years, taking the view that the business would be heavily affected by the major upgrades to London Euston once the High Speed Rail project construction starts in 2019.

FirstGroup chief executive Tim O'Toole also appeared before the select committee, telling the members in his American drawl that Virgin did not have enough information to evaluate his bid. He said "we think we know" how Branson's people had come to the £600m figure, drawing on the fact that First and the Government broadly agreed on the level of the SLF to support his case. O'Toole said the First plan was to boost Virgin's overall seat occupation from 35% to more like 45%, in line with other First franchises.

Virgin's push for a judicial review of the bid was set to come before the High Court on October 17. The Government's decision to throw in the towel came 48 hours ahead of its deadline to submit the outline of its defence and various documents related to the decision process to Virgin's lawyers.

The DfT claimed it was backing down because it had discovered "technical flaws" in the process. But few commentators believe it is this simple. As well as the £40m compensation already mentioned, the Government will have to pay for the two inquiries, plus re-running the West Coast tender and foregoing premium payments on the four other affected franchises until they can move on with the process. Most observers think this will cost around of £100m – not the sort of price tag you accept over a few slips on the spreadsheet. Even Government spin doctors were conceding later in the week that there were "more wide-ranging" issues at play.

SO what is going on? The Government's detractors want answers to some awkward questions. One is: If the DfT was happy with the process, why – on July 27, two weeks before the West Coast announcement – did it announce that the multiple for calculating the SLF would be raised by 66% for the forthcoming Great Western franchise? Had this been applied for the West Coast, it would have added £125m to First's SLF. The Government has said the two franchises cannot be directly compared, but has yet to give a compelling explanation for the change.

Question two is why was O'Toole able to say that First had come to the same final number as the Government? Does this mean the Government had shared information with First that it hadn't shared with other bidders? Or, as rail expert Roger Ford suggests, could this mean the DfT relied on projections or assumptions made by First? The Government has said First followed the process correctly. First last week said it stood by its application.

Thirdly, should Atkins, which provided technical support to the Government on the evaluation process, have noticed anything wrong? Atkins says it had no audit role and insists there were no quality issues around its advice.

Fourthly, why has the Government chosen Centrica chief Sam Laidlaw, a member of its own advisory board, to conduct the West Coast review?

One answer being mooted is that the civil servants were under pressure to accept the highest bid with too little regard for risk. Another claim is that the DfT officials have a longstanding grudge against Branson.

Ford says: "I always go for cock-up over conspiracy; it's always more likely to be human error than anything else. It looks like gross incompetence."

Whatever the case, the result is the same: bad news for rail privatisation. One transport expert suggests the Government is worried it can't defend its future assessments of the risks of each bid against legal challenges from thwarted rivals, in which case this could be to the franchising process what the 2000 Hatfield rail crash was to the handling of rail infrastructure.

McLoughlin and his permanent secretary will appear before the select committee at the end of this month to answer questions. Days after, the review of the West Coast process is supposed to complete, the results of which the Government will be pressed to make public. The wider review will take months, rather than weeks, to complete.

Candlestick or lead piping, kitchen or library, whatever the exact nature of the transgression and the motive around the decision to give FirstGroup the West Coast Mainline franchise, there will be much more to come out before this matter rests.