An uncle, working in the 1930s for the publishers Blackwoods, told them that if they published a book as boringly titled as Forty Years a Public Servant he would resign. They did; and he did.

It was a terrible book title. But 40 years in the public service is a sobering background for self-criticism after the recent public enquiries, especially the horrifying Infected Blood Inquiry. None have been close to my own career. All have raised questions about how I or my colleagues might have behaved when bad things were taking place, or when the first questions were being raised about them.

We saw ourselves as honest. For almost all my career, we reckoned our political masters were honest too, though we sometimes disagreed with them. But we were susceptible to groupthink. Whitehall was its own world, far less diverse socially and ethnically than today. Departments were semi-independent institutions, proud of their own identities.

This brought strong team co-operation, loyalty, commitment. But we did not easily absorb views questioning the departmental line, even from colleagues in other departments. Above all, most civil servants didn’t meet outsiders – journalists, academics, activists – with interests in the same field. When we did it was to support ministers when such people came to complain about something. This didn’t stimulate reflection of policy options.

The Herald:

One exception stands out, because it was rare. Preparing legislation on employment discrimination in Northern Ireland, someone convened a weekend gathering of relevant officials, including junior ones like me, and a range of academics and concerned organisations. It was fascinating; and massively improved the legislation.

I switched later to the Foreign Office. It certainly wasn’t immune from groupthink. But serving in posts abroad, a big, enjoyable difference was being expected to meet people with a wide range of opinions to learn their views. Theoretically, this openness continued back in Whitehall: we were encouraged to attend think-tank seminars. In practice, daily pressures made this almost impossible.

In my final civil service job, in the Cabinet Office in London, I was impressed by the far greater diversity among colleagues – not just ethnically and socially, but people of different ages mixed far more than before. One excellent initiative devised by a lively 25-year-old and warmly endorsed by her bosses, was for staff, mainly young, to invite a stimulating speaker to talk at Wednesday lunchtimes. Often they were colleagues’ recent teachers, professors and lecturers, maybe eminent, sometimes quirky, usually challenging.

They made us think. A year or so after I left, I was saddened to hear that ministers had ordered that the publications and social media records of anyone invited to address officials had to be checked: no-one who had criticised the Government should be invited. I hope this has been rescinded, or ignored.

Read more: The Good Friday Agreement: The untold story of the historic peace deal

Read more: Attacking Houthis puts UK into villain combo with Americans

It is a tricky balance. A civil service needs enough separate identity to nurture the sense of public service which the private sector has no reason to develop. It must also be open enough, through exchanges and regular outside contacts, to have perspectives from which to judge itself and its own practices. Today’s more open civil service is probably more questioning than officials were in the recent inquiries. Confident, loyal independence from ministers is also essential.

The recent enquiries teach us history lessons on their individual subjects. But they all provide lessons for the future on managing our official institutions.

George Fergusson was Secretary of the Defence and Overseas Committee in the UK Cabinet Secretariat, 2003-06