An exclusive interview with the former auditor general

Robert Black's provocative paper, From Good to Great by 2020, for the David Hume Institute (http://www.davidhumeinstitute.com/images/stories/publications/HOP/HOP_96.pdf) was the latest in a series of authoritative warnings by senior apolitical public figures on Scotland's fitness to face the economic realities of the 21st century.

Although its subject overlaps other recent reports such as the Crawford Beveridge Independent Budget Review, and the Commission on the Future Delivery of Public Services, led by the late Campbell Christie, unlike them it was written for an independent think tank rather than the Scottish Government.

In this "constructive challenge", Black called for an "urgent response" to the threats to Scotland's public finances, which included lower public spending levels, questionable quality standards in schools and hospitals, huge backlogs in essential maintenance of the public realm, and the sharply increasing financial obligations aligned to an ageing population.

As well as highlighting the need for MSPs to acquire better knowledge of the complexities of service delivery, his central recommendation was for the establishment of an independent Scottish Commission for Resources and Performance. This body would monitor the costs and benefits of new legislation, and help provide a "safe space" for creative thinking on provide services.

"It is a serious gap in the good government of Scotland," Black wrote, "that this constructive challenge, using the disciplines of economic regulation coupled with performance audit, does not exist in Scotland."

Colin Donald, business editor of the Sunday Herald: Are you pleased with the reception that your proposals have had so far, most recently in the Scottish Parliament's Local Government Committee last week?

Robert Black: It was encouraging that the MSPs on the committee seemed to be pretty intensely interested in the subject and kept wanting to ask more questions. They thought there were some interesting ideas there. From my own point of view, it's just a case of trying to keep this narrative moving along. It seems to me that people recognise that there are issues here that need to be considered quite seriously now.

You retired as Auditor General last July. What inspired you to issue this forceful challenge to the way public services are delivered and paid for at this point?

During my 12 years as Auditor General, the Scottish Parliament and Scottish Government went through the most benign period of public spending that anyone can remember, with substantial increases every year. My concern is that we are now in a totally different environment and therefore need to be more robust in our constructive challenge of the issues of productivity and performance in the public sector. Secondly, I think that small government, regardless of whether it is a devolved federal system or an independent state, can work very effectively in the future, because the scale of government allows effective joining up between different agencies, and you can see that in Scotland. Therefore we need to fulfil the potential of that and think seriously about aspects of service design and getting more productivity from our public services.

You told the committee you are "passionately interested in the good of Scotland". Are you also fearful about what the future holds for subsequent generations if the problems highlighted in your paper are not addressed?

Not at all. I am not fearful about the future, but I think we need to have our eyes wide open to the financial challenges that are ahead, and some of the very severe pressures that are having an impact on our public services. If we want to aspire to maintain the high quality of our public services, we need a stronger and more effective challenge.

The Independent Budget Review (IBR) led by Crawford Beveridge concludes with a desire to move to a more "outcomes-based" approach to public services. That was published in July 2010 and that pace of change has been glacial since then. What makes you confident that the desired change will happen?

I am in no doubt at all that the IBR was an excellent document, and the key messages in that report are as relevant today as they were at the time of writing. When you lay it alongside the report chaired by the late Campbell Christie on the future of public services, you get a very full agenda for the direction of travel for Scotland's public sector, and I guess my message would be that we need to up the pace of reform in order to really seize the opportunities as well as challenges posed in these documents.

You say in your paper that you were "never convinced that boom and bust was over", and hint at discomfort with the cultural effects of ever-rising budgetary settlements in Scotland. Do you feel some culpability for failing to sound the warning bell loudly enough?

I don't think any of us foresaw the dramatic way in which the years of real growth hit the buffers. I'm sure there were very many thoughtful people who took the view that economic cycles would reassert the normal pattern, and it was self-evident that the rapid rate of growth of public spending year-on-year couldn't continue indefinitely. The concern which certainly I had at the time was that new expenditure commitments were being taken on using the extra margin of resources without thinking hard enough about the long-term consequences of that and the sustainability of those services. It may sound defensive but in an auditor's institution there is only so much one can do about commenting on future trends, because auditing is essentially a retrospective activity, and I was debarred from actively engaging in policy matters.

You have been in public life for a long time, and have a good sense of what is practically achievable. What actual effect do you hope and expect your intervention will have?

I am greatly encouraged by the attention being given to the whole issue of public service redesign by the parliament and in particular the Local Government and Regeneration Committee. I think there is undoubtedly a general recognition that things need to change, and if my small contribution helps to inform and encourage that debate then I will feel it's worthwhile.

Do you feel there a genuine appetite for change and improvement within the civil service in Scotland?

I think that within the whole of the public sector at senior management level there is a recognition that change has got to come and a willingness to make it happen. I wouldn't want to single out the civil service.

Do you think that the current constitutional debate is preventing us from giving proper attention to the issues surrounding Scotland's budgetary challenge?

Whatever the outcome of the independence referendum, Scottish public sector and wider society will be facing the same challenges we do today. And therefore my plea is for a balance in our political debate and our policy thinking between the independence discussions and some real focused discussion about the future shape of the public sector against the financial background we are facing and the growing pressures on public services, not least from the ageing population.

There seems to be a discrepancy between the detailed but distinctly underwhelming picture of the performance of services to the Scottish public and your frequent praise of the performance of devolved Scottish institutions and their leadership cadre.

I think it's very important not to oversimplify the narrative of past performance. If we stand back and take a birds eye look against the Scottish Government's outcome targets, on many measures we are seeing improvements in Scotland, no doubt about that. But there also areas where we need to really look hard at areas where the performance is not so great, and I do have a concern that we are not looking hard enough at cost, productivity, quality and outcomes together. In other words we tend to separate things out. I advocated for many years that there should be a rolling programme of major reviews of areas of areas of public expenditure over the lifetime of a parliament which would provide a good basis of robust evidence-based analysis that could be used in the budgetary progress. And we haven't made any progress on that, unfortunately.

Your DHI paper suggests a high level of tolerance in Scotland for "coasting and underperforming" public service providers, strongly defended by vested interests, and supported in the media. Is this phenomenon particularly Scottish, or does it exist elsewhere in your experience.

I'm not in a position to compare or contrast with what's happening in England because I know no more than what we all read in the newspapers. Im not competent to go there, I'm afraid. One thing I will say is that because of the size of Scotland it is possible to have a good dialogue for example between the trades unions and the professional bodies, or representatives of the citizens and users the government on the issues of the day. I do think there is a challenge and indeed an opportunity for professional bodies and trade union bodies to come to the table and talk seriously about these things. You can see that beginning to happen certainly in the health service south of the border where there have been media reports just recently of very senior people, clinicians as well as managers, talking about the need to redesign the arrangements for acute care in the health service. That's the sort of dialogue we need to be encouraging in Scotland also.

How would you like to see a SCRP constituted in terms of personnel?

This is no more than an outline of an idea at present. Clearly it would be for others to think this through and determine how to take it forward. However I would encourage people to have a look at the website of the Australian Productivity Commission (http://www.pc.gov.au/ ). It is constituted under its own legislation, independent of the Australian Government, it makes its reports public and its analysis is undertaken completely independent of government. Its commissioners are people of very high standing and experience across the whole of society and the economy, and that is the sort of entity we should be looking to create. Is there not an important difference between Scotland and a large country like Australia, in that everyone in senior positions knows everybody and it's a little bit difficult to have real objectivity?

I think it's a risk, on the other hand I do think of an advantage in the scale of Scotland because it is that much easier to have deep, well-grounded conversations about issues of performance because the – excuse the jargon – the delivery chains are much shorter and there are much shorter lines of communication, let's say between the Scottish Parliament and local points of service delivery because of the size of Scotland, and some very good dialogue does take place. When I was the Auditor General I never felt at all inhibited about having challenging conversations in all walks of public life and many people actually welcome it. I am very conscious that among the many chief executives and chairs of public bodies who I have encountered, they are very up for the challenge of service redesign, and improving productivity.

You memorably describe quality as the "dark matter" of productivity measurement – meaning we know it's there but we can't measure it. Why do you think the data that would allow us to assess quality remains poor?

The issue of quality data in the public sector has been an intractable problem for years. And it reflects the nature of public service. How does one measure the productivity of a surgeon in a hospital? In any field of surgery you care to mention, does a good surgeon undertake many operations, or does a good surgeon undertake fewer operations and get better outcomes, reducing the requirement for patients to be readmitted to hospital? Now, it's very difficult to get reliable data on that, but it doesn't mean it's impossible. Evaluation and evidence doesn't always have to be the hard numbers, we can use informed story-telling and a professional judgment in these matters, but [currently] there is no place where all of that comes together, and where we can look at the hard data and try and work out what meaning and significance should be attached to it. This needs to be done in a public space, so that everyone can recognise what the issues are.

How high on the list of Scotland's outstanding problems do you place this lack of a challenge function?

I think it's a very significant issue, particularly if we are moving towards a system with mergers of [public] bodies. In the further education sector we've seen colleges come together, and in the police and fire service we have seen single bodies being created for the whole of Scotland. I think alongside that [process], we have got to think quite seriously about how we scrutinise and analyse the efficiency and effectiveness of these kind of [merged] bodies, not only retrospectively but also looking at the options for their future development.

You claim in your DHI paper that "the nature of the political debate in Scotland has not yet fully come to terms with the new challenges with which the public sector is facing" and that it "often seems out of time". Why do you think Scotland became so detached from reality?

I hear the question but wouldn't want that phrase ["detached from reality"] attributed to me I have to say. It's just over a decade since the devolved system of government came into being. There have been many excellent achievements over that time. A lot of the legislation passed by the Scottish Parliament has been very good. There is no doubt that it took several years for the system to bed in, but we are now in a significantly different environment for public spending and also the parliament and the devolved system of government has had a chance to mature. Therefore I think the time is now right to take these tough issues head on.

Are you essentially calling for a rebooting of the way the country thinks about and pays for its public services?

I think rebooting is rather a strong phrase actually as it implies that you shut things down and open up and start again. Now a lot of what is going on in Scotland is actually very good and along the right lines. As I have said many times, the challenge is to make it more effective in the new era of declining resources and growing demands.

Is it more a case of taking what's good and developing it and dropping what is bad? You say that "the challenges are immediate and require an urgent response, we cannot afford to place this agenda to one side until after [the independence referendum]". This does seem to call for a fairly drastic change in approach.

I do think the issue of service redesign has to be taken very seriously and we need to up the pace of change around some of these issues.

Until July 2012, Dr Robert Black, CBE, was the first Auditor General for Scotland, a post which he had held since 2000. Previously he was chief executive of the Accounts Commission for Scotland. He was a local authority chief executive for 10 years, first with Stirling District Council (1985-90) and then with Tayside Regional Council (1990-1995). His earlier career was in policy planning and research with Strathclyde Regional Council, Glasgow City Council and Nottinghamshire County Council.