Complexity Theory and Policymaking

This blog will appear shortly on the CIPFA Scotland website to run alongside my talk at their annual conference. I will then amend the blog after discussions during the two days (informing my round-up talk on the second day).  The CIPFA site will also host comments.  Some discussions can be followed on Twitter #cipfascotlandconference

CIPFA Scotland Annual Conference 2013:  ‘Edge of chaos: leading to new possibilities’

Paul Cairney, Professor of Politics and Public Policy, University of Stirling p.a.cairney@stir.ac.uk

Complexity Theory and Complex Adaptive Systems

The theme running through this year’s conference is: ‘adaptive system thinking … how adaptive public finance and its finance function and people are’.  There are three key sources of background for the discussions:

1. Economic crisis. The UK and devolved governments are addressing a new age of austerity associated with economic crisis, rising government debt and plans to reduce public spending.  The reductions may be so significant, in some areas, that they prompt a complete rethink of the way that governments plan and deliver services (or new ways to identify, understand and seek to solve policy problems). In some areas, governments may simply cut budgets and encourage other bodies to decide how to adapt.
2. Targets and rules.  How can targets and rules ensure the delivery of public services?  This is a perennial topic in the study of government. For CIPFA in particular, the issue may relate to the need to follow rules to ensure that the spending of public money is properly accounted for.  We may ask ourselves if current rules and targets produce the right sort of behaviour or if they encourage rigid rule following (or game playing) which might cause unintended consequences. 
3. Organisational Change in Scotland. Since 2007, the Scottish Government has made some important changes to the Scottish public sector – centralising some areas (such as the police and fire services) and decentralising others (such as local authorities). Its new relationship with local authorities has the potential to place more responsibility on local actors to identify and solve problems rather than implement policies made from the ‘top’.  The Scottish Government’s approach may also be contrasted with that of the UK Government (associated with a more extensive and punitive targets/ inspection regime).

A combination of all three elements draws our attention to the importance of adapting and responding to crises, perhaps by taking a flexible approach to existing rules to ensure that they are not overtaken by events and new problems. Such decisions may increasingly be taken at a local level, often without central government direction; they may fall increasingly to particular branches or professions within larger governmental organisations, often without explicit or detailed government direction.  This is a situation that may be particularly worrying to a profession built on demonstrating financial accountability to the public through government and Parliament.

In this context, one aim of the conference is to find the right balance between following proper rules properly and adopting a flexible approach: to use discretion to secure outcomes for which policies may have been designed; and, to interpret rules in the right way to secure the right outcomes.  This process is subject to uncertainty; it is difficult to know how to use rules flexibly.  Indeed, even the language we might use to describe it – such as ‘stretching’ rules – implies that it is inappropriate.  So, while most of the sessions are devoted to how accountants and other actors in the public sector might become more creative in their contribution to the use of public money, this blog (and my first brief lecture) is devoted simply to the question: how can we justify the flexible use of government rules?   

If put bluntly, the main message that we could take from the policymaking literature is that:

1. No government can control the public sector, its outputs or the subsequent outcomes.
2. Many governments exacerbate this problem by imposing a large number of too-rigid targets backed up by a punitive inspection regime, producing unintended consequences.
3. Giving more discretion to local public sector employees allows them to adapt to local circumstances in a way that central governments cannot anticipate.

A classic discussion of this problem can be linked to key texts in the policymaking literature, including:

• Lipsky’s idea of ‘street level bureaucracy’. He suggests that there are so many targets, rules and laws that no public agency or official can be reasonably expected to fulfil them all.  In fact, many may be too vague or even contradictory, requiring ‘street level bureaucrats’ to choose some over others.  The potential irony is that the cumulative pressure from more central government rules and targets effectively provides implementers with a greater degree of freedom to manage their budgets and day-to-day activities.
• Hjern’s focus on intra-departmental conflict, when central government departments pursue programmes with competing aims, and interdependence, when policies are implemented by multiple organizations. Programmes are implemented through ‘implementation structures’ where ‘parts of many public and private organizations cooperate in the implementation of a programme’. Although national governments create the overall framework of regulations and resources, and there are ‘administrative imperatives’ behind the legislation authorizing a programme, the main shaping of policy takes place at local levels.
• The more recent focus on ‘governance’ as an alternative to the idea of ‘government’ (not to be confused with a discussion of ‘corporate’ or ‘good’ governance).  While such problems of central government control have prompted governments in the past to embrace New Public Management (NPM) and seek to impose order through hierarchy and targeting, local implementation networks (with members from the public, third and private sectors) may not be amenable to such direct control

A more recent contribution comes from the modern study of complexity theory.  As Marco Thiel’s discussion on Friday suggests, complexity theory has been applied to an incredibly wide range of activity, from the swarming behaviour of bees, the weather and the function of the brain, to social and political systems.  The argument is that all such systems have common properties or are subject to the same arguments, including:

• A complex system is greater than the sum of its parts; those parts are interdependent - elements interact with each other, share information and combine to produce systemic behaviour.
• Some attempts to influence complex systems are dampened (negative feedback) while others are amplified (positive feedback). Small actions can have large effects and large actions can have small effects.
• Complex systems are particularly sensitive to initial conditions that produce a long-term momentum or ‘path dependence’.
• They exhibit ‘emergence’, or behaviour that results from the interaction between elements at a local level rather than central direction.  
• They may contain ‘strange attractors’ or demonstrate extended regularities of behaviour which may be interrupted by short bursts of change.

As you might expect from (what seems like) a theory of all things, the language is vague and needs some interpretation in each field. In the policymaking field, the identification of a complex system is often used to make the following suggestions:

• Law-like behaviour is difficult to identify – so a policy that was successful in one context may not have the same effect in another.
• Policymaking systems are difficult to control; policy makers should not be surprised when their policy interventions do not have the desired effect.
• Policy makers in the UK have been too driven by the idea of order, maintaining rigid hierarchies and producing top-down, centrally driven policy strategies.  An attachment to performance indicators, to monitor and control local actors, may simply result in policy failure and demoralised policymakers.
• Policymaking systems or their environments change quickly. Therefore, organisations must adapt quickly and not rely on a single policy strategy.

On this basis, there is a tendency in the literature to encourage the delegation of decision-making to local actors:

1. Rely less on central government driven targets, in favour of giving local organisations more freedom to learn from their experience and adapt to their rapidly-changing environment.
2. To deal with uncertainty and change, encourage trial-and-error projects, or pilots, that can provide lessons, or be adopted or rejected, relatively quickly.
3. Encourage better ways to deal with alleged failure by treating ‘errors’ as sources of learning (rather than a means to punish organisations) or setting more realistic parameters for success/ failure.
4. Encourage a greater understanding, within the public sector, of the implications of complex systems and terms such as ‘emergence’ or ‘feedback loops’. 

In other words, this literature, when applied to policymaking, tends to encourage a movement from centrally driven targets, rules and performance indicators towards a more flexible understanding of rules and targets by local actors who are more able to understand and adapt to rapidly-changing local circumstances.

Of course, you may still end up feeling that the advice is a bit vague (and perhaps not completely convincing), but it provides a starting point for discussion, followed by various conference presentations on how to use, creatively or flexibly, new forms of discretion in the public sector.  There will be the usual forums for participation – including Q&A sessions after lectures and smaller seminar groups – as well as a live twitter feed throughout (#cipfascotlandconference). 

 

For more detail, and further references, see:

Cairney, P. (2012) ‘Complexity Theory in Political Science and Public Policy’, Political Studies Review, 10, 346-58 http://onlinelibrary.wiley.com/doi/10.1111/j.1478-9302.2012.00270.x/abstract (or email Paul for a copy)

See also Robert Geyer on Complexity and the Stacey Diagram - http://vimeo.com/25979052 

 

Should You Play the Citation Game?

 

This blog asks the simple but super-loaded question to academics: should you try, as much as possible, to make sure that proxy measures of your performance put you in a good light?

There are many, generally convincing, reasons to avoid getting sucked into metrics as proxy measures of performance.  Waiting times/ lists as a measure of health service performance is perhaps the best example, although league tables of school exam results give it a run for its money.  In higher education, we have the often-criticised reputation ranking for Universities, alongside other league tables on research performance and teaching reputations.  The professional academic equivalent may be a department’s record of research achievement, measured in terms of grant income and publication output according to external assessment (and the reputation of the journals and book publishers).

It may also relate to the growth in the assessment of individuals, through the income and output routes, but also using metrics on things like citations.  This focus on citations can relate to, for example, the ‘impact factor’ of the journal as a guide for submissions – a proxy that appears to have become almost taken for granted in many Universities/ disciplines, but one that is under increasing criticism (for example, Google ‘lse blog impact factor’). It can also relate to an individual’s h-index, which relates to the number of publications cited a number of times (for example, an h of 16 means that 16 of author’s outputs have been cited at least 16 times). 

We can come up with an incredible number of good reasons to question the value of individual citations measures, including:

1. People may be cited as examples of rubbish academia, or as methods/ approaches to avoid.
2. People may be able to inflate their citation scores by engaging in self-citation and (perhaps more importantly) group-citation, in which a group decides (explicitly or implicitly) to cite each other regularly.
3. Some disciplines and sub-fields will generate a smaller amount of citations (such as in large parts of historical research which follow a long-research-low-output-but-high-quality model) and some will produce a higher amount, such as the life and natural sciences with a tendency to low word count, high output and co-authored papers.

However, I still think you should, as sensibly as possible, play the game, for the following reasons:

1. You may get the impression from the (truly depressing) THE, and from departmental discussions, that we are all against these measures; that we have worked out their weaknesses and everyone else has too.  This would be a mistake.  Many people secretly (and some people openly) think that they are decent measures and act accordingly. For example, there are people like me who make sympathetic noises in discussions, then go off and play the game.
2. More importantly, in my experience, they are favoured most by senior managers (and/ or important disciplines in universities).  The general point is that: (a) policymakers always work in an environment of uncertainty and ambiguity, with limited measures of performance and a limited ability to make sense of the available information; and, (b) they still have to make choices.  Specifically, they often make what we think are the wrong choices – but they make such decisions from a different vantage point.
3. In my experience of promotions panels, someone’s h is discussed and debated quite naturally in the natural, physical and life sciences.  In fact, on a panel of about 10, you might have 4 people with their laptops out, debating the size and significance of applicants’ scores (with, for example, an h of 20 the figure they hope for in a professor).  It may not be the deciding factor, but it really counts. 
4. At least one of those 4 people is likely to be a senior manager.    This is crucial for a discussion of the use of h in the arts, humanities and social sciences.  This is where you get the biggest opposition to h scores and you might have panels that reject them as measures of performance.  However, that senior manager may also be on the panel, applying the same mindset.  More importantly, they might not hold as much sway in this committee, but the recommendations may then go to a more senior committee on which they sit.
5. The same might be said for appointments panels.  You may not be aware of the importance of the h, but there could be at least one person sitting there who has done the background work.  Or, that person is waiting for the recommendations and may use h as a way to argue against them.

So, if I was asked by a new colleague about the importance of h, I could not simply recommend that they ignore it and just focus on good quality research (which is a bit misleading anyway – I am with Silvia on this one).  Instead I would recommend four things:

1. Get on top of the way that your h is measured.  Senior managers tend to use something like ‘Publish or Perish’ which, in my experience, can bring your h down (particularly with books). Or, people in meetings start debating the right number instead of your application. Instead, I began to put on my promotion and application documents a link to my Google Scholar page, which is the list of my citations that I think is the most accurate. 
2. Present a convincing narrative of your h, not just in terms of how misleading the measure is.  Often, this is about pointing out that, for example, your articles have been published very recently (too recent to take off) or, for example, that your citation rates for particular journals are higher than the 5-year median (the measure now used by Google scholar to rank journals). This would sit alongside the usual narrative on your outputs and the quality of journals. My preference is to focus on the h trajectory: it is this high now, which means that we can reasonably expect it to get this high in 5 years.
3. Don’t be a self-flagellating non-citer of your own work for the sake of principle.  If it makes sense to cite your own work, do it.  Reviewers and editors will soon tell you if it is too much.   
4. We operate within a highly-critical profession in which constant rejection and criticism is something that we have to put up with to get ahead.  So, enjoy the occasional pat on the back that Google Scholar gives you.  Sign up for the service that allows you to receive an email when you are next cited – it is one of the very few boosts to the ego on which academics can rely.