'Test, Learn, Adapt' - thoughts on the use of RCTs in social policy

Today, the Cabinet Office's 'nudge unit' has produced a new document advocating the greater use of randomised control trials (RCTs) in the policy-making process.  With 'Bad Science' writer Ben Goldacre on the team, it's received a fair bit of press, so I thought I'd review it and offer my own opinion on their key argument.

The argument
The thesis behind the paper is simple.  RCTs have been used in a range of other fields for some time - medicine, international development and business for example.  However, they have not been adapted to the policy-making process so enthusiastically.  Despite the early New Labour focus on 'what counts is what works', governments maintain the habit of ignoring scientific evidence on policy interventions.  The authors argue that this should change: if it does, we might get better policies and at a significantly cheaper cost.

Sounds too good to be true?
That's because it is.  In my view, the report rehashes old arguments about the use of 'scientific' evidence  in social policy.  The authors appear to come from a very one-sided scientific perspective: unaware of the long-established difficulties - particularly in social policy - of the interaction between science and society, politics and morality.  In fact, in their discussion of the 'myths' about RCTs, the authors focus exclusively on practical or economic objections, rather than any philosophical doubt about using scientific evidence to decide on policies.

The authors offer a deeply tempting and seductive view: that we can determine what policies we choose by scientific methods, removing age-old ideological debates about the Good Society once and for all.  Not only is it wrong to equate the objectives of social policy to other institutions, like medicine, but to my mind it seems quite undesirable.  Sure, we can use scientific methods like RCTs to choose between different policy interventions.  But first we have to decide what type of interventions we want in the first place.  And that requires debates and deliberations that science simply cannot answer.

Is the UK alone in turning against the welfare state?

In a paper I recently had published in the journal Social Policy and Society,  I looked at longitudinal data from the British Social Attitudes (BSA) to explore how attitudes to the welfare state - and in particular the unemployed - had changed over the past few decades.  The data show a strong and profound shift in attitudes against welfare.  Whilst once a majority of us tended to support more spending on benefits, there is now a solid consensus against extra spending on social security (especially for 'undeserving' groups like the unemployed).

In my paper, I try to offer an explanation for why this happened.  In short, I argue that the growth in anti-welfare attitudes has followed the deep and significant reforms to the welfare system that have taken place since the mid-1990s: stronger conditionality, sanctioning and the growing emphasis on personal responsibility.

However, this is not a particularly easy argument to prove one way or another.  I argue that the main basis supporting this theory is the timing of the attitudinal shift: around 1997/8 (or when New Labour came to power with a promise to change the welfare state).

Is this the only explanation?

An alternative hypothesis is that the UK has not been alone in its changing attitudes to welfare.  For example, it could be argued that anti-welfarism here is part of a broader, global trend in attitudes that come as a function of the shift to post-industrial (and largely neo-liberal) societies in the rich world.

One way to test this argument is to look at comparable, cross-national data on attitudes towards the welfare state.  This is not particularly easy: it's difficult to find attitudinal data that has been collected consistently over time and for a decent sample of countries.  There is one dataset though - the International Social Survey Programe - that might be able to give us some clues.

One question the ISSP asks concerns attitudes towards state generosity for the unemployed.  The question is:

"The Government should provide a decent standard of living for the unemployed" Do you strongly agree/agree/neither/disagree/strongly disagree?


This question was asked to respondents in five countries in both 1987 and 2010: Austria, Australia, Germany, the UK and the US.  This is a small sample but an interesting one: it contains three countries from 'liberal' welfare regime types (Australia, the UK and the US) and two from 'conservative' ones (Austria and Germany).  Comparing these five countries for the two time-points should allow us to see if there are any similarities (or differences) between the regime types in terms of changing attitudes towards supporting the unemployed.  It should also suggest whether shifting attitudes are universal, rather than a sole feature of British society.

A universal shift?


The results are quite surprising.  Here is a table that shows the proportion of respondents stating that they either 'strongly agree' or 'agree' that the Government should provide a 'decent standard of living for the unemployed':

A quick glance should tell you that only the UK and Germany have seen a drop in support for supporting the unemployed.  Indeed, even the US has seen a (slight) increase in agreement with decent public support the unemployed.  Here's a graph showing the percentage change between 1987 and 2010:

This kind of data is by no means conclusive: it's based on just one question for a sample of five countries.  But it does suggest two things.  First, that the shift against welfare in the UK - evidenced in domestic surveys like the BSA - is also evident in global, comparative surveys.  Second - and perhaps most importantly - other countries have seemingly had completely different experiences.

This is important.  In Britain, it's often implied by politicians and reformists that welfare retrenchment is inevitable: that it comes part and parcel of a new kind of economy and a changing society.  According to this theory however, other countries should also be undergoing the same kinds of cultural shifts.  The data shown above suggest the picture is more complex.  And that the future of the welfare state is much more uncertain than many of us think.

Welfare: it's not just how much you spend, it's what you spend it on

In an earlier post, I argued that in a time of high unemployment, governments have attached higher levels of importance to so-called 'activation' policies (ALMPS or welfare-to-work measures) that aim to boost and speed up people's return to work.  The data I showed demonstrated how, in comparative terms, the UK spends a small proportion of its national wealth on such policies.  In 2009 for example, we spent just 0.4 per cent of GDP on ALMPs.  The comparable figure for Denmark is nearly 1.2 per cent.

However, this paper by Giuliano Bonoli makes a crucial point: it's not just how much you spend, it's what you spend it on.  In short, Bonoli argues that there are four different 'policies within policies' when it comes to welfare-to-work:

• Occupation: or 'keeping people busy' (e.g. job creation schemes, community programmes)
• Incentive reinforcenment: or 'carrots and sticks' (e.g. sanctions, conditionality, tax credits)
• Employment assistance: or 'jobseeking help' (e.g. placement advice, counselling, personalisation)
• Upskilling: or 'training' (e.g. vocational courses)

Using data from the OECD, Bonoli compares how the structure of spending varies between countries on three of these policies: employment assistance, occupation ('direct job creation') and upskilling ('training'). Differentiating between such policies is important; most of us would agree, I think, that a more effective and fairer approach would be one that combined different elements of all three 'active' components.

The results are shown in the table below, taken from Bonoli's paper.  What the table shows is that not only does the UK spend much less than other countries when it comes to welfare-to-work, but it spends most of its budget on 'employment assistance': a policy basically limited to trying to better place unemployed people within existing jobs.

As is clear, other countries - most notably Denmark, Sweden and France - spend almost half of their welfare-to-work budget on training.  In the UK, the equivalent figure seems almost non-existent: indeed, the Conservative governments of the 1980s appeared to spend far more on training than either New Labour or the Coalition.

In times of high unemployment, there is only so far 'employment assistance' programmes - such as those that the UK government is hugely fond of - can go.  All such programmes attempt to do is make the 'matching' process faster and more efficient; they do nothing to create jobs or genuinely improve the mismatch between labour supply and demand.  Training programmes, alternatively, are capable of doing both of these things.  It's high time the government reassessed its welfare-to-work strategy to put training at its heart.

Welfare should reward contribution

A copy of this article originally appeared in Shifting Grounds 

Academics in social policy like to talk about three different types of ‘welfare state regime’. At one end of the spectrum we have the Nordic regimes: characterised by generous, universal benefits that are paid out as of right. At the other end we have the liberal regimes, where benefits are low (more or less subsistence level), means-tested and highly conditional. In between are the continental systems, where the contributory principle rules the roost.

The conundrum that Labour finds itself in is that of an increasingly liberal British welfare state. For decades the UK was more of a hybrid system, with distinct elements of all three principles: means-testing, contribution and universality. Over time though, liberal principles have taken over the welfare state and, perhaps inevitably, it is the Conservatives who are now seen as the party of welfare.

Labour thus faces two options on its welfare policy. The first is to keep ceding ground to the Coalition. This involves challenging the technical details of welfare reforms but accepting them in principle. This is the position Labour often seems to adopt and, as opinion polls show, it isn’t working.

The second option is to go beyond the liberal model of means-testing, low benefits and high conditionality. Indeed, many Labour members would like to jump straight from liberal welfare to social democratic welfare; where entitlement is universal and poverty minimal. For complex political, economic and cultural reasons, this is not going to happen – at least not overnight.

This means that if Labour wants to change – and strengthen – the welfare state it must look to ideas from elsewhere. This is where the continental system of welfare can help, with its prioritising of the contributory principle. Ed Miliband and Liam Byrne, among others on the Labour front benches, have recently shown their willingness to make it the basis of future policy. Now we need to put meat on the bones.

There are two imperatives for Labour going forward: in considering each, it becomes clear that the problem with Britsh welfare is not ‘something-for-nothing’ but ‘nothing-for-something’. The state needs to do more to recognise people’s contribution and hard work.

First then, Labour should make welfare more attractive to the middle-classes. Middle-class people support welfare when it offers them something back for their contributions. At the moment, the race to the bottom in welfare provision means that people with long records of contributions get the same as those with none. Labour’s first offer on welfare should be to establish a strong contributory principle. If you have worked for twenty years and become unemployed, you should get a decent benefit that ensures you avoid financial hardship. We should still offer basic support to young people and those without contributions, but there should be a clear principle at the heart of the welfare state: when you need it, you will get the support you have paid for.

So establishing a principle of contribution should be a priority for Labour. The next question then is how to apply this to those not covered by social insurance: young people, those with poor work histories or those who have spent time caring.

The policy response from the Coalition towards such groups has been to introduce a raft of compulsory work-based schemes with the threat of removing benefits for those who fail to comply. Participation in such schemes might – or might not – be good for unemployed people, but threats, punishments and coercion do little to strengthen the principles of contribution, reciprocity and responsibility.

Instead of threatening to punish claimants, Labour should offer to reward those who contribute whilst experiencing a spell of unemployment. If you volunteer for community work, find a work experience placement or enrol on a six-week training course, you should be rewarded with a stronger benefit: JSA Plus, for example. In return for making a contribution, you get something back. These two offers are about equal responsibilities throughout the welfare state: or, as Ed Miliband might prefer to say, a ‘something-for-something’ society.

Rather than ‘something-for-nothing’, the problem with welfare is the ‘nothing-for-something’ experience that most people have. Whether that’s paying decades worth of contributions and getting a measly benefit or undertaking compulsory work experience whilst looking for a job, most people sense the state isn’t doing its bit, despite asking them to do theirs. If Labour really wants to mark its ground with a ‘something-for-something’ welfare state, then we must look turn the tables, and look more closely at why so many people feel the state does nothing.

Why Labour should support an 'annual taxpayer statement'

In tomorrow's budget, it is expected that George Osborne will announce plans to issue annual personal tax statements in order to let citizens know precisely where their tax gets spent.  So far, the issue has been set up as one of left versus right, with some conservative commentators clearly believing the policy to be a route to lower taxes.

The theory is that if people know more specifically where their taxes go, they will be less content to pay them.  Indeed, the left-wing tax researcher Richard Murphy was on Radio 5 Live today predicting that this is precisely what will happen.  But such arguments are predicated on a misunderstanding: that 'unpopular' areas of spending, such as unemployment benefits, receive an undesirable proportion of the income tax pie.

On the Guardian's website today, there is a wonderful interactive tool which smashes these myths.  It shows how much an individual spends per day on public services (based on their income).  If we take the average salary - around £25,000 per year - we can see exactly what services cost the average taxpayer on a daily basis.

According to the Guardian's tool, here are the three most expensive services and what they cost per day for the average person:

1. Health - £4.88 per day
2. Education (including universities) - £3.77
3. Pensions - £3.59

And here are the daily costs for three services we might consider much less popular:

1. Unemployment benefits - £0.24
2. Housing -  £0.14
3. The EU - -£0.13 (meaning the average taxpayer gains 13p a day from EU membership)

What does all this tell us?  In my view, it shows us - perhaps unsurprisingly for those on the left - that the things which cost the most money are the services that benefit us all and that we most appreciate: health, education and pensions.  On the other hand, it's clear from data like this that taxpayers are not spending inordinate sums on small numbers of 'undeserving' groups.  The average earner, for example, pays just £7.30 per month to support the ever-swelling ranks of the unemployed.

Conservatives cheerily urging Osborne on with this policy might care to think again.  In future years, taxpayers might receive their annual statement from the Chancellor and think they don't get too bad a deal.  The parent of a secondary school child will see that it costs them just £252 per year to send their children to school; a snip compared to private sector rates.  Equally, we might end up realising how little we spend on needy groups.  Is it fair, for example, that all we contribute towards survivors is 8p a day?  Taken together, the public might end up thinking that the state actually does an efficient job at providing good services, while also believing the public could do a bit more for those in need.  

For years Labour has been shackled by strange and paradoxical public attitudes towards the state.  As the adage goes, the British public want Swedish social services at American rates of tax.  One reason for this might be the misconception that the UK government spends unfairly and unwisely.  Personal tax statements might be one way to combat such misconceptions that have, over a long period of time, been detrimental to British social democracy.  For that reason, Labour should support the Coalition in implementing this measure.

Is workfare all bad? Some evidence from Sweden

The debate about 'workfare' - defined as requiring benefit claimants to participate in some form of work-based activity - has raged this past week.  But the debate has largely been clouded by hyperbole from both   those for and against.  Depending on your position, you are either likely to see the claimants involved as slaves or scroungers.

In times like this it's helpful to have some evidence on the matter at hand.  A key argument against workfare is the humiliation it might involve for participants.  Working for huge, profit-making companies like Tesco for around £1 an hour - whilst those around you get paid 6x as much - is hardly good for self-esteem.

But what if this argument is wrong?  What if participating in work-based schemes is actually good for unemployed people and boosts their well-being, confidence and self-esteem?  This is the question posed in this paper from 2001 by Strandh, who considered whether participation in employment programmes boosted the well-being of participants in Sweden.

The theory that such programmes might be good for unemployed people is based upon arguments from social psychology that work fulfils certain human functions, like time structure, social contacts and collective endeavour.  Consequently, while workfare programmes might not ease the economic woe of unemployment, they may help the unemployed overcome the more psychological damage wrought by being unemployed.

In short, Strandh explored three different types of work-based programme: vocational training, work experience schemes (which take place outside the labour market, e.g. community work) and workplace participation.  It is this latter type of Swedish programme - workplace participation - which is most similar to the Coalition's workfare programme, as it involves unemployed people gaining experience in an ordinary labour market setting.

What Strandh wanted to know was whether participation in these schemes was associated with better well-being for participants (compared to those who didn't participate).  The results are quite surprising - especially given the furore of the past week.  In short, when controlling for a range of other factors, Strandh found that participation in work-based programmes was significantly associated with higher well-being amongst unemployed participants.


Further - and this is where it gets interesting - out of the three different types of programme Strandh looked at, he found that the programme with the strongest effect was the 'workplace participation' scheme (e.g., the one most similar in theory to the workfare scheme here).  Indeed, the 'vocational training' scheme had no effect on well-being.  The positive consequences of workplace participation schemes were even stronger for those who had been unemployed for a year or more, while the other two types of programme had zero effect for this group.

It's true that comparing Swedish programmes to British ones might be like comparing apples with oranges - and I'm also not suggesting that current Government policy is the right one.  But studies like Strandh's show that requiring unemployed people to participate in work-based programmes has the potential to have positive outcomes.  We thus need a much more nuanced debate in the UK. One which isn't about whether conditionality is 'right or wrong', but what about which conditionality is right and which conditionality is wrong.

Using Jahoda's theory of employment to understand the psychology of a PhD

Half the challenge of a PhD seems to be psychology.  Chats with other students about PhD life invariably revolve around the more psychological aspects of work.  Problems seem to centre less around the actual content of study but more about the actuality of doing it: structuring the day, keeping motivated, overcoming barriers and so on.

For my own PhD I'm exploring the psychosocial effects of unemployment and, more specifically, how welfare policies can help mediate these effects.  One key theorist in the literature is Jahoda (1982), who argues that one of the reasons why unemployment is so mentally damaging is because the process of work fulfils five key functions:

1. Time structure
2. Social contacts
3. Participation in collective purpose
4. Status and identity
5. Regular activity

Whilst reading Jahoda, I began to think that this theory might go some way into explaining the psychology of a PhD.  I don't think PhD students have any problem with no.4, status and identity, but I think Jahoda's other four 'psychosocial functions of employment' are challenged whilst doing a PhD. Structuring time is perhaps the most common problem cited by PhD students: the days are long and empty and it's solely up to you to fill them productively and efficiently.  Having social contact is an exception, not a rule.  Whilst our PhDs are contributing to a wider, collective knowledge, most of the time PhDs can feel highly narrow and individualised.  Finally, having regular activity can also be problematic, especially, I imagine, for first-year students engaging almost solely with the literature base.

While PhDs do present a challenge, understanding why this is - as  I think Jahoda's theory helps us to - can enable students to overcome these hurdles.  Structuring the day, linking up with fellow academics with similar interests and making the effort to attend conferences, lectures and workshops may just help us feel slightly - just slightly - like members of the real world.