The Evidence Beast: how short-sighted is it?

A word to my Dutch, German and Belgian readers before I start my article. I am taking part in a carnival of social work blogs, organized by my Canadian fellow-blogger Gord Cummings (http://funcass.blogspot.com/) . As my French is even worse than my English, I decided to do this blogpost in English. But, do not despair: my next article will be in Dutch again.

In the Netherlands, as well as in many other countries, society is calling upon social workers to use only evidence-based interventions. Evidence-based practice has become the central theme in discussions about the effectiveness of social work. Local and national authorities who subsidize social work demand that workers only use evidence-based methods. The majority of social workers doesn’t feel very comfortable about this. Why is this? Why do social workers have so much trouble accepting the outcomes of scientific research as the guideline for their actions?

Three Dutch authors, Jan Steyaert, Tineke van den Biggelaar and Johan Peels recently published a very interesting book in which they try to find an answer to these questions. Its title in English would be something like: The Short-sightedness of Evidence-Based Practice. In case there are any Canadian or American publishers reading this blog: it is a very good book and the authors have a talent for explaining complex matters in a really clear and simple way.

from: http://www.lib.nus.edu.sg/mlb/subj/ebp.html

Part of the answer to the above questions could lie in the fact that evidence based practice is, as the authors say, both democratic and dictatorial. It’s democratic because every intervention that is being tested has an equal chance of being approved as effective. It doesn’t matter who the author is. But EBP is also dictatorial in a double sense. It’s dictatorial because there is a strict hierarchy in the types of research that are being done, with Random Controlled Trials and systematic reviews at the top (a systematic review being a comprehensive review of all research that has been done on a certain topic, e.g. an intervention). It’s also dictatorial because it deprives the professional of part of his autonomy. EBP tells the social worker how to act in a given situation.  Most social workers don’t feel comfortable with that.

But there is more. Good social work research is very rare. It’s easy (well, relatively speaking, that is) to conduct research in a lab with people who have only one, well-defined, problem. But that is not the reality social workers have to deal with. Their clients often have multiple, interdependent problems. They experience domestic violence and at the same time have to face financial problems and raise an autistic child. And to make things more complex: social workers often are not the only professional involved, but they work closely together with others. So, if the situation improves, how can you pinpoint which intervention brought about the change?

Another problem with EBP is that research, especially the type of research that is high in the hierarchy I mentioned before, is mostly conducted in a quantitative way, with reality being reduced to measurable data (e.g. the score on a depression test). But reality is far more complex than that. And probably the things that really make social work tick can not be caught in numbers. On the contrary, there has been a lot of research indicating that the so called common factors (factors having to do with the personality of the client and of the social worker and the quality of their working relationship) are the things that really matter.

Steyaert, Van den Biggelaar and Peels underpin their line of argument with a number of (as they call them) “biographies” of interventions from the present and the past. It is very interesting to see how interventions that once were considered evidence-based, have fallen from favor; not because research has shown that they don’t work, but because the socio-political climate has changed. Other biographies show how concepts like active citizenship arise and all kind of interventions for the promotion of citizenship are deployed without any scientific evidence for their effectiveness. Sometimes an intervention just seems to fit in perfectly with the Zeitgeist.

At the moment, Triple-P is one of the most popular systems of parenting interventions in the Netherlands. A lot of research has been conducted into its effectiveness. Local and regional authorities insist that social workers working with parents use Triple-P, because of its proven effectiveness. But Triple-P consists of a range of interventions that are used in a variety of contexts. Therefore, it is difficult to assess its effectiveness. Moreover, many studies reason that Triple-P is effective because parents say their problems have diminished. But is client satisfaction a good measure for the effectiveness of an intervention? Surely, it’s an important aspect. But there is also something called the hello-goodbye effect. At their first contact with a social worker, people tend to present their problems as very urgent and serious. They want the worker to understand that it is really important that they get help. At the end, when they evaluate the intervention, they tend to exaggerate the progress that has been made. They do so for at least two reasons: first, because they don’t want to disappoint the worker (and themselves) for all the effort made, and second, because of halo-effects. We can observe a halo-effect when people value every aspect of the intervention as positive because they are satisfied with one aspect of it, for instance the fact that the worker was very understanding.

Does this mean that social work can do without research? Of course not. Evidence-based practice can make social work more effective. But we should not raise our expectations sky high. Moreover, the authors argue, we should also look at EBP from a sociological perspective, taking into account all the forces (political, philosophical, societal AND scientific) that shape the biography of an intervention. If we fail to do so, evidence based social work will keep on suffering from short-sightedness.

Reading The Short-sightedness of Evidence-Based Practice gave me great pleasure. It is very well written, in understandable language. I hope there will be an English version soon. 

On his website, you can find many interesting publications by Jan Steyaert, including a PDF of his book. The book is in Dutch, but some of the articles are in English.