With World Mental Health Day passing on October 10 (a UN celebration — the UN is sort of like the Catholic Church in that there’s a special day for everything), I got a chance to speak to NGO reps at the UN about the evaluation of psychosocial programs. There was the usual New York psychological crowd (ever suspicious of quantitative evaluation and randomized control trials–RCTs–in particular), some interested development students from Harvard (thanks for the moral support), and a few UN officers in the back. I enjoy a mixed crowd (even one with hostile elements), but somehow I always feel like I have more to say after I leave than when I get there. Lucky for me, I have a blog.
My frustration with psychosocial programs, particularly those set up for refugees, is really two-fold:
(1) they mean just about anything, from building playgrounds to distributing tricyclics
(2) despite this variety, they focus on mental health outcomes as their primary outcome
Now of course these are generalizations, and there are exceptions (IRC sponsors an innovative domestic violence intervention that gives money to battered women and measures battering as its outcome… it seems to work). But still, if you look at the flood of programs that call themselves psychosocial, they talk a big talk (restoring hope, rebuilding communities, making children smile again), and then, if they evaluate their programs, they measure depression symptoms. Or maybe PTSD.
So, what should they be measuring as their outcomes? Well, first of all measurement should be specific to intervention goals (see IRC program mentioned above). This has been done most successfully with counseling interventions: Paul Bolton of Johns Hopkins has run counseling interventions in East Africa (and now Kurdistan), and evaluates them with RCTs and mental health screening tools (Frank Neuner has also done this in Sudanese refugee camps). But most psychosocial programs claim some sort of higher-level change: rebuilding community, reducing daily stress in camps, etc. For these goals you need to have some approach that looks at community-level assessments, or social-ecological assessments.
The world of social ecological assessment isn’t yet really that developed. The best discussion I can find is in the field of Community Psychology, in which there’s a fair amount of talk about social climate and community organizing as psychological intervention. Perhaps another fruitful angle on social ecology might come through the economic concept of externalities. Ever since Econ 101 my junior year at Columbia, I’ve had a minor obsession with externalities, those things you get (for free) because of someone else’s venture, both things you like and things you don’t like — e.g., hearing Beethoven from your neighbor’s apartment, or acid rain.
What would psychosocial externalities look like? Well, one program I evaluated in Chad involved hiring refugees to identify those people in their blocks that appeared distressed and refer them for counseling. These guys got together and did much more than that — they became health advocates for at the local health center, they accompanied people to NGOs for material goods, they even started a savings and loan organization amongst themselves. These were not the goals of the psychosocial intervention, but I would bet that these things helped more people’s well-being than the counseling aspect of the program.
The funny thing about externalities is that they aren’t really obvious when you set out on your venture. Some (like hearing Beethoven), are perhaps predictable, but many are not. So, when planning psychosocial programs, or at the very least when planning to evaluate them, we should spend some time scenario planning, and then we need to keep our eyes open for those collateral social phenomena that occur because of psychosocial interventions.