The last project I worked on in Chad (an evaluation of an innovative psychosocial program) invovled a large-scale survey to identify prevalence of distress among Darfur refugees living in the eastern part of the country. Doing survey research in the Sahel presented a number of challenges, including an employee pool with little experience doing surveys (remedy: lots of researcher basic training), a population suspicious of people asking any questions whatsoever (remedy: lots and lots of explanations and staff drawn from the camps themselves), and sand getting into laptop computers and scanners (remedy: scotch tape, everywhere). My personal favorite, however, concerned the visual scale we used in order to help survey respondents express their level of distress for each symtpom (e.g., “Have you had trouble sleeping? How much has this bothered you: not at all, a little bit, moderately, a lot, extremely”). Here is a picture of what we came up with:
You can see it is arranged from right to left, as Darfuris are from cultures that are Arabic-speaking and Arabic is oriented from right to left. The woman on the far right is carrying only a few sticks on her head and she represents no distress, the next one a little bit more, and so on until she just can’t carry the sticks anymore–extreme distress (the figure on the far left). We engaged a great graphic artist (known in New York for her comic books) to make these images, using photos from Darfur and consultation with people who had done work with the population. We made one set for women (pictured) and one set for men. Others (e.g., Paul Bolton’s group) have used this type of visual scale with people carrying things in other projects in the region. In general, pilot testing with this scale seemed to work pretty well. Everyone got that one was worse than the other, and could generally point to how badly they were feeling given a specific item.
A couple weeks into the project (the whole survey took three months, over 1500 respondents for a two hour protocol), I was sitting in the room with an interviewer and a participant, and after a few items using the scale the participant started chuckling to himself. The interviewer smiled, repeated the question, and the man started shaking his head. The interviewer asked him a question, and the man answered by pointing to the figure on the ground, the most extreme case to the far left. I asked the interviewer what the man had said.
“He says that this person must be very sick, or else he would be able to carry the wood. That’s really not a lot of wood. And he says that this person [pointing to the figure furthest to the right] must be really lazy. He should be carrying more wood.”
We thought we were measuring participants’ symptom severity, really were asking how lazy or weak they were. Hm.