Donna Mertens
There are times when the state of the world seems over-whelming because the problems are too complex or the people in power deny that there is a problem. It would be easy under these circumstances to feel paralyzed and unable to do anything to improve social, economic, and environmental justice.
The sheer weight of the political climate and our inability to tackle some of the world’s most intractable or ‘wicked’ problems can leave us feeling frozen – unable to take any kind of action. But I am optimistic that change can and will happen and I am calling on researchers everywhere to consider how the methods they use in their research can help.
I should explain that I didn’t just pluck the word ‘wicked problem’ out of the air. It is actually a term created by two researchers back in the seventies to draw attention to problems with no obvious or clear solution. Nowadays, they are also thought of as problems for which time to find a solution is running out, problems, which, if not solved, will have dire consequences for us all.
A good example of a ‘wicked problem’ is climate change, but there are numerous others: lack of access to healthcare and clean water, to agricultural land, sovereignty and self-determination, the prevalence of poverty and violence are just a few to get us started.
In the case of climate change for example, we can say that the bulk of the evidence supports the findings of the U.S. Global Change Research Program Climate Science Special Report, which states that “it is extremely likely that human activities, especially emissions of greenhouse gases, are the dominant cause of the observed warming since the mid-20th century”.
They reviewed thousands of studies from around the world that documented “changes in surface, atmospheric, and oceanic temperatures; melting glaciers; diminishing snow cover; shrinking sea ice; rising sea levels; ocean acidification; and increasing atmospheric water vapor.”
Yet, even with this large body of research-based evidence, we see the continuation of practices that are destructive to the environment – often in the name of economic development and job creation. So, whatever its provenance, a wicked problem seems a pretty apt description to my mind.
Playing a key role
So if the world seems to be ignoring the evidence produced and presented by researchers, how come I still think we have a key role to play? The answer lies in the methods we use and the way in which those methods can be most effective in strengthening the credibility of evidence.
First, our research needs to be transformative, by which I mean it needs to acknowledge that many of the problems we are facing are systemic; based in a system that is unjust, discriminatory and oppressive.
Transformation is multi-levelled; it can be both personal and social. Personal transformative moments can influence us to recognize inequities and commit to societal transformation. The meaning of transformation itself is important within the context of each research study. It helps us determine: What is accepted as the reality of transformation? This question has different answers depending on who you ask.
30 years of working with marginalised groups including the deaf community, indigenous people, women, disabled people and members of minority ethnic groups has taught me that many of them don’t think researchers are accurately capturing their experiences. What they tell me is that researchers come, take data, leave and they see no substantive change.
Respectful and responsive
Researchers can change this by being culturally respectful and responsive, by recognising power differences, working with people to create a platform for a change that they see as valuable for themselves and their communities.
We can do this by not assuming that we understand the nature of a problem and the nature of the solution. We also need to accept and take into account that there are different perspectives on the same issue.
In the case of climate climate change, for example, there are two versions of the same reality. One version is that climate change does not exist. Another version is that climate change does exist; it is a serious problem, and human activity is a contributor to that problem. This is exemplified by two viewpoints about the effect of dozens of trucks driving over a bridge and the finding the highest rates of asthma, heart disease, and cancer for poor people who live nearest to the bridge. Is this a coincidence or is there a link between pollution and poor health?
To move forward we have to make both versions visible, examine the consequences of both and ask ourselves who suffers and who benefits.
There are plenty of other examples from many sectors of different versions of reality associated with different positions of power. In Australia and Indonesia, indigenous people see mining and palm oil production as issues that are salient in terms of land rights, prevention of pollution, protection of the coral reef and other waterways, loss of agricultural land, need to protect the forests, water security, well-being, and the need to reduce the gap between rich and poor.
However, government officials and corporate interests see these topics as being connected to job creation, energy production, economic growth, and profits. There are consequences of accepting one version of the reality over the other in terms of respect for human rights and protection of the environment. These different versions of reality are associated with different levels of power.
We are all interconnected and we have a responsibility to all living and nonliving things, as well as to the consequences of power inequities. It means that researchers need to establish an interactive link between themselves and the stakeholders that are based on cultural respect and addressing power inequities.
When we set out on our research, we must ask who needs to be included, how we build relationships and work with them, how we can be transparent and understand that things may need to be flexible and change. We need to understand the history and not treat our participants like animals in the zoo that we have come to observe.
Mixed methods
Transformative methodology generally involves the use of mixed methods, i.e. the use of both quantitative and qualitative methods to capture the full complexity of a phenomenon. Mixed methods are used to address power differentials in the stakeholder groups by providing multiple avenues for data collection that are culturally responsive. They are used to consciously give voice to the less powerful while making visible the contrasts between versions of reality as put forth between the less and the more powerful.
It involves the development of cyclical, culturally responsive design that allows for engagement of multiple stakeholder groups. It allows for time to develop relationships, build coalitions, and design strategies for working together, a contextual analysis, determining levels of incidence of the problem, available data on the nature of the problem, differences in experiences of the problem by various constituencies.
A mixed methods cyclical design to study the performance of Korean eco-parks might include several phases of quantitative and qualitative data collection.
In Phase 1 researchers need to establish who should be involved; this may include recruiting co-researchers from the communities who can serve as liaisons with groups that might be suspicious of the researchers. Dimensions of diversity that are relevant within the contexts need to be identified and members of those communities need to be consulted in culturally appropriate ways to determine recommended ways of interacting. Documents that are relevant to the culture and the issues need to be systematically reviewed. The results of this qualitative phase of data collection can be combined with quantitative data that is available concerning demographic characteristics and environmental quality indicators. Documents concerning climate change and the evaluation of efforts to address this issue would be relevant reading.
Based on the results of Phase 1, the researchers can implement systematic data collection of both quantitative and qualitative nature to gain a better and broader understanding of the context and to establish baseline in terms of relevant variables, such as pollutants and health conditions.
Information from Phase 2 can be used in Phase 3 to develop multi-level interventions that address stakeholders’ concerns. The proposed intervention can then be pilot tested through the collection of both quantitative and qualitative data. This information would be used to refine the intervention and data collection strategies.
Evaluation
In a 2014 blog, World Bank Senior Adviser, Ken Chomitz discusses the challenges and advantages of evaluating demonstrations and pilot projects to inform decisions about interventions, pointing out it is not as if there were a pre-existing, clear, roadmap for economic development and poverty reduction. He writes:
“Now the way forward is further obscured by the need for pervasive changes in the way that we produce energy, grow food, use water, and prepare for droughts, floods, and storms. There are lots of good ideas, but not all of them will pan out as expected. What’s needed, at every level from the community to the planet, is the acuity to recognize both dead ends and promising pathways as rapidly as possible… Demonstration and pilot projects – which prove technical feasibility, work out regulatory issues, and reduce perceived investment risks – can have far-reaching impacts, but are successful only when they specify what is being demonstrated to whom, why, and how. The CIF [Climate Investment Funds] evaluation found that some would-be transformative energy interventions were likely to be stymied by unfavourable national energy policies.”
Unless we put the systemic nature of the world’s wicked problems out on the table, we will not come up with solutions and our research will simply help reinforce and perpetuate the status quo.
Researchers stand at an important point in history – they can be frozen in their tracks, continuing to do research ‘on people’, collecting, analysing and reporting and moving on, or they can work in a way that stands a better chance of solving some of the world’s ‘wicked’ problems and help achieve economic and social justice for people.
As for me, I refuse to be paralysed any more. This won’t get the job done! I am throwing all my creative energies into doing research that I believe can change the world and spreading the word to as many people who will listen.
Donna Mertens is giving a keynote talk at the ESRC Research Methods Festival in Bath in July, 2018. This article for Discover Society is based on her book Mixed Methods Design in Evaluation published by Sage. It is a longer version of an article that first appeared in the Guardian. Professor Emeritus, Donna taught research and evaluation at Gallaudet University for 31 years; she is a past president of the American Evaluation Association and served as the editor for the Journal for Mixed Methods Research for five years. She conducts and consults on evaluations in many countries, for example, reduction of gender based violence in Kyrgyzstan, environmental justice in Korea for Yonsei University, Kazakhstan for UN Women, Nepal and India for the World Bank and the Canadian International Development Agency, Sri Lanka for the Sri Lanka Evaluation Association, Australia for the Aboriginal Health Education and Research Unit, and Egypt for Higher Education Development. The primary focus of her work is transformative mixed-methods inquiry in diverse communities that prioritizes ethical implications of research and evaluation in support of human rights and social justice.
Image: LoraCBR CC BY 2.0