Use of research by NGOs: a call for reflection and action

The authors stress the importance of taking research findings into account for NGO interventions, and identify the technical and political challenges this presents. They put forward several approaches favorable to taking research into account, while explaining that knowledge and initiatives are still lacking in this area. This article provides a brief outline of the current state of affairs and puts forward the idea of an issue of Humanitarian Alternatives with a focus on knowledge transfer by NGOs, to be published in 2021.

Generally science (in the academic sense), and specifically the findings of research and evaluations, is rarely spontaneously used by stakeholders, regardless of who they are. Humanitarian and development NGOs (non-governmental organisations) are no exceptions to the rule. Regarding the areas we know best, the same rule applies in Burkina Faso, even though the situation is obviously not specific to that country. In Cameroon, for instance, few NGOs draw on evaluation findings to inform their activities. Although evaluations are often commissioned, the way in which their findings are used is typically ineffective or insufficient^[1]Charly Gabriel Mbock et al., « Utilisation des résultats de la recherche dans l’action publique au Cameroun », Revue internationale des sciences sociales, 2004/1, n° 179..

Indeed, most of the research and evaluations conducted at the NGOs operating in the region have been limited to producing documents and reports, perceived as an “end” and not the “means”^[2]Valéry Ridde, Seni Kouanda et Jean-François Kobiané (dir.), Pratiques et méthodes d’évaluation en Afrique, L’Harmattan, 2016.. Furthermore, the sharing of findings is limited, in the best case scenario, to dissemination workshops (often very costly^[3]Valéry Ridde, N’koué Emmanuel Sambieni, Larissa Kojoue, L. et Oumar Mallé Samb, « Réformer les per diempar le dialogue », notes techniques, AFD, n° 40, 2018. and ineffective^[4]Yanick Jaffré, « Les objectifs, les séminaires et les recommandations permettent d’améliorer la santé des populations », in Valéry Ridde et Fatoumata Ouattara (dir.), Des idées reçues en … Continue reading ), with no real consideration to how the findings can be harnessed and used with a view to improving practices.

Why NGOs should be interested in using research?

While the findings of research and evaluations are clearly useful for any public initiative, the same can be said for NGOs. A thesis on how a German NGO based in Burkina Faso used findings has shown that any information learned by the evaluation of its interventions had improved the implementation of its activities, taking new decisions, planning initiatives more effectively, and supporting advocacy work based on robust data calling for public policy change^[5]Léna D’Ostie-Racine, Christian Dagenais and Valéry Ridde, “Examining Conditions that Influence Evaluation Use within a Humanitarian Non-Governmental Organisation in Burkina Faso (West … Continue reading. Consequently, research and evaluation should no longer be viewed as an exercise of judgement, but rather an activity that supports development and helps improve NGOs’ work, making it more effective, fairer and better suited to the context. It is here that “the science of using science”^[6]Laurenz Langer, Janice Tripney and David Gough, The Science of Using Science: Researching the Use of Research Evidence in Decision-Making, London: EPPI-Centre, Social Science Research Unit, UCL … Continue reading can certainly provide assistance.

“Research and evaluation should no longer be viewed as an exercise of judgement.”

“The science of using science”– a few definitions

We have opted for the expression “knowledge transfer” to describe the concept of “research towards and for action”. Our definition encompasses all the phases of the process of knowledge transfer, from the production of the research to its use. It includes all the efforts made to promote the use of research. It generally covers interaction with users in order to take into consideration their context, areas of expertise, and needs, without showing a patronizing, unidirectional, or demeaning attitude. Therefore, in our view, “knowledge transfer” is the term that most effectively encompasses all of the practices involved.

There are several types of knowledge use. When a participant from a social NGO attends the presentation of research findings at an academic conference and learns about research demonstrating that a new method is more effective than the standard practice in his or her organisation, we speak about a “conceptual use” of knowledge. Once back to work at the NGO, the same participant draws on the findings learned to convince their colleagues to change their modus operandi; in this case the participant is making “persuasive use” of knowledge. Should this act of persuasion prompt the participant’s colleagues to change their practices and processes, they then make an “instrumental use” of knowledge.

“The use of intermediaries known as ‘knowledge brokers’ is increasingly widespread.”

Low uptake of knowledge generated by research can be observed in a very large number of systems and institutions (humanitarian, health, education, justice, etc.) worldwide, from north to south, and east to west. To rectify the situation, the use of intermediaries known as “knowledge brokers” is increasingly widespread. This intermediary (a person or an organisation) fosters close relationships between knowledge users, such as decision-makers, stakeholders and professionals, and researchers. Research recently undertaken in Burkina Faso shows the challenges of using knowledge brokers in the field of health^[7]Esther Mc Sween-Cadieux, Christian Dagenais, Donmozoun Télesphore Somé and Valéry Ridde, “A health knowledge brokering intervention in a district of Burkina Faso: A qualitative retrospective … Continue reading.

The challenges in ensuring research use by NGOs

The main challenge is to change standard practice and successfully embed a research and evaluation culture, based on knowledge utilisation^[8]Michael Quinn Patton, « L’évaluation axée sur l’utilisation », inValéry Ridde et Christian Dagenais (dir.), Approches et pratiques en évaluation de programmes, Presses de l’Université … Continue reading, into NGO operating methods. NGOs (and their funding bodies) in particular need to work on shifting their perceptions of researchers and assessors. The latter are still too frequently perceived as “judges” tasked with singling out the imperfections or flaws in interventions, and not as people there to help the decision-making process in the interests of change. Apart from producing results, evaluators and their commissioners should make it a priority to encourage and support the effective use of findings by putting forward realistic operational recommendations. An NGO in Afghanistan has demonstrated that while this is not impossible, neither is it easy^[9]Valéry Ridde, Sylvie Goossens and Sahibullah Shakir, “Short-term consultancy and collaborative evaluation in a post-conflict and humanitarian setting: lessons from Afghanistan”, Evaluation and … Continue reading. Another experience in Burkina Faso showed that eight years of collaboration between researchers and an NGO helped influence political decision-making in the field of health^[10]Valéry Ridde and Pierre Yaméogo, “How Burkina Faso used evidence…”, art. cit.; Christian Dagenais, Ludovic Queuille and Valéry Ridde, “Evaluation of a knowledge transfer strategy from a … Continue reading.

The benefits for NGOs of using research and evaluation are relatively clear. In an ideal world, an NGO’s funding body would certainly be less reluctant to fund a project whose content is based on solid evidence, rather than the opposite. A significant amount of funding is obviously allocated on political or strategic grounds, but one might dream of a future when this position shifts and an NGO that disregards research or evaluation may struggle to develop in an increasingly competitive aid economy. For instance, at the present time, most of the Unitaid funding for NGOs is subject to a research team being included in the project consortium. In early 2020, an AFD (French Development Agency) call for bids for a sexual and reproductive health project in Senegal stipulated that a research institute must be involved in the consortium, headed up by an NGO. In a recent project in the Sahel, we suggested to the NGOs involved that knowledge brokers should be used in the countries in question.

A failure to use research, or, on the contrary, misusing it, notably to avoid critical appraisal of its work, means that an NGO runs the risk of being categorised as a member of what some have called the “success cartel”^[11]Yogesh Rajkotia, “Beware of the success cartel: a plea for rational progress in global health”, BMJ Global Health, 3(6), 2018. in which members find a way to sweep problems under the carpet and only publicise their successes. The study of the unintended effects of NGOs’ interventions is also often ignored, as was shown in Niger and Burkina Faso^[12]Valéry Ridde and Aïssa Diarra, “Case 16. From unintended to undesirable effects of health intervention: The case of user fees abolition in Niger, West Africa”, inJonathan A. Morell (ed.), … Continue reading. These results would, however, have been undeniably useful for improving the initiatives. In the short term, this “ignorance is bliss” strategy may be effective in order to continue operations, but in the long term it may prove detrimental to the reputation and, more importantly, the success of the organisation’s work in helping a given population.

“There is also a need to be mindful about conflicts of interest in the use (or non-use) of research…”

There is also a need to be mindful about conflicts of interest in the use (or non-use) of research, as this can also be detrimental to initiatives. For example, if a researcher has spent their entire career promoting mutual health insurance structures or results-based financing in Africa, while also undertaking numerous paid consultancy jobs, questions should be raised about their credibility when evaluating an NGO that is putting in place this type of tool without being objective. Our doubt only grows when we learn that the researcher is also a shareholder of a firm selling IT services to support their rollout. This is just one example to show to what extent the use of evaluations and research must also take into consideration the people and institutions producing them, just as much as the rigour of their analysis. NGOs often have to contend with challenges when deploying their interventions: what has been planned and funded is rarely fully rolled out on the ground. A fully-implemented initiative tends to be the exception that proves the rule. Upstream and downstream research can help NGOs ensure that implementation is more effective and context-based.

As a last point, failure to take into consideration current academic knowledge before deploying an intervention also raises a major ethical question. There needs to be a constant campaign to address certain popular beliefs (for instance, wormwood tea for treating malaria) to stop actions that science has long proven to be useless and even dangerous. 

Examples of knowledge transfer strategies

A recent study of the current state of research on the effectiveness of knowledge transfer grouped strategies according to underlying mechanisms of change^[13]Laurenz Langer, Janice Tripney, David Gough, The Science of Using Science…, op. cit.; Jonathan Breckon and Jane Dodson, “Using evidence: What works”, op. cit.. Six mechanisms were identified: raising user awareness about adopting positive attitudes towards the use of research (M1); developing a mutual understanding of research questions relevant to practices and decision-making (M2); improving access to and the way research is communicated (3); fostering interaction between researchers and users (M4); developing user skills so that they can utilise research (M5); and influence decision-making processes and organisational structures (M6).

The strategies would operate at three levels of intervention in view of bringing about behaviour change: the capacity to use knowledge, the motivation to do so, and the opportunity to do so. Strategies should have an impact on one or more of these levels to ensure that research is used.

However, there is still relatively little solid evidence of the effectiveness of these mechanisms, and there is notably insufficient robust data on NGOs. The difficulty of identifying the impact of some of these mechanisms can be explained by the fact that they are often combined with other strategies meaning it is difficult to confirm the specific impact of a given mechanism. Strategies revolving around communication tools (M3) and training (M5) are easier to isolate and therefore to assess. However, this is still done all too rarely. More research and analysis are required to determine the potential effectiveness of these mechanisms. However, several studies mention the importance of relying on multiple combined strategies to be able to raise awareness about using research, build the capacity of users and researchers to share their knowledge, make useful knowledge tailored to the local context more available, create forums for co-producing research, and help drive change in practices.

The findings of this research reiterate the ineffective nature of passive and linear methods of knowledge dissemination, such as traditional dissemination workshops at the end of the process^[14]Esther Mc Sween-Cadieux, Christian Dagenais, Paul-André Somé and Valéry Ridde, “Research dissemination workshops: observations and implications based on an experience in Burkina Faso”, Health … Continue reading. For example, strategies designed to improve access to and communication of research (M3) should offer the opportunity to use knowledge, while working on user motivation to do so.

The findings also show the importance of carefully defining the target audience of a strategy to communicate research in order to tailor the message to their needs and preferences. Dissemination workshops to which everyone is invited, without having firstly considered the target audience, do not meet this requirement. Furthermore, it is often beneficial to involve users in the process, to enable them to choose their preferred communication channel for accessing knowledge, and give them knowledge at the right time and not several months or even years after the research has been completed. This means diversifying and increasing the methods used to disseminate research.

Finally, strategies which aim to develop users’ skills so that they can access and understand research findings would only be effective if the educational component was sufficiently big (a half-day training course is rarely enough) and if subsequent support is provided. These capacity-building strategies would produce equally positive results if they were delivered online as opposed to face-to-face, and if they also applied interactive teaching methods.

A few knowledge transfer tools

The strategies that we have just described also need to make use of specific knowledge transfer tools. Many are available but we will focus here on presenting the tools we used most in Africa. 

Policy briefs

Several terms are used to refer to what we call a note de politiquein French. In English, the term is “policy brief” (and this is certainly the most widely used term, even by French speakers). Policy briefs can be placed on a continuum from the most neutral to the most interventionist. In the teams in which we work, we often focus on interventionist policy briefs, which take the form of a short document, written in clear language, and with an eye-catching format. This policy brief summarises the findings of one or more pieces of research and makes recommendations to a non-specialist public, with the idea being that readers will apply these recommendations in their professional practices or for decision-making purposes^[15]Christian Dagenais et Valéry Ridde, “Policy brief (PB) as a knowledge transfer tool: To ‘make a splash’, your PB must first be read”, Gaceta sanitaria, 32(3), 203, 2018; … Continue reading.

Decision-making workshops

To promote the use of knowledge produced by research, it is increasingly recommended that deliberative dialogue be implemented, based on solid evidence and bringing together multiple participants in view of establishing a rigorous and comprehensive decision-making process^[16]Juliet Nabyonga-Orem et al., “Policy dialogue to improve health outcomes in low income countries: what are the issues and way forward?”, BMC Health Serv Res, 16, 217, 2016..

In Burkina Faso, deliberative workshops were organised to enable researchers to present and discuss with stakeholders the key findings of two research projects, and to initiate decision-making dialogue based on the recommendations^[17]Équipe RENARD, « Organiser un atelier délibératif pour partager les résultats de recherche ? », Coup d’œil sur la recherche, n° 2, … Continue reading. The researchers briefly presented the main findings and potential solutions in order to guide the decision-making process. This phase helped familiarise the stakeholders with the content and determine how to implement the relevant and applicable recommendations. After the findings were presented, a large portion of the time was devoted to discussing the findings, and specifically the recommendations formulated, with a view to making decisions and changing practices or circumstances. The discussions resulted in the development of an agreed action plan to reduce traffic accidents. Safety measures were taken by the police: the number of police officers posted to accident hotspots was increased. A few years later, the posters produced on the basis of this research evidence are still displayed on the walls of the police station.

Video

Video can be an effective knowledge transfer tool^[18]Fiorella Logan and Richard Mayer, “What works and doesn’t work with instructional video”, Computers in Human Behavior, vol.89, 2018, p.465-470.. In public health, it can be a valid knowledge transfer and exchange strategy^[19]Pascale Lehoux, Patrick Vachon, Geneviève Daudelin and Myriam Hivon, “How to summarize a 6,000-word paper in a six-minute video clip”, Healthcare Policy, 8(4), 2013, p.19-26.. Visuals have advantages over text^[20]Albert Mehrabian, Silent Messages: Implicit Communication of Emotions and Attitudes, Belmont, California: Wadsworth Publishing Company, 1981.. The use of images to transfer knowledge is especially relevant because learning is more effective when visual and verbal information is provided simultaneously rather than successively. Videos therefore have the power to make a significant contribution to the organisation-to-organisation learning process^[21]Paul Van Mele, “Video-mediated farmer-to-farmer learning for sustainable agriculture”, 2011, http://agroinsight.com/downloads/articles-divers/Farmer-to-farmer-video-FINALREPORT-Van-Mele-2011.pdf. They have also been used in several programmes and have delivered conclusive results.

A video was used as part of a research project into dengue fever in Burkina Faso^[22]« La dengue au Burkina Faso », https://vimeo.com/240513650. This research took three narrative genres (report, drama performance, and graphic animation) and sought to determine which was the best tool for transferring knowledge in order to train healthcare professionals to treat victims of dengue fever. The videos contained the same information about the transmission, diagnosis and treatment of dengue fever. They were then presented to nursing students. The findings showed that the drama performance and graphic animation videos facilitated the transmission and retention of new knowledge about dengue fever, notably because of specific narrative devices. These devices were the changing rhythm of the story, viewers being addressed directly, and a visual universe created using bright colours combined with images to illustrate the explanations. The video was widely disseminated online and shared on social media. It also helped to raise public awareness about dengue prevention.

A call for reflexivity

This article has no pretension other than to offer an introduction to the importance and challenges of NGOs using science, in the sense of academic knowledge. Readers wishing to increase their knowledge will find many resources that can be used to raise their skills. For French speakers, we are going to offer a free online course (https://www.equiperenard.org) containing 45 hours of training to enable course participants to gain a better understanding of knowledge transfer and how to draft a policy brief. The course will be translated into English and Spanish in early 2021.

However, with this article, we also wish to appeal to the community of NGOs and researchers or evaluators. We still have much to learn about how NGOs try to use research and evaluations in their work. We are asking people to stand back and take a reflexive approach, and we ask that anyone interested in the subject submit their papers to us. We would like to read about your experiences, thoughts, strategies, challenges, failures and successes when using research and evaluation evidence produced by and for NGOs*. In the meantime, we are also calling for action, for NGO knowledge transfer initiatives to be launched. Not enough is being done to make research useful to and used by NGOs. We all need to act and reflect so that science is better used.

* This appeal could serve as a prelude to producing an issue of Humanitarian Alternativeswith this subject as a main focus, be published in 2021. If you wish to submit a draft article, please send a summary of your article and an outline (one page maximum) to the following address by Monday, 4 May 2020: valery.ridde@ird.fr 

 Translated from the French by Gillian Eaton