Exploring the perspectives of International Nongovernmental Organizations (INGOs) on the use of program evaluation and impact assessment in their work
DOI:
https://doi.org/10.22230/cjnser.2013v4n2a143Keywords:
International Nongovernmental Organizations, Program Evaluation, Impact Assessment, Qualitative ResearchAbstract
In the twenty-first century, the call for International Non-governmental Organizations (INGOs) to demonstrate their effectiveness has become popularized. This has given rise to scholarly attention examining the roles of program evaluation and impact assessment in assisting INGOs in demonstrating their effectiveness. While previous studies suggest that INGOs actively conduct program evaluation and impact assessment, this article explores the perspectives of two Canadian INGOs on how they understand, use, and experience evaluation and assessment as it relates to their work. Our study uncovers three continuing challenges: evaluation and assessment are largely descriptive and lack more sophisticated analyses; efforts to conduct evaluation and assessment are consolidated within organizations’ head offices, while staff members and volunteers are largely excluded; and evaluation and assessment remain rooted in the paradigm of quantifiable results, which do not truly reflect the nature of work being conducted on the ground.
Au vingt-et-unième siècle, on veut de plus en plus que les organisations non gouvernementales internationales (ONGI) démontrent leur efficacité. Ce désir a motivé les chercheurs à se pencher sur les évaluations de programme et les études d’impact pour voir dans quelle mesure celles-ci peuvent aider les ONGI à montrer qu’elles sont efficaces. Des études antérieures suggèrent que les ONGI mènent de manière concertée des évaluations de programme et des études d’impact. Cet article explore comment aujourd’hui deux ONGI canadiens comprennent, utilisent et vivent l’évaluation et la mesure de leur travail. Notre étude relève trois défis actuels : l’évaluation et la mesure tendent à être descriptives sans offrir d’analyses plus poussées; ce sont les sièges sociaux des organismes qui gèrent l’évaluation et la mesure en excluant ainsi bon nombre de fonctionnaires et volontaires; l’évaluation et la mesure se limitent au paradigme des résultats mesurables et par conséquent elles ne reflètent pas nécessairement le véritable travail mené sur le terrain.
Downloads
References
Abdel-Kader, M. G., & Wadongo, B. (2011). Performance Management in NGOs: Evidence from Kenya. Retrieved January 20th, 2013, from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1909863.
Alaimo, S. (2008). The Board’s Role in Program Evaluation Capacity for Nonprofit Human Service Organizations. Retrieved from http://www.bloch.umkc.edu/mwcnl/Conferences/GovernanceConf2009/Papers/Alamio 2009.pdf.
Atack, I. (1999). Four Criteria of Development NGO Legitimacy. World Development, 27(5), 855–864.
Barber, M., & Bowie, C. (2008). How international NGOs could do less harm and more good. Development in Practice, 18(6), 748–754.
Bouchard, M. J. (2009). The Worth of Social Economy. In M. J.
Bouchard (Ed.), The Worth of Social Economy: An International Perspective (pp. 19–34). Bruxelles: P.I.E. Peter Lang.
Brown, L. D., & Moore, M. H. (2001). Accountability, Strategy, and International Nongovernmental Organizations. Nonprofit and Voluntary Sector Quarterly, 30(3), 569–587.
Campbell, D. (2002). Outcomes Assessment and the Paradox of Nonprofit Accountability. Nonprofit Management and Leadership, 12(3), 243–259.
Campbell, M. L., & Gregor, F. M. (2002). Mapping Social Relations: A Premier in Doing Institutional Ethnography. Aurora, Ontario: Garamond Press.
Campos, L., Andion, C., Serva, M., Rossetto, A., & Assumpção, J. (2010). Performance Evaluation in Non-Governmental Organizations (NGOs): An Analysis of Evaluation Models and their Applications in Brazil. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 22(2), 238–258.
Dawson, A. (2010). A case study of impact measurement in a third sector umbrella organisation. International Journal of Productivity and Performance Management, 59(6), 519–533.
Ebrahim, A., & Rangan, V. K. (2010). The Limits of Nonprofit Impact : A Contingency Framework for Measuring Social Performance (Harvard Business School General Management Unit Working Paper 10-099). Cambridge, MA: Harvard Business School. Retrieved January 20th, 2013 from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1611810.
Edwards, M. (1997). Organizational learning in non-governmental organizations: What have we learned? Public Administration and Development, 17(2), 235–250.
Edwards, M., & Hulme, D. (1996). Introduction: NGO Performance and Accountability. In M. Edwards & D. Hulme (Eds.), Beyond the Magic Bullet: NGO Performance and Accountability in the Post-Cold War World (pp. 1–21). West Hartford, Connecticut: Kumarian Press.
Fisher, W. F. (1997). Doing good? The politics and antipolitics of NGO practices. Annual Review of Anthropology, 26(1), 439.
Fowler, A. (1996). Assessing NGO Performance: Difficulties, Dilemmas, and a Way Ahead. In M. Edwards & D. Hulme (Eds.), Beyond the Magic Bullet: NGO Performance and Accountability in the Post-Cold War World (pp. 169–187). West Hartford, CT: Kumarian Press.
Fowler, A. (1997). Striking a Balance: A Guide to Enhancing the Effectiveness of Non-Governmental Organizations in International Development. London, UK: Earthscan Publications Limited.
Gibelman, M., & Gelman, S. R. (2004). A Loss of Credibility: Patterns of Wrongdoing Among Nongovernmental Organizations. Voluntas: International Journal of Voluntary & Nonprofit Organizations, 15(4), 355–381.
Gosling, G., & Edwards, M. (2003). Toolkits: A Practical Guide to Planning, Monitoring, Evaluation and Impact Assessment. London, UK: Save the Children.
Harsh, M., Mbatia, P., & Shrum, W. (2010). Accountability and Inaction: NGOs and Resource Lodging in Development. Development & Change, 41(2), 253–278.
Hseih, H.-F., & Shannon, S. E. (2005). Three Approaches to Qualitative Content Analysis. Qualitative Health Research, 15(9), 1277–1288.
Keats, P. A. (2009). Multiple Text Analysis in Narrative Research: Visual, Written, and Spoken Stories of Experience. Qualitative Research, 9(2), 181–195.
Kelly, L., & Chapman, R. (2003). A Process to Define and Enhance NGO Effectiveness. Australian Project Manager, 23(2), 1–9.
Lecy, J. D., Schmitz, H. P., & Swedlund, H. (2011). Non-Governmental and Not-for-Profit Organizational Effectiveness: A Modern Synthesis. VOLUNTAS, 23(2), 434–457.
Lockie, S. (2001). SIA in review: setting the agenda for impact assessment in the 21st century. Impact Assessment and Project Appraisal, 19(4), 277–287.
Mason, J. (2004). Qualitative Researching (2nd ed.). London, UK: Sage Publications.
Moxham, C. (2009). Performance Measurement: Examining the Applicability of the Existing Body of Knowledge to Nonprofit Organizations. International Journal of Operations & Production Management, 29(7), 740–763.
Neuman, W. L. (2004). Basics of Social Research: Qualitative and Quantitative Approaches. Boston, MA: Pearson Education, Inc.
Nicolau, I., & Simaens, A. (2009). The Evaluation of Social Solidarity Organizations in the Portugese Context. In M. J.
Bouchard (Ed.), The Worth of Social Economy: An International Perspective (pp. 193–212). Bruxelles: P.I.E. Peter Lang.
Parks, T. (2008). The rise and fall of donor funding for advocacy NGOs: understanding the impact. Development in Practice, 18(2), 213–222.
Perret, B. (2009). Evaluating the Social Economy. Clarifying Complex Rationality. In M. J. Bouchard (Ed.), The Worth of Social Economy: An International Perspective (pp. 35–42). Bruxelles: P.I.E. Peter Lang.
Serva, M., Andion, C., Campos, L., & Onozato, E. (2009). Evaluation of the Social Economy in Brazil: An Analysis of the Practices in some NGOs. In M. J. Bouchard (Ed.), The Worth of Social Economy: An International Perspective (pp. 171–192). Bruxelles: P.I.E. Peter Lang.
Smith, M. B. (2008). International non-governmental development organizations and their Northern constituencies: development education, dialogue and democracy. Journal of Global Ethics, 4(1), 5–18.
Spar, D., & Dail, J. (2002). Of measurement and mission : Accounting for performance in non-governmental organizations. Chicago Journal of International Law, 3(1), 171–181.
Szporluk, M. (2009). A Framework for Understanding Accountability of International NGOs and Global Good Governance. Indiana Journal of Global Legal Studies, 16(1), 339–361.
Travers, S. (2011). Canadian Civil Society Organizations Influencing Policy and Practice: The Role of Research. Retrieved from the International Development Research Centre Web Site: http://www.idrc.ca/EN/Documents/Stacie-Traver-working-paper-5-final.pdf.
University of Oxford (2012). International Development. Retrieved from University of Oxford Career Services Web site: http://www.careers.ox.ac.uk/options-and-occupations/sectors-and-occupations/international-development/.
Vitale, D. C., Armenakis, A. A., & Field, H. S. (2008). Integrating Qualitative and Quantitative Methods for Organizational Diagnosis: Possible Priming Effects? Journal of Mixed Methods Research, 2(1), 87–105.
Walsh, E., & Lenihan, H. (2006). Accountability and effectiveness of NGOs: adapting business tools successfully. Development in Practice, 16(5), 412–424.
Downloads
Published
Issue
Section
License
Submission of an original manuscript to the Canadian Journal of Nonprofit and Social Economy Research / Revue canadienne de recherche sur les OSBL et l'économie sociale [thereafter CJNSER] will be taken to mean that it represents original work not previously published, and that it is not being considered elsewhere for publication.
The journal takes the stance that the publication of scholarly research is meant to disseminate knowledge and, in a not-for-profit regime, benefits neither publisher nor author financially. It sees itself as having an obligation to its authors and to society to make content available online now that the technology allows for such a possibility. In keeping with this principle, the journal will publish all of its issues online.
Authors who publish in CJNSER agree to release their articles under the Creative Commons 4.0 International Licence (CC BY). This licence allows anyone to copy and distribute the article provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.
[Content published between 2010 and 2019-10 was licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 2.5 Canada (CC BY-NC-ND 2.5 CA) License. Content published between 2019-10 to 2023-12 was licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Content published after 2023-12 is licensed under the Creative Commons 4.0 International (CC BY) License.]
