Organizational characteristics that influence capacity building in evaluation in community organizations in Quebec: a review of the literature
DOI:
https://doi.org/10.29173/cjnser.2019v10n2a287Keywords:
Community organizations, Organizational evaluation capacity, Evaluation capacity building, Program evaluation / Organismes communautaires, Capacités organisationnelles en évaluation, Renforcement des capacités en évaluation, Évaluation des programmesAbstract
Few studies have conceptualized evaluation capacity for community organizations (COs), and even fewer in the specific context of COs in Québec. The objective of this article, therefore, is twofold: 1) to identify barriers to, and opportunities for, building evaluation capacity among OCs and 2) to verify the extent to which an existing conceptual framework of organizational evaluation capacity established by Isabelle Bourgeois and J. Bradley Cousins could be useful in its current form for understanding the evaluation capacity needs of COs in Québec. We first conducted a review of the scientific literature to identify the barriers and opportunities for evaluation capacity building in OCs, and analyzed the results of the literature review against the conceptual framework. Our results point to the fact that Bourgeois and Cousins’ framework is appropriate for understanding the key factors influencing the capacity to do and to use evaluations among OCs. Additional efforts would, however, be necessary to improve the framework’s compatibility with the practices and values of COs in Québec.
Les capacités en évaluation sont peu étudiées et encore moins conceptualisées en fonction du contexte particulier des organismes communautaires (OC) du Québec. Ainsi, l’objectif de cet article est double : 1) identifier chez les OC les barrières et les éléments qui facilitent le renforcement des capacités en évaluation et 2) vérifier dans quelle mesure un cadre conceptuel établi par Isabelle Bourgeois et J. Bradley Cousins pourrait s’avérer propice dans sa forme actuelle pour l’analyse des capacités en évaluation du milieu communautaire québécois. Nous avons d’abord identifié les barrières et les éléments qui facilitent le renforcement des capacités en évaluation des OC à partir d’une recherche documentaire d’articles scientifiques pertinents pour ensuite effectuer une analyse à partir du cadre conceptuel choisi. Nos résultats démontrent que le cadre conceptuel de Bourgeois et Cousins est pertinent pour répertorier les facteurs clés influençant la capacité à effectuer et à utiliser l’évaluation chez les OC. Des efforts supplémentaires seraient toutefois nécessaires pour améliorer la compatibilité du cadre conceptuel avec les pratiques et valeurs du milieu communautaire québécois.
Downloads
References
Adams, A. E., Nnawulezi, N. A., & Vandenberg, L. (2015). "Expectations to Change" (E2C): A Participatory Method for Facilitating Stakeholder Engagement With Evaluation Findings. American Journal of Evaluation, 36(2), 243-255. doi:10.1177/1098214014553787
Akintobi, T. H., Yancey, E. M., Daniels, P., Mayberry, R. M., Jacobs, D., &
Berry, J. (2012). Using Evaluability Assessment and Evaluation Capacity-building to Strengthen Community-based Prevention Initiatives. Journal of Health Care for the Poor and Underserved, 23(2), 33-48.
Alaimo, S. P. (2008). Nonprofits and evaluation: Managing expectations from the leader's perspective. New Directions for Evaluation, 2008(119), 73-92. doi:10.1002/ev.269
Andrews, A., Motes, P. S., Floyd, A., Flerx, V. C., & Fede, A. L.-D. (2005). Building Evaluation Capacity in Community-Based Organizations: Reflections of an Empowerment Evaluation Team. Journal of Community Practice, 13(4), 85-104. doi:10.1300/J125v13n04_06
Bourgeois, I. (2008). Understanding the dimensions of organizational evaluation capacity (Doctoral dissertation). Retrieved from Library and Archives Canada Database. (Accession No. 9780494483893).
Bourgeois, I., & Cousins, J. B. (2013). Understanding Dimensions of Organizational Evaluation Capacity. American Journal of Evaluation, 34(3), 299-319. doi:10.1177/1098214013477235
Bourgeois, I., Hotte, N., Simmons, L., & Osseni, R. (2016). Measuring evaluation capacity in Ontario public health units. Canadian Journal of Program Evaluation, 31(2), 165-183. doi:10.3138/cjpe.306
Bourgeois, I., Simmons, L., & Buetti, D. (2018). Building evaluation capacity in Ontario's public health units: promising practices and strategies. Public Health. doi:10.1016/j.puhe.2018.01.031
Bourgeois, I., Whynot, J., & Theriault, E. (2015). Application of an organizational evaluation capacity self-assessment instrument to different organizations: Similarities and lessons learned. Evaluation and Program Planning, 50, 47-55. doi:10.1016/j.evalprogplan.2015.01.004
Bourque, M., Grenier, J., Pelland, D., & St-Germain, L. (2016). L'action communautaire autonome, mouvance des contextes, place et autonomie des acteurs : quelles perspectives ? Sciences et actions sociales, 4, 1-20.
Carman, J. G. (2007). Evaluation Practice among Community-Based Organizations: Research into the Reality. American Journal of Evaluation, 28(1), 60-75. doi:10.1177/1098214006296245
Carman, J. G., & Fredericks, K. A. (2010). Evaluation Capacity and Nonprofit Organizations: Is the Glass Half-Empty or Half-Full? American Journal of Evaluation, 31(1), 84-104. doi:10.1177/1098214009352361
Carnochan, S., Samples, M., Myers, M., & Austin, M. J. (2014). Performance Measurement Challenges in Nonprofit Human Service Organizations. Nonprofit and Voluntary Sector Quarterly, 43(6), 1014-1032. doi:10.1177/0899764013508009
Cheverton, J. (2007). Holding Our Own: Value and Performance in Nonprofit Organisations. Australian Journal of Social Issues, 42(3), 427-436.
Cousins, J. B., Goh, S. C., Elliott, C., Aubry, T., & Gilbert, N. (2014). Government and voluntary sector differences in organizational capacity to do and use evaluation. Evaluation and Program Planning, 44, 1-13. doi:10.1016/j.evalprogplan.2013.12.001
Daniel, C. (1999). L'évaluation d'un organisme social et communautaire. Montréal: Centre de formation populaire.
Despard, M. R. (2016). Strengthening Evaluation in Nonprofit Human Service Organizations: Results of a Capacity-Building Experiment. Human Service Organizations Management Leadership & Governance, 40(4), 352-368. doi:10.1080/23303131.2016.1140101
Ducharme, É., Lesemann, F., Charbonneau, J., Martin, C., & Lefèvre, S. (2011). Les fondations et la « nouvelle philanthropie » : un changement de paradigmes scientifiques et politiques. Lien social et Politiques(65), 203-224.
Fonds, M. (2012). Milieu communautaire : à quoi sert l’évaluation ? À babord: revue sociale et politique. Retrieved from https://www.ababord.org/Milieu-communautaire-a-quoi-sert-l
Garcia-Iriarte, E., Suarez-Balcazar, Y., Taylor-Ritzler, T., & Luna, M. (2011). A Catalyst-for-Change Approach to Evaluation Capacity Building. American Journal of Evaluation, 32(2), 168-182. doi:10.1177/1098214010387114
Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies (Vol. 26, pp. 91-108). Oxford, UK.
Harper, G. W., Contreras, R., Bangi, A., & Pedraza, A. (2003). Collaborative Process Evaluation: Enhancing Community Relevance and Cultural Appropriateness in HIV Prevention. Journal of Prevention & Intervention in the Community, 26(2), 53-69. doi:10.1300/J005v26n02_05
Houlbrook, M. (2011). Critical perspectives on results-based accountability: Practice tensions in small community-based organizations. Third Sector Review, 17(2), 45-68.
Janzen, R., Ochocka, J., Turner, L., Cook, T., Franklin, M., & Deichert, D. (2017). Building a community-based culture of evaluation. Evaluation and Program Planning, 65, 163-170. doi:10.1016/j.evalprogplan.2017.08.014
Kapucu, N., Healy, B. F., & Arslan, T. (2011). Survival of the fittest: Capacity building for small nonprofit organizations. Evaluation and Program Planning, 34(3), 236-245. doi:10.1016/j.evalprogplan.2011.03.005
Kegeles, S. M., Rebchook, G. M., & Tebbetts, S. (2005). Challenges and Facilitators to Building Program Evaluation Capacity among Community-Based Organizations. AIDS Education and Prevention, 17(4), 284-299.
Kelly, C. M., LaRose, J., & Scharff, D. P. (2014). A Method for Building Evaluation Competency Among Community-Based Organizations. Health Promotion Practice, 15(3), 431-437. doi:10.1177/1524839913496427
Kitchenham, B. (2004). Procedures for Performing Systematic Reviews Keele: Keele University
Labin, S. N., Duffy, J. L., Meyers, D. C., Wandersman, A., & Lesesne, C. A. (2012). A Research Synthesis of the Evaluation Capacity Building Literature. American Journal of Evaluation, 33(3), 307-338. doi:10.1177/1098214011434608
Larose, F., Bédard, J., Couturier, Y., Grenon, V., Béland, S., Larivée, S., & Vincent, F. (2018). Étude évaluative des interventions éducatives et socioéducatives de La Relance, jeunes et familles. Rapport final (2014-2017) Version courte (sans annexes).
Lemay, J.-C., Leclair, L., Bélanger, S., & Messikh, D. (2013). Les particularités de l’évaluation pour les groupes communautaires : réflexions et enjeux. Bulletin de la Société québécoise d’évaluation de programme, 25(1), 8-10.
Lum, J., Evans, B., & Shields, J. (2016). Co-constructing Performance Indicators in Home and Community Care: Assessing the Role of Nongovernmental Organizations in Three Canadian Provinces. Canadian Journal of Nonprofit and Social Economy Research, 7(1), 46-67. doi:10.22230/cjnser.2016v7n1a215
Mayne, J. (2017). Building Evaluative Culture in Community Services: Caring for Evidence. Evaluation and Program Planning. doi:10.1016/j.evalprogplan.2017.05.011
McKegg, K., Wehipeihana, N., & Pipi, K. (2016). The Evaluation Capacity Building Project Developing an evaluation capacity assessment tool and process for New Zealand NGOs. Wellington: The Social Policy Evaluation and Research Unit’s Retrieved from http://www.superu.govt.nz/sites/default/files/Evaluation%20Capacity%20report.pdf.
Mitchell, L., & Fiellin, D. A. (2014). Assessing Research Activity and Capacity of Community-Based Organizations: Development and Pilot Testing of an Instrument. Progress in Community Health Partnerships: Research, Education, and Action, 8(4), 413-414. doi:10.1353/cpr.2014.0067
Pierson, J. (2008). Going local : working in communities and neighbourhoods. London : Routledge.
Poth, C., Lamarche, M., Yapp, A., Sulla, E., & Chisamore, C. (2014). Toward a Definition of Evaluation Within the Canadian Context: Who Knew This Would Be So Difficult? The Canadian Journal of Program Evaluation, 29(1), 87. doi:10.3138/cjpe.29.1.87
Potvin, P. (2010). Éléments et critères d'évaluabilité d'un programme d'intervention psychosociale. In M. Alain & D. Dessureault (Eds.), Élaborer et évaluer les programmes d’intervention psychosociale (pp. 195-216). Québec: Presses de l’Université du Québec.
Preskill, H. (2014). Now for the Hard Stuff: Next Steps in ECB Research and Practice. American Journal of Evaluation, 35(1), 116-119. doi:10.1177/1098214013499439
Preskill, H., & Boyle, S. (2008). A Multidisciplinary Model of Evaluation Capacity Building. American Journal of Evaluation, 29(4), 443-459. doi:10.1177/1098214008324182
SacAction. (2015). L'évaluation : pour aller plus loin! Québec: Gouvernement du Québec Retrieved from https://www.mtess.gouv.qc.ca/publications/sacaction/no81/historique.asp.
Schalock, R. L., Lee, T., Verdugo, M., Swart, K., Claes, C., van Loon, J., & Lee, C.-S. (2014). An evidence-based approach to organization evaluation and change in human service organizations evaluation and program planning. Evaluation and Program Planning, 45, 110-118. doi:10.1016/j.evalprogplan.2014.03.012
Sébastien, S., & Jean, P. (2012). Les organismes communautaires au Québec : De la coexistence à la supplémentarité. Canadian Journal of Nonprofit and Social Economy Research, 3(2), 24-32.
Service aux collectivités de l'UQAM. (2005). Analyse des pratiques d'évaluation dans les organismes communautaires : rapport de recherche. Montréal: Université du Québec à Montréal.
Simmons, L. (2015). Building Evaluation Capacity In Ontario’s Public Health Units: A Locally Driven Collaborative Project. Ontario: Eastern Ontario Health Unit.
Simmons, V. N., Klasko, L. B., Fleming, K., Koskan, A. M., Jackson, N. T., Noel-Thomas, S., . . . Gwede, C. K. (2015). Participatory evaluation of a community-academic partnership to inform capacity-building and sustainability. Evaluation and Program Planning, 52, 19-26. doi:10.1016/j.evalprogplan.2015.03.005
Sobeck, J. L. (2008). How cost-effective is capacity building in grassroots organizations? Administration in Social Work, 32(2), 49-68. doi:10.1300/J147v32n02_04
Société canadienne d'évaluation. (2010). Qu'est-ce que l'évaluation? Ottawa: Société canadienne d'évaluation Retrieved from https://evaluationcanada.ca/fr/quest-ce-que-levaluation.
Stockdill, S. H., Baizerman, M., & Compton, D. W. (2002). Toward a Definition of the ECB Process: A Conversation with the ECB Literature. New Directions for Evaluation(93), 1-25.
Weir, A., & Fouche, C. (2016). “Dancing with data”: Investing in capacity building for non-government organizations (NGOs). Evaluation Matters—He Take Tō Te Aromatawai, 2, 131-160.
Yu, S., & McLaughlin, D. (2013). Program Evaluation and Impact Assessment in International Non-Governmental Organizations (INGOs): Exploring Roles, Benefits, and Challenges. Canadian Journal of Nonprofit and Social Economy Research, 4(2), 23-38.
Yung, B., Leahy, P., Deason, L. M., Fischer, R. L., Perkins, F., Clasen, C., & Sharma, M. (2008). Capacity-building needs of minority health nonprofits. Evaluation and
Downloads
Published
Issue
Section
License
Copyright (c) 2019 Canadian journal of nonprofit and social economy research
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an original manuscript to the Canadian Journal of Nonprofit and Social Economy Research / Revue canadienne de recherche sur les OSBL et l'économie sociale [thereafter CJNSER] will be taken to mean that it represents original work not previously published, and that it is not being considered elsewhere for publication.
The journal takes the stance that the publication of scholarly research is meant to disseminate knowledge and, in a not-for-profit regime, benefits neither publisher nor author financially. It sees itself as having an obligation to its authors and to society to make content available online now that the technology allows for such a possibility. In keeping with this principle, the journal will publish all of its issues online.
Authors who publish in CJNSER agree to release their articles under the Creative Commons 4.0 International Licence (CC BY). This licence allows anyone to copy and distribute the article provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.
[Content published between 2010 and 2019-10 was licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 2.5 Canada (CC BY-NC-ND 2.5 CA) License. Content published between 2019-10 to 2023-12 was licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Content published after 2023-12 is licensed under the Creative Commons 4.0 International (CC BY) License.]
