Our Publications

Executive Brief - On Equity in Contracting with Small and Solo Local Evaluation Firms: An Open Letter to Funders, October 2023. (co-authors: Carlisle Levine, BLE Solutions, and Hippolyt Pul, Institute for Peace and Development)

The Institute for Peace and Development (Ghana) and BLE Solutions (USA) recognize that, as funders increasingly seek to support locally-owned development, small and solo local evaluation firms have an important role to play in guiding learning and reflection. Yet, often funders' systems, structures, policies, and practices are not well designed for partnerships between the two. Through our open letter, we invite funders to reflect on small and solo local evaluation firms' strengths, as well as the challenges they face, and then to consider some suggestions we offer that might help both more successfully partner together to improve people's lives.

On Equity in Contracting with Small and Solo Local Evaluation Firms: An Open Letter to Funders, October 2023. (co-authors: Carlisle Levine, BLE Solutions, and Hippolyt Pul, Institute for Peace and Development)

The Institute for Peace and Development (Ghana) and BLE Solutions (USA) recognize that, as funders increasingly seek to support locally-owned development, small and solo local evaluation firms have an important role to play in guiding learning and reflection. Yet, often funders' systems, structures, policies, and practices are not well designed for partnerships between the two. Through our open letter, we invite funders to reflect on small and solo local evaluation firms' strengths, as well as the challenges they face, and then to consider some suggestions we offer that might help both more successfully partner together to improve people's lives.

Nurturing and Sustaining Facilitator Networks: Key Considerations for Support Organizations, September 2021. (co-authors: Ruben Grangaard and Carlisle Levine, BLE Solutions)

Effective and sustainable peacebuilding must be locally owned, and national and local facilitator networks can be important contributors to locally owned peacebuilding efforts. This report highlights findings from a meta-review that explored the experiences of facilitator networks supported by the United States Institute of Peace (USIP) in Afghanistan, Colombia, Iraq, Pakistan, and Tunisia. It examines what made those facilitator networks successful, the challenges they faced, and the external supports that benefited them. Contributors to that meta-review included Carlisle Levine (BLE Solutions, USA), Dlawer Ala’Aldeen (Middle East Research Institute, Iraq), Vanessa Corlazzoli (independent, Canada), Eya Jrad (independent, Tunisia), Diana Maria Montealegre (independent, Colombia), Hassan Nasir Mirbahar (independent, Pakistan), and Mohammed Ehsan Zia (Tadbeer Consultancy and Research Organization, Afghanistan, at the time of the review).

Contribution Analysis in Policy Work: Assessing Advocacy’s Influence, November 2017. (co-authors: Carlisle Levine, Robin Kane, Claire Reinelt and Carlyn Orians)

Those involved in policy change struggle to determine whether or in what ways their advocacy efforts influence policy outcomes. Contribution analysis is a theory-based approach for assessing causal linkages that is well suited for addressing this kind of challenge. In this brief, the co-authors draw on their experience using contribution analysis to evaluate the impact of advocacy efforts to offer guidance to advocacy evaluators who would like to use a similar approach.

The Modernizing Foreign Assistance Network (MFAN) Evaluation Report: 2008-2016, July 2017. (co-authors: Carlisle Levine, Robin Kane and Claire Reinelt; other contributors: Ann Emery, Bruce Hoppe, and Marc Smith)

The Modernizing Foreign Assistance Network (MFAN) is a bipartisan coalition that bolsters the United States’ leadership in promoting global economic growth and reducing poverty and suffering by improving the effectiveness, efficiency, and transparency of U.S. foreign assistance. This evaluation studied MFAN’s achievements from mid-2008 through mid-2016. The MFAN Evaluation Report page contains the full evaluation report, executive summary, and four contribution analysis stories.

APC TIG Week: Carlisle Levine on Using the Most Significant Change Approach to Explore Potential Evidence of Impact in Advocacy and Policy Change, July 21, 2016.

Assessing advocacy impact is a challenging undertaking. This blog describes how the Most Significant Change approach can be used to explore potential evidence of advocacy impact.

Measuring International NGO Agency-Level Results, May 2016, InterAction. (co-authors: Carlisle Levine, Tosca Bruno van Vijfeijken and Sherine Jayawickrama)

Many international non-governmental organizations (NGOs) are under pressure – internal and external – to present evidence of their accomplishments at an organizational level. In response, they wrestle with whether or not to build agency-level measurement systems, what these might look like, whom they might serve, and what it will take to build and maintain them. This paper builds on the experiences of 17 US-based international NGOs specifically and the experiences of the broader international NGO community more generally to respond to these questions.

So, What Does It All Add Up To? Measuring International NGO Results at the Agency Level, May 2016, InterAction. (co-authors: Carlisle Levine, Tosca Bruno van Vijfeijken and Sherine Jayawickrama)

This executive brief summarizes key learning and recommendations from Measuring International NGO Agency-Level Results.

Saving Newborn Lives: Champions Toolkit. February 2016, Save the Children, (co-authors: Sarah Roma and Carlisle Levine)

Advocates draw on many strategies in their efforts to influence policy change. One important strategy is partnering with champions. These champions can be recognized experts in a particular field, journalists, government officials, members of parliament, religious leaders, celebrities and others. Most importantly, they have stature with decision makers and are committed to an advocacy cause. But how can organizations identify and successfully partner with champions, and how do they know that these partnerships are making a difference? This toolkit, designed primarily for country offices, provides guidance and a set of practical tools to help advocates strategically partner with a broad range of champions to influence policy change. The guidance covers when, with whom and how to partner; how to foster the partnership relationship; and how to know if that partnership is making a difference.

How To: A Local Ownership Approach To Evaluation In Practice, May 2015, InterAction. (co-authors: Carlisle Levine, Laia Griñó)

While Local Ownership in Evaluation: Moving from Participant Inclusion to Ownership in Evaluation Decision provides a rationale and identifies necessary conditions for embracing local ownership in evaluation, this “how to” document provides guidance for putting such an approach into practice.

Local Ownership in Evaluation: Moving from Participant Inclusion to Ownership in Evaluation Decision Making, February 2015, InterAction. (co-authors: Carlisle Levine, Laia Griñó)

Who should be involved in evaluation decision making? More specifically, what role should those who are intended to benefit from international assistance have in evaluation decision making processes? This paper provides a rationale and identifies necessary conditions for embracing local ownership in evaluation.

Embracing Evaluative Thinking For Better Outcomes: Four NGO Case Studies, June 2014, InterAction and CLEAR-Anglophone Africa. (co-editors: Laia Griño, Carlisle Levine, Stephen Porter, Gareth Roberts)

What is evaluative thinking? Why is it important? And how can organizations put it into practice? This study provides an introduction to evaluative thinking, and then presents case studies capturing the experiences of four international non-governmental organizations (NGOs) using evaluative thinking at organizational, program and project levels.

AEA365: APC TIG Week: Carlisle Levine on Using Contribution Analysis to Explore Causal Relationships in Advocacy and Policy Change, March 13, 2014.

Assessing advocacy impact is a challenging undertaking. This blog describes how contribution analysis can be used to explore potential evidence of advocacy impact.

AEA365: IC Week: Carlisle Levine on Finding Community as an Independent Evaluator, October 29, 2013.

For independent evaluators, finding ways to build community with colleagues is important. This blog describes why, and then offers tips for finding community and building collaborative working relationships.

AEA365: APC Week: Carlisle Levine on Useful Tools for Assessing Advocacy Progress, February 4, 2013.

Advocacy effectiveness is inherently challenging to measure, given the number of factors that influence policy change. This blog describes some of those challenges and then points to a few resources that can help evaluators address this challenge.

Catholic Relief Services’ (CRS) Guidance for Developing Logical and Results Frameworks, Catholic Relief Services, Baltimore, MD, 2007. (author: Carlisle Levine). (translated into French, Portuguese and Spanish).

This document was primarily written to provide guidance for conceptualizing, writing, selecting and measuring project performance indicators. In this document, that guidance is placed within the larger context of articulating theories of change, conceptualizing and writing objective and results statements (taken largely from Catholic Relief Services’ ProPack, with slight adjustments) and creating logical and results frameworks.