Our Publications

Contribution Analysis in Policy Work: Assessing Advocacy’s Influence, November 2017. (co-authors: Carlisle Levine, Robin Kane, Claire Reinelt and Carlyn Orians)

Those involved in policy change struggle to determine whether or in what ways their advocacy efforts influence policy outcomes. Contribution analysis is a theory-based approach for assessing causal linkages that is well suited for addressing this kind of challenge. In this brief, the co-authors draw on their experience using contribution analysis to evaluate the impact of advocacy efforts to offer guidance to advocacy evaluators who would like to use a similar approach.

The Modernizing Foreign Assistance Network (MFAN) Evaluation Report: 2008-2016, July 2017. (co-authors: Carlisle Levine, Robin Kane and Claire Reinelt; other contributors: Ann Emery, Bruce Hoppe, and Marc Smith)

The Modernizing Foreign Assistance Network (MFAN) is a bipartisan coalition that bolsters the United States’ leadership in promoting global economic growth and reducing poverty and suffering by improving the effectiveness, efficiency, and transparency of U.S. foreign assistance. This evaluation studied MFAN’s achievements from mid-2008 through mid-2016. The MFAN Evaluation Report page contains the full evaluation report, executive summary, and four contribution analysis stories.

APC TIG Week: Carlisle Levine on Using the Most Significant Change Approach to Explore Potential Evidence of Impact in Advocacy and Policy Change, July 21, 2016.

Assessing advocacy impact is a challenging undertaking. This blog describes how the Most Significant Change approach can be used to explore potential evidence of advocacy impact.

Measuring International NGO Agency-Level Results, May 2016, InterAction. (co-authors: Carlisle Levine, Tosca Bruno van Vijfeijken and Sherine Jayawickrama)

Many international non-governmental organizations (NGOs) are under pressure – internal and external – to present evidence of their accomplishments at an organizational level. In response, they wrestle with whether or not to build agency-level measurement systems, what these might look like, whom they might serve, and what it will take to build and maintain them. This paper builds on the experiences of 17 US-based international NGOs specifically and the experiences of the broader international NGO community more generally to respond to these questions.

So, What Does It All Add Up To? Measuring International NGO Results at the Agency Level, May 2016, InterAction. (co-authors: Carlisle Levine, Tosca Bruno van Vijfeijken and Sherine Jayawickrama)

This executive brief summarizes key learning and recommendations from Measuring International NGO Agency-Level Results.

Saving Newborn Lives: Champions Toolkit. February 2016, Save the Children, (co-authors: Sarah Roma and Carlisle Levine)

Advocates draw on many strategies in their efforts to influence policy change. One important strategy is partnering with champions. These champions can be recognized experts in a particular field, journalists, government officials, members of parliament, religious leaders, celebrities and others. Most importantly, they have stature with decision makers and are committed to an advocacy cause. But how can organizations identify and successfully partner with champions, and how do they know that these partnerships are making a difference? This toolkit, designed primarily for country offices, provides guidance and a set of practical tools to help advocates strategically partner with a broad range of champions to influence policy change. The guidance covers when, with whom and how to partner; how to foster the partnership relationship; and how to know if that partnership is making a difference.

How To: A Local Ownership Approach To Evaluation In Practice, May 2015, InterAction. (co-authors: Carlisle Levine, Laia Griñó)

While Local Ownership in Evaluation: Moving from Participant Inclusion to Ownership in Evaluation Decision provides a rationale and identifies necessary conditions for embracing local ownership in evaluation, this “how to” document provides guidance for putting such an approach into practice.

Local Ownership in Evaluation: Moving from Participant Inclusion to Ownership in Evaluation Decision Making, February 2015, InterAction. (co-authors: Carlisle Levine, Laia Griñó)

Who should be involved in evaluation decision making? More specifically, what role should those who are intended to benefit from international assistance have in evaluation decision making processes? This paper provides a rationale and identifies necessary conditions for embracing local ownership in evaluation.

Embracing Evaluative Thinking For Better Outcomes: Four NGO Case Studies, June 2014, InterAction and CLEAR-Anglophone Africa. (co-editors: Laia Griño, Carlisle Levine, Stephen Porter, Gareth Roberts)

What is evaluative thinking? Why is it important? And how can organizations put it into practice? This study provides an introduction to evaluative thinking, and then presents case studies capturing the experiences of four international non-governmental organizations (NGOs) using evaluative thinking at organizational, program and project levels.

AEA365: APC TIG Week: Carlisle Levine on Using Contribution Analysis to Explore Causal Relationships in Advocacy and Policy Change, March 13, 2014.

Assessing advocacy impact is a challenging undertaking. This blog describes how contribution analysis can be used to explore potential evidence of advocacy impact.

AEA365: IC Week: Carlisle Levine on Finding Community as an Independent Evaluator, October 29, 2013.

For independent evaluators, finding ways to build community with colleagues is important. This blog describes why, and then offers tips for finding community and building collaborative working relationships.

AEA365: APC Week: Carlisle Levine on Useful Tools for Assessing Advocacy Progress, February 4, 2013.

Advocacy effectiveness is inherently challenging to measure, given the number of factors that influence policy change. This blog describes some of those challenges and then points to a few resources that can help evaluators address this challenge.

Catholic Relief Services’ (CRS) Guidance for Developing Logical and Results Frameworks, Catholic Relief Services, Baltimore, MD, 2007. (author: Carlisle Levine). (translated into French, Portuguese and Spanish).

This document was primarily written to provide guidance for conceptualizing, writing, selecting and measuring project performance indicators. In this document, that guidance is placed within the larger context of articulating theories of change, conceptualizing and writing objective and results statements (taken largely from Catholic Relief Services’ ProPack, with slight adjustments) and creating logical and results frameworks.