Putting the Planning back into M&E – PME or PM&E what’s the acronym going to be?

In a posting on Linkedin, Leslie Ayre-Jaschke talked about the growth of PME – or maybe it will end up being called PM&E, or something else. Regardless of the acronym, it’s the movement to put planning back into monitoring and evaluation. ‘Putting the P back into M&E’ was the subtitle of a workshop I ran in South Africa for UNFPA several years ago. I think that it’s a concept that’s going to get a lot more traction over the next few years.

It’s consistent with what evaluators like Michael Patton, and many of us in the evaluation community, have been talking about for years. We’ve been talking up the key role of formative evaluation – evaluation aimed at making sure that programs are optimized. And formative evaluation is all about making sure that programs are well planned.

The point of this approach within evaluation is that it’s often pointless to evaluate a badly planned program. Evaluation resources would be better spent on making sure that the program is better planned than on measuring the fact that it often will not achieve its outcomes due to the fact that planning has been poor.

The new PM&E movement is not just about evaluators and evaluation, it is much broader than that taking in people from a range of disciplines. This new integrated approach which is emerging needs an underlying theory which will appeal to all of the different disciplines involved – strategic planners, performance managers, evaluators, contract managers, policy analysts etc. The work I’ve been doing in outcomes theory has been designed to meet this need.

The purpose of outcomes theory is to provide an integrated conceptual basis for PM&E-type approaches. A common conceptual basis is needed if people across the different disciplines and sectors are going to be able to share conceptual insights about how they identify, measure, attribute and hold parties to account for outcomes when doing planning, monitoring and evaluation. Good theory is needed to help them quickly sort out the type of conceptual confusion that current characterizes much of the discussion of outcomes related issues. As the famous social scientist Kurt Lewin said – ‘there’s nothing so practical as a good theory’.

This aspiration of outcomes theory is summarized in the diagram below showing how it’s a meso-level theory reaching across strategic planning, monitoring, evaluation etc.

(see http://www.outcomescentral.org/outcomestheory.html for more on this)

For people just working out in the field, who don’t  need to know much theory, outcomes theory principles have been hard-wired into the DoView Visual Planning, Monitoring and Evaluation approach http://doview.com/plan. Using the approach means that they will avoid many of the technical problems which are highlighted by outcomes theory.

Large-scale visual models of a program (drawn in the correct way, for instance as ‘DoViews’) provide the ideal foundation for the new fully integrated approach to planning, monitoring and evaluation which many are now seeking. http://doview.com/plan/draw.html.

The importance of 'looking behind the numbers' in performance management systems

A colleague Stan Capela recently highlighted the importance of ‘looking behind the numbers’ in performance management systems. Pointing out that, if this is not done, false conclusions can be drawn from such systems. I think that most people would agree with this sentiment. The key issue for me is what is the most effective way of us ‘looking behind the numbers’ when measuring people’s, project’s or organization’s performance.

New How-To Guides on DoView Site – What's and outcomes (results) model

I have not been blogging for a while as I’ve been caught up in preparing multiple resources on outcomes models and also actually developing many outcomes models for clients. I now have many great examples which I want to share with you in the coming months. It’s only now that a number of these projects are coming to a conclusion and clients are becoming willing to share them with others. So watch this space.

In the meantime, on the DoView site some new How-To Guides are starting to be put up. The first is on What’s a DoView Outcomes (Results) Model and Why Should I Use One? This is in response to requests from DoView enthusiasts who want to be able to refer people to a quick article about what an outcomes model is and why people should use one for all of their project and organizational planning.

The evolution of the logic model

I’ve just posted an article on the evolution of the logic model within evaluation. Over the last couple of decades, increasing numbers of evaluators have started using logic models. For those not familiar with what logic models are – they are simply tabular or visual representations of all of the lower-level steps needed to achieve high-level outcomes for a program, organization or other intervention. They go by different names, for instance: program logics, intervention logics, results maps, theories of change, program theories, results hierarchies, strategy maps, end-means diagrams etc.). A traditional way of drawing logic models has evolved (known as the inputs, outputs, intermediate outcomes, final outcomes structured logic model) which often attempts to restrict logic models to a single page. However, many evaluators are now breaking away from the constraints of this traditional format and exploring various alternative ways of representing logic models.

How many evaluators does it take to change a light bulb?

In response to a series of ‘How many evaluators does it take to change a light bulb?’ jokes on the evaluators list EVALTALK, I whipped up an outcomes model (logic model) for a Changing Light Bulbs Project (some days one does wonder if this is what evaluators do for fun, it must be some sort of illness!).

Anyway here it is http://www.outcomesmodels.org/models/lightbulb62.html

Paul Duignan, PhD. (Follow me on my Outcomes Blog; Twitter; or via my E-Newsletter).

Flow of causality in outcomes models and feedback loops

A quick technical blog here. Fellow evaluator Rick Davies pointed out in a post on one of my outcomes theory articles (on how to best represent causal models), that strictly visualizing causality as flowing in one direction within an outcomes model (logic model, results map, logframe, theory of change etc.)  could be seen as preventing the representation of feedback loops. This is because if you are, as I usually do, representing causality as flowing from bottom to top within a model (others do it left to right) then when you want to represent a feedback loop it will, of necessity, have to flow back down the logic model against the direction in which causality is being represented.

The good old 'one pager' contraint again

I’ve been up against the good old ‘one pager’ constraint again in a couple of contexts recently. This is where there is the demand that the outcomes for an organization or program be ‘put on one page’. It comes in various forms and is often a demand from senior management or a perceived demand from them to ‘keep things simple’. Now, there’s nothing wrong with the idea of summarizing things and paper overviews play a role in that. But such things should be seen as one of a range of different types of summaries and products which are produced by an underlying outcomes system, not the beginning and end of an outcomes effort.

Separating analysis from writing in philanthropic grant applications

The other night I was doing some pro bono work with a small non-profit organization which is attempting to restore a significant landmark building and promote its use for educational and community purposes. I was building a DoView results map (outcomes model) for them in real-time in the course of one of their monthly meetings. They’re using the visual model as their strategic planning approach (instead of the traditionally long narrative text-based plans many people in such small organizations spend many hours sweating over).

Building a DoView model is a very quick way for a community organization to build a simple but effective strategic plan. Once built they can then show others the model and quickly communicate that they’ve thought through what they’re planning to do.

Visual model of what I'm trying to do with my outcomes work

Thought that I would apply a taste of my own medicine to my own work, so I drew a visual outcomes model of what it is that I’m trying to do with my work in the outcomes area. It is here. At the top is my high level outcome ‘Make working with outcomes, monitoring, evaluation etc. easier’ and below that is all of the lower-level steps I am using to get to this high-level outcome. I have included hyperlinks out to the various web sites where I am attempting to do the things listed in the lower-level steps.

Paul Duignan, PhD

Outcomes and Evaluation Blog (OutcomesBlog.org)

Making outcomes theory more concrete – checklist for assessing outcomes systems

Most normal people would think that it’s very very obscure, but I’ve just put up a Checklist for Analyzing Outcomes Systems in the Outcomes Theory Knowledge Base and it’s a very exciting development. Up until now the Outcomes Theory Knowledge Base has consisted of a set of articles which outline various aspects of outcomes theory. Outcomes theory is a general theory which covers all types of outcomes systems. Outcomes systems are any type of performance management system, results-base system, monitoring system, evaluation system, outcomes-focused contracting system, or strategic planning system (the term even includes evidence-based practice systems). Such systems have, in the past, been seen as somewhat different types of things without a common theory existing to analyze them. Outcomes theory is based on the insight that we can theorize them as a common type of system and then use the theory to work out how such systems should be best structured. This approach becomes powerful at the moment that we can start applying it to actual real-world outcomes systems. This is the role of the checklist I’ve just developed.