Can outcomes, monitoring and evaluation planning be standardized?

I was involved in an interesting discussion recently with a group of evalutors about whether outcomes, monitoring and evaluation planning can be standardized. In my experience, much evaluation planning starts from a blank slate with evaluators and project staff sitting around wondering about how they’re going to evaluate a specific program. Or, in other cases, for budgetary or other reasons, people who are not trained in evaluation have to work their way through basic texts about evaluation trying to work out how to do an evaluation. This all takes a great deal of time. Does it have to be like this? I’m not sure that it does.

Every time an organization sets up an accounting system, you don’t get the feeling that accountants have to build the entire accounting system from scratch. They simply put in place a number of basic building blocks of such systems and tailor them to the requirements of the particular organization. Why should monitoring and evaluation be any different? What I’ve been trying to do over a number of years in developing Systematic Outcomes Analysis is to develop such a standardized approach.

New outcomes modeling standards released

These days many people are involved in drawing outcomes models for organizations or projects. They go by various names – intervention logics, program logics, causal chains, results chains, program theories, program theories of action. I get to look at a lot of them and, to be frank, many of the ones I see are problematic. They’re either all over the place, or in many cases they are so limited (by limiting them to measurable and attributable outcomes – see later in this post) that they’re almost useless for most purposes. Don’t get me wrong, there’re also people out there drawing good models, but its often in spite of the received wisdom about drawing such models, rather than being because of it.

The problem with the bad outcomes models is that many of them are drawn using arbitrary rules which severely limit their usefulness. For instance in some cases, only three levels are allowed within such models (outputs, intermediate outcomes, final outcomes). While perhaps some simple programs can be modeled within this constraint (I suspect that there are few programs which can), increasingly I’ve come to think that it’s often crazy to limit people in this way when they are drawing outcomes models. I’ll be saying why in detail in later blog postings.