W.K. Kellogg Foundation Logic Model Development Guide

W.K. Kellogg Foundation

Logic Model Development Guide

Using Logic Models to Bring Together Planning, Evaluation, and Action

Logic Model Development Guide

To help people help themselves through the practical

application of knowledge and resources to improve

their quality of life and that of future generations.

Updated January 2004

W.K. Kellogg Foundation

One East Michigan Avenue East

Battle Creek, Michigan 49017-4058

www.wkkf.org

To receive additional copies of the Logic Model Development Guide,

please call 1/800/819-9997 and request item #1209.

W.K. KELLOGG

FOUNDATION

Introduction ..........................................................................................................III

Chapter 1

Introduction to Logic Models ........................................................................................1

The What and Why of the Logic Model ........................................................................1

Logic Model Definition ............................................................................................1

Logic Model Purpose ................................................................................................3

Trip Planning Logic Model Example..........................................................................3

Why Use a Logic Model?................................................................................................5

Program Success ..........................................................................................................5

Program Investments ..................................................................................................6

Simple Logic Model Basics ............................................................................................7

Logic Model Development..........................................................................................7

Reading a Logic Model ..............................................................................................7

Other Logic Model Examples ........................................................................................8

Theory Model ..........................................................................................................10

Outcomes Model ......................................................................................................11

Activities Model ........................................................................................................12

Chapter 2

Developing a Basic Logic Model for Your Program ......................................................15

Demonstrating Progress Toward Change ......................................................................16

Exercise 1 – Describing Results ....................................................................................16

Exercise 1 Checklist......................................................................................................20

Exercise 2 – Describing Actions....................................................................................21

Exercise 2 Checklist......................................................................................................23

Program Implementation Template – Exercise 1 & 2 ..................................................25

Chapter 3

Developing a Theory-of-Change Logic Model for Your Program..................................27

Exercise 3 - Constructing a Program Theory ................................................................28

Program Planning ......................................................................................................28

Exercise 3 Checklist....................................................................................................33

Program Planning Template – Exercise 3....................................................................34

Logic Model Development GuidePage I

Contents

Chapter 4

Using Your Logic Model to Plan for Evaluation ..........................................................35

Exercise 4 – Posing Evaluation Questions ....................................................................35

Formative/Summative Evaluation Questions ............................................................35

Evaluation Vantage Points Context, Implementation, Outcomes ..............................36

Focus Areas, Audiences, Questions, Information Use ................................................38

Audiences and Evaluation..........................................................................................42

Exercise 4 Checklist ..................................................................................................43

Evaluation Planning Template – Exercise 4 ..................................................................44

Exercise 5 – Establishing Indicators..............................................................................45

Indicators of Success..................................................................................................45

Exercise 5 Checklist ..................................................................................................47

Indicators Development Template – Exercise 5 ........................................................48

Resource Appendix ........................................................................................49

Forms Appendix ................................................................................................53

Logic Model Development GuidePage II

Introduction

If you don’t know where you’re going, how are you gonna’ know when you get there?

–Yogi Berra

In line with its core mission – To help people help themselves through the practical application of knowledge

and resources to improve their quality of life and that of future generations – the W.K. Kellogg Foundation has

made program evaluation a priority. As our staff and grantees work on a spectrum of social improve-

ment programs, the need for shaping and contributing to the body of knowledge regarding evaluation

becomes increasingly clear. Our first guide, the W.K. Kellogg Foundation Evaluation Handbook, was pub-

lished in 1998, and has been made available to nearly 7,500 people.The Evaluation Handbook is a prac-

tical, step-by-step manual for conducting evaluations.With the Handbook, we introduced the concept

of the program logic model and the ways in which applying this concept has added value to our

own work.

The program logic model is defined as a picture of how your organization does its work – the theory and

assumptions underlying the program. A program logic model links outcomes (both short- and long-term)

with program activities/processes and the theoretical assumptions/principles of the program.

The W.K. Kellogg Foundation Logic Model Development Guide, a companion publication to the Evaluation

Handbook, focuses on the development and use of the program logic model.We have found the logic

model and its processes facilitate thinking, planning, and communications about program objectives and

actual accomplishments.Through this guide, we hope to provide an orientation to the underlying prin-

ciples and language of the program logic model so it can be effectively used in program planning,

implementation, and dissemination of results.

The premise behind this guide – and our view of the role of evaluation in programming – is simple:

Good evaluation reflects clear thinking and responsible program management. Over the years, our

experience in using logic models in initiatives such as the Kellogg Youth Initiative Partnerships,

Devolution, ENLACE (Engaging Latino Communities for Education), and the Native American

Higher Education Initiative, to name just a few, has provided ample evidence of the effectiveness of

these methods.

Learning and using tools like logic models can serve to increase the practitioner’s voice in the domains

of planning, design, implementation, analysis, and knowledge generation.The process of developing the

model is an opportunity to chart the course. It is a conscious process that creates an explicit under-

standing of the challenges ahead, the resources available, and the timetable in which to hit the target. In

addition, it helps keep a balanced focus on the big picture as well as the component parts.

In general, logic modeling can greatly enhance the participatory role and usefulness of evaluation as a

management and learning tool. Developing and using logic models is an important step in building

community capacity and strengthening community voice.The ability to identify outcomes and antici-

pate ways to measure them provides all program participants with a clear map of the road ahead. Map

in hand, participants are more confident of their place in the scheme of things, and hence, more likely

to actively engage and less likely to stray from the course – and when they do, to do so consciously

and intentionally. Because it is particularly amenable to visual depictions, program logic modeling can

be a strong tool in communicating with diverse audiences – those who have varying world views and

different levels of experience with program development and evaluation.

Logic Model Development GuidePage III

Introduction

The Logic Model Development Guide contains four chapters and two comprehensive appendices.

Chapter 1 presents a basic introduction to the logic model as an action-oriented tool for program

planning and evaluation. It also offers an array of sample logic models.

Chapter 2 consists of exercises and examples focused on the development of a simple program logic

model. Exercises include practical examples, checklists for reviewing content quality, and a template

for developing a logic model.

Chapter 3 gives instructions on how to expand a basic logic model to explore and explain the theory-

of-change that describes the rationale for your program. A template and checklist are provided.

Chapter 4 offers two exercises that afford the reader with an introduction to how the basic logic

modeling techniques introduced in the previous chapters can be applied to inform thinking about

what should be included in an evaluation plan.Templates and checklists are also provided.

The Resources Appendix provides logic model development resources – references and Web sites worth

visiting.The Forms Appendix includes blank templates to copy when developing your own logic models.

Acknowledgements

This work builds on the experience of many at the W.K. Kellogg Foundation who pioneered the appli-

cation of logic modeling to their initiatives. For example, logic models were first used with the Kellogg

Youth Initiative Partnerships (KYIP). In this application, the models were instrumental in helping staff

establish program direction, implementation, an evaluation framework, and outcomes across three sites.

In KYIP, logic modeling was used to facilitate and guide the development of the specific assumptions

and processes that ultimately led to the transition of the initiative from a WKKF-operated program to a

community-owned program.WKKF program staff, including Tyrone Baines, Phyllis Meadows, Gerald

Smith, Judy Watson Olson, Steve Peffers, Joyce Brown, and John Seita were instrumental in these efforts.

Our work in developing the Logic Model Development Guide began at the request of Kellogg Foundation

Program Director Blas Santos who expressed a need for user-friendly tools and processes to support the

work of grantees in Latin America and the Caribbean.

The Logic Model Development Guide represents a collaborative effort.We particularly want to acknowl-

edge the efforts of the Kellogg Foundation’s former director of evaluation, Ricardo Millett, and his

team of evaluation managers, including Astrid Hendricks-Smith and Mark Lelle, who have since left

the organization.Their tireless work among staff and grantees continues to promote the use of logic

models to plan, design, and manage initiatives. Dale Hopkins and Karin Ladley were instrumental in

bringing the material to print.We also wish to acknowledge the work of the Kellogg Foundation Vice

Presidents of Programs Rick Foster, Gail McClure, Dan Moore, and Gloria Smith, along with Senior

Vice President of Programs Anne Petersen, who have underscored the importance of evaluation,

embraced the logic model approach, and adopted it as a valued program support tool.

Special thanks are extended to Cynthia Phillips, a primary writer and consultant throughout the

development of this guide, and Work Volk Consultants, LLP, for formatting and editorial assistance.

Thanks, also, to Beverly Parsons of In Sites; Andrew Hahn and the students at the Florence Heller

Graduate School for Advanced Studies in Social Welfare, Brandeis University; Marc Osten, Summit

Consulting Collaborative; Sally Bond,The Program Evaluation Group; Joel Meister and Eva Moya,

University of Arizona; Amy Coates-Madsen and staff at Maryland Association of Nonprofit

Organizations; and Gail Randall, Greater Worchester Community Foundation.

–The Program Staff of the W.K. Kellogg Foundation

Logic Model Development GuidePage IV

Introduction to Logic Models

Chapter One defines logic models and explains their usefulness to program stakeholders. You

will learn the relevance of this state-of-the-art tool to program planning, evaluation, and

improvement.

ffective program evaluation does more than collect, analyze, and provide data. It

makes it possible for you – program stakeholders – to gather and use information,

to learn continually about and improve programs that you operate in or fund.

The W.K. Kellogg Foundation believes evaluation – especially program logic model

approaches – is a learning and management tool that can be used throughout a program’s

life – no matter what your stake in the program. Using evaluation and the logic model

results in effective programming and offers greater learning opportunities, better docu-

mentation of outcomes, and shared knowledge about what works and why. The logic

model is a beneficial evaluation tool that facilitates effective program planning, imple-

mentation, and evaluation.

The What and Why of the Logic Model

The WHAT: Logic Model Definition

Basically, a logic model is a systematic and visual way to present and share your under-

standing of the relationships among the resources you have to operate your program, the

activities you plan, and the changes or results you hope to achieve.

Figure 1. The Basic Logic Model.

The most basic logic model is a picture of how you believe your program will work. It uses

words and/or pictures to describe the sequence of activities thought to bring about change

and how these activities are linked to the results the program is expected to achieve.

Logic Model Development GuidePage 1

A program logic model is a

picture of how your pro-

gram works – the theory

and assumptions underly-

ing the program. ...This

model provides a road map

of your program, high-

lighting how it is expected

to work, what activities

need to come before others,

and how desired outcomes

are achieved (p. 35).

W.K. Kellogg

Foundation Evaluation

Handbook (1998)

Chapter 1

Resources/

Inputs

Activities Outputs Outcomes Impact

1 5432

Your Planned Work Your Intended Results

The Basic Logic Model components shown in Figure 1 above are defined below. These

components illustrate the connection between your planned work and your intended results.

They are depicted numerically by steps 1 through 5.

YOUR PLANNED WORK describes what resources you think you need to implement

your program and what you intend to do.

1. Resources include the human, financial, organizational, and community resources a

program has available to direct toward doing the work. Sometimes this component

is referred to as Inputs.

2. Program Activities are what the program does with the resources. Activities are the

processes, tools, events, technology, and actions that are an intentional part of the program

implementation. These interventions are used to bring about the intended program

changes or results.

YOUR INTENDED RESULTS include all of the program’s desired results (outputs, out-

comes, and impact).

3. Outputs are the direct products of program activities and may include types, levels and

targets of services to be delivered by the program.

4. Outcomes are the specific changes in program participants’ behavior, knowledge, skills,

status and level of functioning. Short-term outcomes should be attainable within 1 to 3

years, while longer-term outcomes should be achievable within a 4 to 6 year timeframe.

The logical progression from short-term to long-term outcomes should be reflected in

impact occurring within about 7 to 10 years.

5. Impact is the fundamental intended or unintended change occurring in organizations,

communities or systems as a result of program activities within 7 to 10 years. In the cur-

rent model of WKKF grantmaking and evaluation, impact often occurs after the conclu-

sion of project funding.

The term logic model is frequently used interchangeably with the term program theory in

the evaluation field. Logic models can alternatively be referred to as theory because they

describe how a program works and to what end (definitions for each employed by leading

evaluation experts are included in the Resources Appendix).

The What: How to “Read” a Logic Model

When “read” from left to right, logic models describe program basics over time from

planning through results. Reading a logic model means following the chain of reasoning

or “If...then...” statements which connect the program’s parts. The figure below shows

how the basic logic model is read.

Logic Model Development GuidePage 2

Most of the value in a logic

model is in the process of

creating, validating, and

modifying the model …

The clarity of thinking

that occurs from building

the model is critical to the

overall success of the pro-

gram (p. 43).

W.K. Kellogg Foundation

Handbook (1998)

Chapter 1

Figure 2. How to Read a Logic Model.

The WHY: Logic Model Purpose and Practical Application

The purpose of a logic model is to provide stakeholders with a road map describing the

sequence of related events connecting the need for the planned program with the pro-

gram’s desired results. Mapping a proposed program helps you visualize and understand

how human and financial investments can contribute to achieving your intended program

goals and can lead to program improvements.

A logic model brings program concepts and dreams to life. It lets stakeholders try an idea on

for size and apply theories to a model or picture of how the program would function. The

following example shows how the logic model approach works. (If you are familiar with logic

models, you may wish to skip ahead to the section entitled “Why Use A Logic Model?”)

Logic Model Development GuidePage 3

Sample Factors

influencing the trip:

• Family members’ school

and work schedules

• The holidays

• Winter weather

• Frequent Flier availability

Sample Activities:

• Creating/checking family

schedules

• Gathering holiday flight

and FF information

• Getting airport

transportation

• Notifying Iowa relatives

An Example:

We are proposing an inexpensive family trip from Charleston, South Carolina, to Des Moines, Iowa,

to visit relatives during December school holidays. The seasonal trip we dream of taking from

Charleston to Des Moines is the “program.” Basic assumptions about our trip “program” are:

• We want to visit relatives between 12/10/00 and 1/5/01 while the children are out of school.

• We will fly from South Carolina to Iowa because it takes less time than driving and because

frequent flier (FF) miles are available.

• Using frequent flier miles will reduce travel costs.

We have to determine the factors influencing our trip, including necessary resources, such as, the

number of family members, scheduled vacation time, the number of frequent flier miles we have,

round trip air reservations for each family member, and transportation to and from our home to

the airport. The activities necessary to make this happen are the creation of our own family holiday

schedule, securing our Iowa relative’s schedule, garnering air line information and reservations

and planning for transportation to and from the airport.

Resources/

Inputs

Activities Outputs Outcomes Impact

Certain

resources are

needed to

operate your

program

If you have

access to

them, then you

can use them

to accomplish

your planned

activities

If you

accomplish

your planned

activities, then

you will

hopefully deliver

the amount of

product and/or

service that

you intended

If you

accomplish

your planned

activities to the

extent you

intended, then

your participants

will benefit in

certain ways

If these

benefits to

participants are

achieved, then

certain changes

in organizations,

communities,

or systems

might be

expected to

occur

1 5432

Your Planned Work Your Intended Results

In this example, the results of our activities – or outputs – are mostly information, such as

family schedules, flight schedules, and cost information based on the time frame of the trip.

This information helps identify outcomes or immediate goals. For instance, if we make

reservations as soon as possible, we are able to find flights with available frequent flier slots

and probably have more options for flights that fit within the time frame. Knowing this,

our outcomes improve – reservations made well in advance result in flight schedules and

airline costs that suit our timeline and

travel budget. Longer-term impact of our trip is not

an issue here, but might be projected as continued good family relationships in 2010.

Using a simple logic model as a trip-planning tool produced tangible benefits. It helped

us gather information to influence our decisions about resources and allowed us to meet

our stated goals. Applying this process consistently throughout our trip planning posi-

tions us for success by laying out the best course of action and giving us benchmarks for

measuring progress – when we touch down in Charlotte and change planes for

Cincinnati, we know we’re on course for Des Moines.

Typical logic models use table and flow chart formats like those presented here to cata-

logue program factors, activities, and results and to illustrate a program’s dimensions.

Most use text and arrows or a graphic representation of program ideas. This is what our

trip planning “program” could look like in logic model format.

It was easy to organize travel plans in a flow chart, but we could also choose to organize and

display our thinking in other ways. A logic model does not have to be linear. It may appear

as a simple image or concept map to describe more complex program concepts. Settling on

a single image of a program is sometimes the most difficult step for program stakeholders.

Logic Model Development GuidePage 4

You can’t do “good” evalua-

tion if you have a poorly

planned program.

Beverly Anderson Parsons

(1999)

Resources/

Inputs

Activities Outputs Outcomes Impact

Holiday

flight

schedules

Family

schedules

Frequent

flyer

holiday

options

Holiday

weather

Create

family

schedule

Get

holiday

flight info

Get

tickets

Arrange

ground

transport

Tickets

for all

family

members

Frequent

flyer miles

used

Money

saved

Family

members

enjoy

vacation

Continued

good

family

relations

1 5432

Your Planned Work

Trip Planning

Your Intended Results

Trip Results

•••••

•

Chapter 1

Why Use a Logic Model?

As you can see from the travel plan example, logic models are useful tools in many ways.

Because they are pictorial in nature, they require systematic thinking and planning to

better describe programs. The visual representation of the master plan in a logic model

is flexible, points out areas of strength and/or weakness, and allows stakeholders to run

through many possible scenarios to find the best. In a logic model, you can adjust

approaches and change courses as program plans are developed. Ongoing assessment,

review, and corrections can produce better program design and a system to strategically

monitor, manage, and report program outcomes throughout development and

implementation.

Effective evaluation and program success rely on the fundamentals of clear stakeholder

assumptions and expectations about how and why a program will solve a particular prob-

lem, generate new possibilities, and make the most of valuable assets. The logic model

approach helps create shared understanding of and focus on program goals and method-

ology, relating activities to projected outcomes.

Logic Models Better Position Programs For Success

Many evaluation experts agree that use of the logic model is an effective way to ensure

program success. Using a logic model throughout your program helps organize and sys-

tematize program planning, management, and evaluation functions.

1. In Program Design and Planning, a logic model serves as a planning tool to develop

program strategy and enhance your ability to clearly explain and illustrate program con-

cepts and approach for key stakeholders, including funders.

Logic models can help craft structure and organization for program design and build in

self-evaluation based on shared understanding of what is to take place. During the plan-

ning phase, developing a logic model requires stakeholders to examine best practice

research and practitioner experience in light of the strategies and activities selected to

achieve results.

2. In Program Implementation, a logic model forms the core for a focused management

plan that helps you identify and collect the data needed to monitor and improve

programming.

Using the logic model during program implementation and management requires you to

focus energies on achieving and documenting results. Logic models help you to consider

and prioritize the program aspects most critical for tracking and reporting and make

adjustments as necessary.

3. For Program Evaluation and Strategic Reporting, a logic model presents program

information and progress toward goals in ways that inform, advocate for a particular pro-

gram approach, and teach program stakeholders.

Logic Model Development GuidePage 5

If program planners

don’t have any hypotheses

guiding them, their

potential for learning from

the initiative is low, and

the program is probably

in trouble (p. 1).

Everything You Wanted to

Know About Logic Models

but Were Afraid to Ask,

Connie Schmitz and

Beverly Anderson Parsons

(1999)

The bane of evaluation is a

poorly designed program.

Ricardo Millett, Director,

WKKF Evaluation Unit

We all know the importance of reporting results to funders and to community stakehold-

ers alike. Communication is a key component of a program’s success and sustainability.

Logic models can help strategic marketing efforts in three primary ways:

• Describing programs in language clear and specific enough to be understood and evaluated.

• Focusing attention and resources on priority program operations and key results for the

purposes of learning and program improvement.

• Developing targeted communication and marketing strategies.

The Table below describes the relationship between a successful program and the benefits

derived from the use of logic models.

How Logic Models Better Position Programs Toward Success.

Logic Models Strengthen the Case for Program Investment

Clear ideas about what you plan to do and why – as well as an organized approach to

capturing, documenting, and disseminating program results – enhance the case for

investment in your program.

Logic Model Development GuidePage 6

Program Elements

Planning and Design

Program

Implementation and

Management

Evaluation,

Communication, and

Marketing

Criteria for Program Success

Program goals and objectives,

and important side effects are

well defined ahead of time.

Program goals and objectives are

both plausible and possible.

Relevant, credible, and useful per-

formance data can be obtained.

The intended users of the evalua-

tion results have agreed on how

they will use the information.

Benefits of Program Logic Models

Finds “gaps” in the theory or logic of a

program and work to resolve them.

Builds a shared understanding of what

the program is all about and how the

parts work together.

Focuses attention of management on the

most important connections between

action and results.

Provides a way to involve and engage

stakeholders in the design, processes,

and use of evaluation.

Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). (1994). Handbook of Practical Program Evaluation.

San Francisco: Jossey-Bass Publishers.

Barley, Z., Phillips, C., & Jenness, M. (1998). Decoding Program Logic Models. Workshop presented at the

Annual Meeting of the American Evaluation Association, Chicago, IL, November, 1998.

There are many ways to

conduct evaluations, and

professional evaluators tend

to agree that there is no

“one best way” to do any

evaluation. Instead, good

evaluation requires careful-

ly thinking through the

questions that need to be

answered, the type of pro-

gram being evaluated, and

the ways in which the

information generated will

be used. Good evaluation,

in our view, should provide

useful information about

program functioning that

can contribute to program

improvement.

W.K. Kellogg Foundation

Evaluation Unit

Chapter 1

Developing a Program Logic Model Requires a Simple Image and a

Straightforward Approach

A picture IS worth a thousand words. The point of developing a logic model is to come

up with a relatively simple image that reflects how and why your program will work.

Doing this as a group brings the power of consensus and group examination of values and

beliefs about change processes and program results.

Logic Models Reflect Group Process and Shared Understanding

Frequently, a professional evaluator is charged with developing a logic model for program

practitioners. But a logic model developed by all stakeholders – program staff, partici-

pants, and evaluators – produces a more useful tool and refines program concepts and

plans in the process. We recommend that a logic model be developed collaboratively in an

inclusive, collegial process that engages as many key stakeholders as possible. This guide

provides a step-by-step process to assist program planners.

Like Programs, Logic Models Can Change Over Time

As a program grows and develops, so does its logic model. A program logic model is

merely a snapshot of a program at one point in time; it is not the program with its actual

flow of events and outcomes. A logic model is a work in progress, a working draft that

can be refined as the program develops.

Simple Logic Model Basics

Creating a logic model:

What they look like and what needs to be included

Logic models come in as many sizes and shapes as the programs they represent. A simple

model focuses on project-level results and explains five basic program components. The

elements outlined below are typical of the model promoted by United Way of America to

support an outcomes-based approach to program planning and evaluation.

Developing and Reading a Basic Logic Model

Read from left to right, logic models describe program basics over time, beginning with

best practice information or knowledge about “what works” from successful program

practitioners and other trusted authorities. Reading a logic model means following the

chain of reasoning or “If...then...” statements which connect the program’s parts. The gray

box in the left column defines the assumptions stated in “If...then...” terms.

Logic Model Development GuidePage 7

LOGIC MODEL

IF…THEN

Assumptions:

• Certain resources are

needed to operate your

program.

• If you have access to

them, then you can use

them to accomplish

your planned activities.

• If you accomplish your

planned activities, then,

you will, it is hoped,

deliver the amount of

product and/or service

that you intended.

• If you accomplish your

planned activities to the

extent intended, then

your participants will

benefit in specific ways.

• If these benefits to par-

ticipants are achieved,

then certain changes in

organizations, commu-

nities, or systems might

occur under specified

conditions.

Building a Logic Model by Basic Program Components

As you conceptualize your program, begin by describing your basic assumptions and then

add the following program components in the order that they should occur.

1. Factors are resources and/or barriers, which potentially enable or limit program effec-

tiveness. Enabling protective factors or resources may include funding, existing organizations,

potential collaborating partners, existing organizational or interpersonal networks, staff

and volunteers, time, facilities, equipment, and supplies. Limiting risk factors or barriers

might include such things as attitudes, lack of resources, policies, laws, regulations, and

geography.

2. Activities are the processes, techniques, tools, events, technology, and actions of the

planned program. These may include products – promotional materials and educational

curricula; services – education and training, counseling, or health screening; and

infrastructure – structure, relationships, and capacity used to bring about the desired

results.

3. Outputs are the direct results of program activities. They are usually described in terms

of the size and/or scope of the services and products delivered or produced by the program.

They indicate if a program was delivered to the intended audiences at the intended

“dose.” A program output, for example, might be the number of classes taught, meetings

held, or materials produced and distributed; program participation rates and demography;

or hours of each type of service provided.

4. Outcomes are specific changes in attitudes, behaviors, knowledge, skills, status, or level of

functioning expected to result from program activities and which are most often

expressed at an individual level.

5. Impacts are organizational, community, and/or system level changes expected to result

from program activities, which might include improved conditions, increased capacity,

and/or changes in the policy arena.

Thinking about a program in logic model terms prompts the clarity and specificity

required for success, and often demanded by funders and your community. Using a sim-

ple logic model produces (1) an inventory of what you have and what you need to oper-

ate your program; (2) a strong case for how and why your program will produce your

desired results; and (3) a method for program management and assessment.

Other Logic Model Examples

In practice, most logic models are more complex and fall into one of three categories: the

theory approach model (conceptual), outcome approach model, or activities approach

model (applied) – or a blend of several types. It is not unusual for a program to use all

three types of logic models for different purposes. No one model fits all needs, so you will

Logic Model Development GuidePage 8

Chapter 1

need to decide exactly what you want to achieve with your logic model – and where you

are in the life of your program – before deciding on which model to use.

Types of Logic Models: Emphasis and Strengths

Descriptions of Three Approaches to Logic Models:

Which Fits Your Program?

1. Theory Approach Models emphasize the theory of change that has influenced the

design and plan for the program. These logic models provide rich explanation of the

reasons for beginning to explore an idea for a given program. Sometimes they have

additional parts that specify the problem or issue addressed by the program, describe

the reasons for selecting certain types of solution strategies, connect proven strategies

to potential activities, and other assumptions the planners hold that influence effective-

ness. These models illustrate how and why you think your program will work. They

are built from the “big picture” kinds of thoughts and ideas that went into conceptual-

izing your program. They are coming to be most often used to make the case in

grant proposals. Models describing the beginnings of a program in detail are most

useful during program planning and design.

Logic Model Development GuidePage 9

Types of Logic Models:

Emphasis and Strengths

A program is a theory and

an evaluation is its test. In

order to organize the eval-

uation to provide a respon-

sible test, the evaluator

needs to understand the

theoretical premises on

which the program is based

(p. 55).

Carol Weiss (1998)

Intended Results

Beginnings

Should

contribute

to the results you

expect based on

this theory of

change

your assumptions

about the factors

that influence your

issues hold true...

Planned Work

Then

, the activities you

plan to do which build

on these assumptions...

Evaluation,

Communication,

Marketing

Implementation

Planning &

Design

Grant Proposal

Reports

& Other

Media

Management

Plan

theory type

outcomes type

activities type

Program

Logic

Model

what we have

done so far

how we will do

what we say we will do

what we

hope

to do

2. Outcomes Approach Models focus on the early aspects of program planning and

attempt to connect the resources and/or activities with the desired results in a workable

program. These models often subdivide outcomes and impact over time to describe

short-term (1 to 3 years), long-term (4 to 6 years), and impact (7 to 10 years) that may

result from a given set of activities. Although these models are developed with a theory

of change in mind, this aspect is not usually emphasized explicitly. Models that outline

the approach and expectations behind a program’s intended results are most useful in

designing effective evaluation and reporting strategies.

3. Activities Approach Models pay the most attention to the specifics of the implementation

process. A logic model of this type links the various planned activities together in a

manner that maps the process of program implementation. These models describe

what a program intends to do and as such are most useful for the purposes of program

monitoring and management. This type provides the detailed steps you think you will

need to follow to implement your program. It shows what you will actually do in your

community if your proposal is funded. Models that emphasize a program’s planned

work are most often used to inform management planning activities.

Working Through Theory Approach

Logic Models Emphasizes Assumptions

A theory approach logic model links theoretical ideas together to explain underlying pro-

gram assumptions. The focus here is on the problem or issue and the reasons for propos-

ing the solution suggested in your program’s approach. Remember, the theory logic model

is broad and about “big ideas,” not about specific program “nuts and bolts.”

Noted evaluator and program theorist Carol Weiss (1998) explains that for program plan-

ning, monitoring, and evaluation, it is important to know not only what the program

expects to achieve but also how. We must understand the principles on which a program

is based, a notion not included in evaluation until recently. Discussions about the

whethers, hows, and whys of program success require credible evidence and attention to the

paths by which outcomes and impacts are produced.

The theory logic model is suitable for use by funders and grantees. A case example of its

use is provided below.

In this case, the model describes a WKKF cluster initiative’s (Comprehensive Community

Health Models of Michigan) programming strategy or its theory of change. Notice that

this model places emphasis on “Your Beginnings” by including the assumptions identified

by program planners as the principles behind the design of the initiative.

Logic Model Development GuidePage 10

The purpose of using pro-

gram logic models in

WKKF grantmaking is to

help internal and external

stakeholders understand

how the Foundation’s

investment will contribute

to achieving the intended

goals. This understanding

should help these various

stakeholders make

informed decisions about

program priorities, funding

priorities, assistance to

grantees, evaluation of pro-

gramming impact, and

marketing, communica-

tion, and marketing strate-

gies.

W.K. Kellogg

Foundation

Evaluation Handbook

(1998)

Chapter 1

Working with Outcome Approach Models Highlights Activities and

Program Implementation

Outcome approach logic models display the interrelationships between specific program

activities and their outcomes. On the next page is an example drawn from the Calhoun

County Health Improvement Program, funded under the Comprehensive Community

Health Models of Michigan initiative.

This linear, columnar model emphasizes the causal linkages thought to exist among pro-

gram components. The arrows show which sets of activities program developers believed

would contribute to what outcomes. These statements serve as logical assertions about the

perceived relationship among program operations and desired results and are the hallmark

of the logic model process.

Notice that this model emphasizes “Your Intended Results” in the greatest relative detail

and anticipates achievement outside the time allotted for the initiative.

Logic Model Development GuidePage 11

These models help build a

common understanding

between managers and

evaluators.... Such agree-

ment is a prerequisite for

evaluation work that is

likely to be useful to man-

agement. [These models]

display the key events

(inputs, activities,

outcomes) that could be

monitored and the assumed

causal linkages that could

be tested in evaluations

of the program.

Joseph S. Wholey,

Harry P. Hatry, and

K.E. Newcomer (1994)

Assumptions

Health is a community issue and

communities will form partnerships

to resolve health care problems.

Commnities can influence and

shape public and market policy at

the local, state, and national levels.

External agents, working in

partnership with communities, can

serve as catalysts for change.

Shifting revenues and incentives to

primary care and prevention will

improve health status.

Information on health status and

systems is required for informed

decision making.

Your Beginnings Your Planned Work Your Intended Results

Outputs Outcomes ImpactInputs Activities

Active

Participation

in the Reform

Process

Inclusive

Community

Decision-

Making

Community-wide

Coverage and

Access

Comprehensive,

Integrated

Health Care

Delivery System

Community Health

Assessment

Community-based

Health Information

Systems

Improved

Health Status

Increased

Health Care

System

Efficiency

More Effective

Distribution of

Community

Health Care

Resources

Administrative

Processes for

Health Data,

Policy, and

Advocacy

External

Technical

Assistance

Consumers

Providers

Payers

Staff

Example of a Theory Logic model (Adapted from WKKF’s Comprehensive Community Health Models of

Michigan).

Using the Activities Approach Models to Track Outcomes

The activities approach logic model also connects program resources and activities to

desired results but does so in very great detail. Each outcome is usually dealt with sepa-

rately by the activities and events that must take place to keep the program on track.

The model emphasizing “Your Planned Work” can be used as a work plan or manage-

ment tool for program components and in conjunction with other models.

Notice how it points out what program activities need to be monitored and what kind of

measurements might indicate progress toward results. Below is one model describing the

connections between project tasks and outcome achievement for the community coverage

strand from the outcome approach example provided earlier.

Logic Model Development GuidePage 12

Not only will a logic model

clarify each element of your

program, it will enable you

to respond to the question:

“To what do I want to be

held accountable?”.

The Evaluation Forum

(1999)

Activities that encourage

consumers, providers, and payers

to seek support, and achieve

common goals.

Activities that increase consumer

awareness and access to health

promotion, disease prevention,

and primary care services.

Activities that increase linkages

among medical, health, and

human service systems.

Activities that lead to the

development of a community

access and coverage plan.

Activities that lead to the

development of a community

health information network.

Activities that lead to the

development of a community

health assessment and

reporting program.

Activities OutputsInputs

Consumers,

providers, and

payers to

participate in

governance

processes.

Sufficient staff with

expertise and

leadership skills to

implement the

program at the

local level.

Sufficient external

technical

assistance to

support staff in

program

implementation

Consumers, providers, and

payers serving on the CCHIP

Governing Board seek, support,

and achieve common goals.

Increased community access and

participation in health promotion,

disease prevention, and

primary care services.

Linkages are forged among

medical, health, and human

service systems.

Third-party administered contract

for community-wide coverage

is in place.

Fiber-optic information network

is in place (CHIN).

Community health assessment and

reporting program is in place.

CCHIP Governing Board is

deemed inclusive and

accountable by the

community stakeholders.

Increased numbers of community

members utilize the health

promotion, disease prevention,

and primary care service provided

Improved access/coverage for the

insured, under-, and non-insured

in the community.

Improved Health Status

Community members utilize the

CHIN for information collection,

storage, analysis, and exchange.

Information provided by the

Health Report Card is used to

make community health decisions.

Outcomes

Impact

Your Planned Work Your Intended Results

Chapter 1

Example of an Outcome Approach model (example drawn from the Calhoun County Health Improvement

Program, funded under the Comprehensive Community Health Models of Michigan initiative).

Adapted from the Calhoun County Health Improvement Program, one site of WKKF’s Comprehensive

Community Health Models of Michigan initiative

There Is No Best Logic Model

Try several on for size. Choose the model that fits your program best and provides the

information you need in the format that is most helpful. Like anything else, it takes prac-

tice to use logic models as effective program tools. We learn through trial and error to

find what works best for what program. Don’t hesitate to experiment with program logic

model design to determine what works best for your program. And don’t be concerned if

your model doesn’t look like one of the case examples.

The following show how the logic model forms gather information that can be used

throughout your program’s life – from defining the theory on which your program rests

to evaluating program impact.

Logic Model Development GuidePage 13

Insurance market issues are

identified and documented.

Insurance market issues are

prioritized based on potential

for successful reform.

The Purchasing Alliance will

identify insurance market issues

and strategies to reform those

identified issues will be

developed and implemented.

High priority issues are

identified and examined.

Strategies to reform the

high priority issues identified

have been developed.

Change agents with sufficient

capacity and resources to

successfully execute insurance

market reform are identified.

Equitable access to

community-wide coverage.

Change agents contracted to

implement insurance market

reform (minimum of 2).

% decrease of people

uninsured (201).

% decrease of new Medicaid

eligible consumers

achieving coverage before

in the hospital (203).

% in Medicaid participating

providers, using $1000

threshold level (204).

Deliverable–6

Your Planned Work Your Intended Results

Milestone Activities Outputs Outcomes

Activities to increase beneficiary

enrollment and provider

participation in Medicaid and

other third party sponsored

insurance and

reimbursment plans (2P1)

How to use a Logic Model Through the Life of Your Program:

Logic Model Development GuidePage 14

CLARIFYING PROGRAM THEORY:

1. PROBLEM OR ISSUE STATEMENT: Describe the problem(s) your program is attempting

to solve or the issue(s) your program will address.

2. COMMUNITY NEEDS/ASSETS: Specify the needs and/or assets of your community that

led your organization to design a program that addresses the problem.

3. DESIRED RESULTS (OUTPUTS, OUTCOMES AND IMPACTS): Identify desired results, or

vision of the future, by describing what you expect to achieve near- and long-term.

4. INFLUENTIAL FACTORS: List the factors you believe will influence change in your community.

5. STRATEGIES: List general successful strategies or “best practices” that have helped

communities like yours achieve the kinds of results your program promises.

6. ASSUMPTIONS: State the assumptions behind

how

and

why

the change strategies will

work in your community.

1. Program Planning

2. Program Implementation

3. Program Evaluation

DEMONSTRATING YOUR PROGRAM’S PROGRESS:

1. OUTPUTS: For each program activity, identify what outputs (service delivery/implementation

targets) you aim to produce.

2. OUTCOMES: Identify the short-term and long-term outcomes you expect to achieve for

each activity.

3. IMPACT: Describe the impact you anticipate in your community in 7 to 10 years with each

activity as a result of your program.

4. ACTIVITIES: Describe each of the activities you plan to conduct in your program.

5. RESOURCES: Describe the resources or influential factors available to support your program

activities.

PROGRAM EVALUATION QUESTIONS AND INDICATORS:

1. FOCUS AREA: From your program theory logic model, list the components of the most

important aspects of your program.

2. AUDIENCE: Identify the key audiences for each focus area. Who has an interest in your

program?

3. QUESTIONS: For each focus area and audience, list the questions they may have about

your program.

4. INFORMATION USE: For each audience and question you have identified, identify the ways

you will use the evaluation information.

5. INDICATORS: Describe what information could be collected that would indicate the status of

your program and its participants for each question.

6. TECHNICAL ASSISTANCE: Indicate the extent to which your organization has the evaluation

and data management expertise to collect and analyze the data that relates to this indicator.

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Focus Area

Question Indicators

Technical Assistance

Needed

“

RESOURCES

In order to accomplish

our set of activities we

will need the following:

ACTIVITIES

In order to address our

problem or asset we will

accomplish the following

activities:

OUTPUTS SHORT

We expect that once

accomplished these

activities will produce

the following evidence

or service delivery:

SHORT & LONG-TERM

OUTCOMES

We expect that if accom-

plished these activities

will lead to the following

changes in 1-3 then 4-6

years:

IMPACT

We expect that if accom-

plished these activities

will lead to the following

changes in 7-10 years:

Evaluation

Focus Area

Audience Question

Use

Chapter 1

For more detail, see the Program

Planning Template on p. 57.

For more detail, see the Evaluation

Planning Template on p. 59.

For more detail, see the Indicators

Development Template on p. 61.

For more detail, see the Program

Implementation Template on p. 54.

Developing a Basic Logic Model For Your Program

Drawing a picture of how your program will achieve results

hether you are a grantseeker developing a proposal for start-up funds or a

grantee with a program already in operation, developing a logic model can

strengthen your program. Logic models help identify the factors that will

affect your program and enable you to anticipate the data and resources you will need to

achieve success. As you engage in the process of creating your program logic model, your

organization will systematically address these important program planning and evaluation

issues:

• Cataloguing of the resources and actions you believe you will need to reach intended

results.

• Documentation of connections among your available resources, planned activities and

the results you expect to achieve.

• Description of the results you are aiming for in terms of specific, measurable, action-ori-

ented, realistic and timed outcomes.

The exercises in this chapter gather the raw material you need to draw a basic logic model

that illustrates how and why your program will work and what it will accomplish. You

can benefit from creating a logic model at any point in the life of any program. The logic

model development process helps people inside and outside your organization understand

and improve the purpose and process of your work.

Chapter 2 is organized into two sections – Program Implementation, and Program Results.

The best recipe for program success is to complete both exercises. (Full-size masters of each

exercise and the checklists are provided in the Forms Appendix at the back of the guide for

you to photocopy and use with stakeholder groups as you design your program.)

Exercise 1: Program Results. In a series of three steps, you describe the results you plan to

achieve with your program.

Exercise 2: Program Resources and Activities by taking you through three steps that con-

nect the program’s resources to the actual activities you plan to do.

The Mytown Example

Throughout Exercises 1 and 2 we’ll follow an example program to see how the logic

model steps can be applied. In our example, the folks in Mytown, USA, are striving to

meet the needs of growing numbers of uninsured residents who are turning to Memorial

Hospital’s Emergency Room for care. Because that care is expensive and not the best way

to offer care, the community is working to create a free clinic. Throughout the chapters,

Mytown’s program information will be dropped into logic model templates for Program

Planning, Implementation, and Evaluation.

Logic Model Development GuidePage 15

Chapter 2

Over the past few years, I

have markedly changed my

approach to logic modeling.

I have become convinced

that it makes a consider-

able difference if you do the

outcomes before planning

the activities.

I definitely advocate doing

the outcomes first! I find

that people come up with

much more effective activi-

ties when they do. Use the

motto, “plan backward,

implement forward.”

Beverly Anderson Parsons,

WKKF Cluster Evaluator

Novice logic modelers may want to have copies of the Basic Logic Model Template in

front of them and follow along. Those readers with more experience and familiarity may

want to explore the text and then skip ahead to the completed Basic Logic Model for the

Mytown Example on page 34.

Demonstrating Progress Toward Change

The Importance of Documenting Progress

According to many funders, grant applications frequently lack solid descriptions of how

programs will demonstrate their effectiveness. Some grantees think activities are ends unto

themselves. They report the numbers of participants they reach or the numbers of train-

ing sessions held as though they were results.

Conducting an activity is not the same as achieving results from the accomplishment of that

activity. For example, being seen by a doctor is different from reducing the number of unin-

sured emergency room visits. Tracking data like meetings held or patients enrolled does

monitor your program’s implementation and performance, but those data are outputs

(activ-

ity data), not outcomes (which refer to the results you expect to achieve in future years).

“Do the outcomes first” is sage advice. Most logic models lack specific short- and long-

term outcomes that predict what will be achieved several years down the road. Specifying

program milestones as you design the program builds in ways to gather the data required

and allows you to periodically assess the program’s progress toward the goals you identify.

For that reason, Exercise 1 isn’t filled out from left to right. This exercise asks you to

“do the outcomes first.” We will focus our attention first on what we have called “your

intended results.”

As you implement your program, outcome measures enhance program success by assess-

ing your progress from the beginning and all along the way. That makes it possible to

notice problems early on. The elements (Outputs, Outcomes, and Impact) that comprise

your intended results give you an outline of what is most important to monitor and gauge

to determine the effectiveness of your program. You can correct and revise based on your

interpretation of the collected data.

Exercise 1 – Describing Results

Describe the results you desire – Outputs, Outcomes and Impact

If you were running the Mytown Free Clinic, how would you show that your desired out-

come (a reduction in uninsured emergency care) didn’t result from a mass exodus of unin-

sured residents from Mytown, USA, or a sudden increase in number of employees offered

health insurance coverage by local businesses?

Logic Model Development GuidePage 16

Chapter 2

How will you demonstrate that your program contributed to the change you intend? A

well-crafted logic model can assert it is reasonable to claim that your program made a sub-

stantive contribution to your intended change. When programs operate in real communi-

ties where influences and forces are beyond your control, evaluation is generally more

about documenting a program’s contribution than about proving something. Community-

based initiatives operate in complex environments where the scientific certainty of “proof”

is seldom attainable. This is where logic models can be especially helpful.

INSTRUCTIONS: Exercise 1 will use the Basic Logic Model Development Template. In

particular, you will use the information presented in the gray text boxes that follow about

the Mytown example program to determine what results are intended for this program.

Example information about outcomes, impacts, and outputs are provided. You will fill in

the blank Basic Logic Model Development Template to illustrate first the outcomes and

impacts sought and then the outputs. You can then look at the completed template on

page 25 to see compare your interpretation with that produced by the Mytown folks.

Exercise 1 uses the Basic Logic Model Development Template

Outcomes and Impacts should be SMART:

• Specific

• Measurable

• Action-oriented

• Realistic

• Timed

Logic Model Development GuidePage 17

Resources

In order to accom-

plish our set of

activities we will

need the following:

Activities

In order to address

our problem or

asset we will con-

duct the following

activities:

Outputs

We expect that once

completed or under

way these activities

will produce the fol-

lowing evidence of

service delivery:

Short- & Long-

Term Outcomes

We expect that if

completed or ongo-

ing these activities

will lead to the fol-

lowing changes in

1–3 then 4–6 years:

Impact

We expect that if

completed these

activities will lead

to the following

changes in

7–10 years:

Some logic models number the lists within a column to aid discussion. Some tabular logic

models use rows to order and show the relationships among components. Some logic

models, like the outcome and activity examples provided in Chapter One, use a box and

arrow format to illustrate the “causal linkages” demonstrating how your resources, activi-

ties, outputs, outcomes, and impact connect to form chains. These depictions add to the

clarity of your logic model/evaluation plan. However, for the most basic of logic models,

the inventory approach we illustrate is sufficient to capture your thinking about how a

program will work. The other techniques will improve its utility, but the most important

task is to first get the component parts categorized and described. Once you have com-

pleted the inventory table for this and Exercise 2 feel free to experiment with identifying

the relationships among the items across columns.

Short-term outcomes are results you expect to achieve one to three years after a program

activity is under way.

Short-term outcomes are specific changes in things like attitudes, behaviors, knowledge,

skills, status, or level of functioning expected to result from program activities. These usu-

ally are expressed at an individual level among program participants.

Long-term outcomes are results you expect to achieve in four to six years.

Long-term outcomes are also specific changes in things like attitudes, behaviors, knowl-

edge, skills, status, or level of functioning expected to result from program activities.

These usually build on the progress expected by the short-term outcomes.

Logic Model Development GuidePage 18

EXAMPLES: Signed Memorandum of Agreement from the local technical college donating

clinic space, change in participants’ attitudes about the need for a medical home, increase in

numbers of scheduled annual physicals, increased patient follow-up visits, change in staff's

awareness of patient scheduling challenges, increased appropriate referrals from ER’s.

Insert Mytown’s short-term outcomes in the Short- and Long-Term Outcomes Column of the

Basic Logic Model Development Template

EXAMPLES: The clinic serves as a medical home for 500 uninsured patients. The clinic has

sustained funding sources: patient co-payments ($10/visit) provide 20% of the Clinic’s oper-

ating costs, United Way provides 20%, Memorial Hospital donates 20%, the Medical Society

contributes 20% and an endowment established at the Community Foundation provides the

final 20%. An annual golf tournament organized by the Kiwanis Club funds special clinic proj-

ects. There has been a 25% reduction in uninsured emergency care since Mytown Free Clinic

opened five years ago. In the Clinic’s fifth year there is a 15% reduction in uninsured ER vis-

its. Seventy-five volunteer administrators and 300 volunteer medical professionals regularly

serve at the clinic each year. Five companies donate all necessary medical supplies. Grant

funds purchase the computers and software needed to create electronic patient records. For

five years patient satisfaction ratings have been 90%.

Insert Mytown’s long-term outcomes in the Short- and Long-Term Outcomes column of the

Basic Logic Model Development Template

Chapter 2

Impact refers to the results expected seven to ten years after an activity is under way – the

future social change your program is working to create.

Impacts are the kinds of organizational, community, or system level changes expected to

result from program activities and which might include improved conditions, increased

capacity, and/or changes in the policy arena.

Outputs are data about activities.

They are the direct results of program activities. They are usually described in terms of

size and scope of the services or products delivered or produced by the program. They

indicate whether or not a program was delivered to the intended audiences at the intended

“dose.” A program output, for example, might include the number of classes taught, meet-

ings held, materials distributed, program participation rates, or total service delivery

hours.

Logic Model Development GuidePage 19

EXAMPLES: Specific reduction in inappropriate emergency room use, increased donations of

clinic supplies to meet identified needs, a stable supply of medical volunteers, an endowment

supporting 35% of the clinic’s operating funds, 900 patients served/year.

Insert Mytown’s impacts in the Impact Column of the Basic Logic Model Development

Template.

EXAMPLES: Number of patients referred to the Free Clinic from Memorial ER/year, the num-

ber of patients screened/year, the number of qualified patients enrolled in the Free Clinic/year,

the average number of patient visits/day, the total number of patient visits/year, the number

and specialties of medical volunteers, the number of volunteer administrators trained, the

number and locations of clinic posters distributed, the number of potential patients calling for

information/ month.

Insert Mytown’s outputs in the Outputs Column of the Basic Logic Model Development

Template.

Exercise 1 Checklist:

Review what you have created using the checklist below to assess the quality of your draft.

Logic Model Development GuidePage 20

Progress Toward Results Quality Criteria

1. A variety of audiences are taken into consideration when

specifying credible outputs, outcomes, and impacts.

2. Target participants and/or partners are described and quanti-

fied as outputs (e.g. 100 teachers from 5 rural high schools).

3. Events, products, or services listed are described as outputs

in terms of a treatment or dose (e.g. 30 farmers will partici-

pate in at least 3 sessions of program, or curriculum will be

distributed to at least 12 agencies).

4. The intensity of the intervention or treatment is appropriate

for the type of participant targeted (e.g. higher-risk partici-

pants warrant higher intensities).

5. The duration of the intervention or treatment is appropriate

for the type of participant targeted (e.g. higher-risk partici-

pants warrant longer duration).

6. Outcomes reflect reasonable, progressive steps that partici-

pants can make toward longer-term results.

7. Outcomes address awareness, attitudes, perceptions, knowl-

edge, skills, and/ or behavior of participants.

8. Outcomes are within the scope of the program’s control or

sphere of reasonable influence.

9. It seems fair or reasonable to hold the program accountable

for the outcomes specified.

10. The outcomes are specific, measurable, action-oriented, real-

istic, and timed.

11. The outcomes are written as change statements (e.g. things

increase, decrease, or stay the same).

12. The outcomes are achievable within the funding and reporting

periods specified.

13. The impact, as specified, is not beyond the scope of the pro-

gram to achieve.

Comments

Revisions

Not

Yet

Yes

■■ ■■

Chapter 2

Exercise 2 – Describing Actions

Linking It All Together

Exercise 2 illustrates exactly how you plan to put your program theory to work. It leads

you to identify the resources and activities your program will need to achieve your intend-

ed results. This exercise documents your knowledge of the community resources you have

available and specific activities your program will implement.

Program rationales in grant proposals are usually strong. Grantees tend to have a very

good sense of what they want to do. However, they frequently fail to make specific

connections between their program and related best practice literature and practitioner

wisdom that could and should support their approach and their work.

To connect actions to program results, this exercise links your knowledge of what works

with specific descriptions of what your program will do. It requires you to anticipate what

will be needed to support program activities. The elements that comprise your program

implementation act as a game plan for the program you propose.

Most logic models list activity items and resources (like planning meetings, curriculum

purchase or design, training workshops, and service delivery). Depending on the nature

of your effort, other types of products and processes may be included. Management-

oriented logic models also include program and evaluation development, staff and volun-

teer training, recruitment of partners and participants, and the publicity needed to support

your work along the way.

As mentioned earlier, if your program addresses multiple issues you may find it helpful to

go through the exercises for each issue in turn and then aggregate them into a larger

model that highlights the relationships among issues.

We recommend referring to a literature review on the problem your program is designed

to address when you specify program activities. From this explicit knowledge of what

works, you can more clearly connect the abstract strategies supporting the program to its

concrete activities.

When Exercise 2 is complete and you are satisfied that you have an accurate inventory of

the Mytown program’s component parts, transfer the information to the Basic Logic

Model Development Template. Remember you have already filled in the three columns

on the right with what you have learned about the intended results for the Mytown pro-

gram example.

What activities are planned? Based on what you know about effective ways to solve

problems or build assets, what specific activities have you planned?

Logic Model Development GuidePage 21

I would emphasize that

people may well change

their minds about the

activities that are the most

useful after having done

the results work.

Beverly Anderson Parsons,

WKKF Cluster Evaluator

What resources are needed? Once you have specified what you plan to do, determine the

resources you will need to support the solutions your program proposes. For some types

of programs, it may also be helpful to describe the influential factors you are counting on

to support your efforts in the community.

Logic Model Development GuidePage 22

EXAMPLES: Personnel Committee launches and completes search for full-time director.

Director is hired and oriented to the board and the community. Board and staff visit the

Anywhere Free Clinic to learn from its experience and to select documents to replicate (i.e.,

policies and procedures, job descriptions, equipment needs, budgets, funding strategies, vol-

unteer and patient records). Board and staff conduct program-planning retreat. Based upon

Anywhere’s funding plan, board secures Free Clinic’s first-year funding. Marketing Committee

creates public relations campaign in collaboration with Volunteer Committee to secure volun-

teers and patients. Facility Committee creates and completes MOA with technical college to

secure a clinic facility. Quality Assurance Committee creates evaluation plan in cooperation

with Memorial Hospital’s Emergency Room staff and the local Chamber of Commerce.

Summarize Mytown’s activities in the Activities column of the Basic Logic Model

Development Template

EXAMPLES: Medical Society/Memorial Hospital Task Force for the Uninsured will become a

Free Clinic Board of Directors and secure a 501(c)(3) status from the IRS. The Board will

recruit 7–10 additional representatives from drug companies, the local technical school,

Mytown’s United Way, the Chamber of Commerce, the Community Foundation, the Volunteer

Center, the Nurses Association, etc. During a 6-month planning period, board committees will

be launched; staff will be recruited/hired/oriented; a site visit will be conducted; and the

Clinic’s first-year’s funding ($150,000/year) will be secured. Committees will create an MOA

with Memorial Hospital and the Medical Society to secure equipment required: 5 exam tables,

7 desks, 5 blood pressure cuffs, 5 otoscopes, 5 stethoscopes, 5 PDR’s, 1 set of scales, 10

thermometers, three computers, one first aid emergency kit.

Summarize Mytown’s resources in the Resources column of the Basic Logic Model

Development Template

Chapter 2

Exercise 2 Checklist:

Review what you have created using the checklist below to assess the quality of your draft.

Logic Model Development GuidePage 23

Theory into Action Quality Criteria

1. Major activities needed to implement

the program are listed.

2. Activities are clearly connected to the

specified program theory.

3. Major resources needed to implement

the program are listed.

4. Resources match the type of program.

5. All activities have sufficient and

appropriate resources.

Comments/Revisions

Not

Yet

Yes

■■ ■■

Here we include a flowchart that summarizes the steps to complete your basic logic

model. Keep in mind that you could use this inventory style template to then further

describe the relationships among the components using numbered items, rows, or boxes

and arrows as we mentioned earlier.

Flowchart for Exercises 1 & 2 –

Describing Results, Resources, and Activities

Logic Model Development GuidePage 24

Exercise 1 Describing Results

Exercise 2 Describing Resources and Activities

Step 1.1

For each of the specific activities you have planned to do, what

short-term and then long-term outcomes do you expect to

achieve as indicators of the progress made by your program

toward its desired results?

Step 1.2

For each of the specific activities that you have planned to do,

what outputs (service delivery or implementation targets) do you

hope to reach through the operation of your program?

Step 1.3

For each of the specific activities you have planned to do, what

impact do you expect to achieve in your community?

Step 2.1

Knowing what you know about what works to solve problems or

build assets as specified in the theory of change for your pro-

gram, what specific activities have you planned to do?

Step 2.2

What resources are available to your program to support the

specific activities you have planned to do (for some programs, it

may also be important to state those influential factors you are

counting on to support your work)?

RESOURCES ACTIVITIES OUTPUTS OUTCOMES IMPACT

Chapter 2

OUTCOMES

OUTPUTS

IMPACT

ACTIVITIES

RESOURCES

Logic Model Development GuidePage 25

RESOURCES

In order to accomplish

our set of activities we

will need the following:

• IRS 501(c)(3) status

• Diverse, dedicated

board of directors

representing potential

partners

• Endorsement from

Memorial Hospital,

Mytown Medical

Society, and United

Way

• Donated clinic facility

• Job descriptions for

board and staff

• First year’s funding

($150,000)

• Clinic equipment

• Board & staff orienta-

tion process

• Clinic budget

ACTIVITIES

In order to address our

problem or asset we will

accomplish the following

activities:

• Launch/complete

search for executive

director

• Board & staff conduct

Anywhere Free Clinic

site visit

• Board & staff conduct

planning retreat

• Design and implement

funding strategy

• Design and implement

volunteer recruitment

and training

• Secure facility for clinic

• Create an evaluation

plan

• Design and implement

PR campaign

OUTPUTS

We expect that once

accomplished these

activities will produce

the following evidence

or service delivery:

• # of patients referred

from ER to the

clinic/year

• # of qualified patients

enrolled in the

clinic/year

• # of patient visits/year

• # of medical volunteers

serving/year

• # of patient flyers dis-

tributed

• # of calls/month seek-

ing info about clinic

SHORT- AND LONG-

TERM OUTCOMES

We expect that if accom-

plished these activities

will lead to the following

changes in 1–3 then 4–6

years:

• Memorandum of

Agreement for free

clinic space

• Change in patient atti-

tude about need for

medical home

• Change in # of

scheduled annual phys-

icals/follow-ups

• Increased # of

ER/physician referrals

• Decreased volume of

un-reimbursed emer-

gencies treated in

Memorial ER

IMPACT

We expect that if accom-

plished these activities

will lead to the following

changes in 7–10 years:

• Patient co-payments

supply 20% of clinic

operating costs

• 25% reduction in # of

uninsured ER

visits/year

• 300 medical volunteers

serving regularly each

year

• Clinic is a United Way

Agency

• Clinic endowment

established

• 90% patient satisfaction

for 5 years.

• 900 patients

served/year

Logic Model Development

Program Implementation Template – Exercise 1 & 2

Logic Model Development GuidePage 26

Developing a Theory-of-Change Logic Model

For Your Program

Drawing a picture of why your program should succeed

hether you are a grantseeker developing a proposal for start-up funds or a

grantee with a program already in operation, developing a logic model can

strengthen your program. Logic models help identify the factors that will

affect your program and enable you to anticipate the data and resources you will need to

achieve success. As you engage in the process of creating your program logic model, your

organization will systematically address these important program planning and evaluation

issues:

• Description of the change strategy that your program supports.

• Definition of the problem you are attempting to address.

• Quantification of the scope of the needs or assets that make the case for your selection

of the problem you address.

• Acknowledgement of the factors that may influence your ability to create change in

your community.

• Application of best practice research that supports plausible solution strategies for iden-

tified problem area.

• Statement of your assumptions about why your selected strategies will work in your

community in the ways you described.

Exercise 3: Program Planning constructs a program theory. Successful programs create

change and are built on a solid knowledge of what works – your program’s theory.

Exercise 3 guides you through a series of six steps that diagram the fundamental theory

that supports your program. This supports and builds upon the basic logic model. In

most cases, if you are developing a new program, this step should come first to inform

your preliminary thinking. We have placed it after basic logic models because it is a

slightly more complex exercise.

Logic Model Development GuidePage 27

Chapter 3

Exercise 3 – Constructing a Program Theory

Program Planning

The Importance of Framing Your Problems or Issues

with Sound Program Theory

Imagine you work for a funding organization. Each quarter you review a mountain of grant

proposals from worthy organizations doing good work. All of them need money. What

information would you require to determine which programs to fund? Funders tell us they

look for organizations that have done their homework by clearly defining the problem they

plan to address, describing the reasons behind their approach, and outlining how they antic-

ipate measuring their achievements. Funding decisions are more favorable if you can

demonstrate clearly how and why they will succeed. Logic models help you do just that.

Most grantees know what they want to do in their communities; Exercise 1 makes a

sound case for how and why funders should invest in your program.

It is crucial to begin program design with the basics. Funders encourage grantees to start by

clearly and succinctly explaining the problems they plan to address. Completing Exercise 1

describes the issues your program will address, identifies the needs and assets of your com-

munity that are related to your issues, and specifies why certain results are desired. Funders

and donors generally limit their investments to certain areas of interest, so if your program

addresses several issues, you may want to construct a logic model for each one.

Exercise 3 Uses The Theory-of-Change Template

Logic Model Development GuidePage 28

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Logic Model Development

Program Planning Template – Exercise 1

Chapter 3

INSTRUCTIONS: Exercise 3 will use the Theory-of-Change Template. In particular,

you will use the information presented in the gray text boxes that follow about the

Mytown example program to determine what theory-of-change was used to design and

develop this program. Example information about influential factors, the problem, com-

munity needs/assets, strategies, and assumptions are provided. You will fill in the blank

Theory-of-Change Template provided in the Forms Index (p. 57) to illustrate the pro-

gram theory for the Mytown example. You can then look at the completed template on

page 34 to compare your interpretation with that produced by the Mytown folks.

What problems are you attempting to solve or what issues are you striving to address? A

well-constructed program theory points toward your program’s eventual effectiveness.

Begin your problem statement explaining concisely the issue you will address, stating the

issue either as a community problem or asset. Your theory-of-change logic model will be

built upon this statement, which illustrates how the program will function and what it

expects to achieve in your community. It is smart to refer to research about your pro-

gram’s problem or issue in your statement; Internet searches can provide other successful

program or “best practice” information.

What needs or assets led you to address this issue? If a community needs assessment has

been conducted or if you have prioritized community needs and capacity, data exist that

make your case stronger and more specific by identifying and targeting your program’s

participants and activities. Documentation of community needs and assets also helps your

evaluation plan later on. It can become a baseline providing indicators that measure

progress made by your program over time. (Discussed in more detail in Chapter 4.)

Logic Model Development GuidePage 29

DOCUMENTED NEEDS/ASSETS EXAMPLE: Memorial Hospital’s Annual Report states that

28% of uninsured male patients, aged 40–55, received emergency room care in the previous

year. Last year’s United Way Community Needs Assessment identified health care for the

uninsured as the #1 community health care issue. The Medical Society and Memorial

Hospital’s Task Force on the Uninsured is researching ways to address the needs of the unin-

sured AND reduce costly, inappropriate ER use.

Insert Mytown’s community needs/assets in the Community Needs/Assets box of the

Theory-of-Change Template

PROBLEM STATEMENT EXAMPLE: There are increasing numbers of uninsured male workers,

aged 40–55, in Mytown, USA, due to local plant closings. As the bottom line of hospitals

shrink, the costs of uninsured care in local emergency rooms are negatively affecting local

health systems. To meet the human and financial needs of Mytown, USA, an accessible, free

medical home must be created to offer medical care and health education for Mytown’s unin-

sured residents.

Insert Mytown’s Problem or Issue in the Problem or Issue box of the Theory-of-Change

Template

What are your desired results? Identify what you expect your program to achieve in the

near and longer term. These become your outputs, outcomes and impact.

What influential factors (protective and/or risk) could influence change in your community?

What are the potential barriers and/or supports that might impact the change you hope

for? Are there policies or other factors that could affect your program?

Logic Model Development GuidePage 30

DESIRED RESULTS EXAMPLE: Increase accessible, affordable health care for the uninsured

and reduce the incidence of un-reimbursed care provided in emergency rooms. Create a free

clinic that combines an appropriate, accessible, free medical home and patient education to

reduce the numbers of uninsured males, aged 40–55, seeking care in emergency rooms.

Anticipate a 15%

crease in males, aged 40–55, with a free medical home and a 25%

crease in the incidence of uninsured men seeking care in the ER within 5 years.

Insert Mytown’s desired results (notice these are targeting men, which is more specific than

in the basic logic model example) in Desired Results box of the Theory-of-Change Template.

INFLUENTIAL FACTOR EXAMPLE: There is documented need for a free clinic. In its

Report

for the New Millennium

, the Mytown Chamber of Commerce projects a 35% increase in the

number of small businesses unable to afford employee health care benefits over the next five

years. There is strong community support for a free clinic. At the request of Mytown United

Way, Memorial Hospital and The Medical Society have created a joint task force to explore

the creation of a free clinic.

Insert Mytown’s influential factors in the Influential Factors box of the Theory-of-Change

Template.

Chapter 3

Flowchart for Exercise 3

Logic Model Development GuidePage 31

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Program Planning – Clarifying Program Theory

1 Problem or Issue Statement

Describe the problem(s) your program is attempting to

solve or the issue(s) your program will address.

2 Community Needs/Assets

Specify the needs and/or assets of your community that

led your program to address the problem(s) or issue(s).

3 Desired Results (Outputs, Outcomes, and

Impacts)

Identify your desired results, or vision of the future, by

describing what you expect to achieve, near- or long-

term, if your program is funded.

4 Influential Factors

List the factors (e.g., protective or risk factors, existing

policy environment, or other factors) you believe will

influence change in your community.

5 Strategies

List general, successful strategies or “best practices”

your research identified that have helped communities

like yours achieve the kinds of results your program

promises.

6 Assumptions

State the assumptions behind

how

and

why

the identi-

fied change strategies will work in your community

(e.g., principles, beliefs, ideas).

For more detail, see the Program

Planning Template – Exercise 3

on p. 34.

Why do you believe your program will work? Look for strong rationale based on “best

practice” research that connects what you plan to do with why your approach will succeed.

Funders are eager for evidence that supports why you propose the solutions you do. It’s a

good idea to relate your approach to similar change strategies that have proven effective in

communities like yours. Reviewing literature and past evaluation reports from other pro-

grams (or your own work) will provide you with ample information to construct your pro-

gram rationale. The Internet makes it easier to research effective program strategies.

Why will your approach be effective? After you make the case for selecting a specific strategy

from among the alternatives you researched, state out loud why this strategy is needed and

why it will work in your community. It is important early on to document instances that

describe the general condition of public reaction to your problem/issue and possible solutions.

You should draw direct conclusions about the statement of need and capacities in your com-

munity in your assumption. In addition, it should be quite apparent how your program

intends to function as an intervention – to solve identified problems or build existing assets.

We list assumptions last in this exercise because in this abstracted learning format, the

logic modeler has the benefit of all the information that supports assumptions. They are

easier to spot and articulate with all the facts in front of you. In real-world conditions,

assumption are best stated up-front – much earlier in the logic model development

process – many basic logic models we have seen include a supporting page with the dia-

gram that lists the assumptions that belie the model drawn.

Logic Model Development GuidePage 32

PROGRAM STRATEGY EXAMPLE: A clinic using volunteer medical professionals reduced

emergency room care visits in Anothertown, USA, in 1997 by 25%. A free clinic in Mytown,

USA, using volunteer medical professionals could provide crucial, affordable medical homes

for growing numbers of uninsured residents preventing costly, inappropriate emergency

room use by males 40–55 experiencing coronary emergencies.

Insert Mytown’s strategies in the Strategies box of the Theory-of-Change Template.

ASSUMPTION EXAMPLE: As proven in Anothertown, access to affordable medical care

reduces the incidence of emergency visits by providing appropriate, preventive primary care.

A free medical clinic should prove successful in Mytown, because of its history of extraordi-

nary volunteerism. Mytown’s Medical Society officially encourages its 400 medical profes-

sional members to volunteer 20 hours each year to help medically underserved community

residents. Mytown’s Nursing Association is also interested in collaborating with a free clinic.

Memorial Hospital has agreed to assist in planning and funding a free clinic. There is prece-

dence for lending free facilities to medical projects serving those in need. Mytown’s technical

college donates space for Mytown’s volunteer dental clinic. Mytown’s Free Clinic will be

strongly supported by the people, businesses and institutions of Mytown, USA.

Insert Mytown’s assumptions in the Assumptions box of the Theory-of-Change Template.

Chapter 3

Exercise 3 Checklist:

After completing Exercise 3 and constructing your program theory, you can use the fol-

lowing checklist to assess the quality of your draft. It’s helpful if someone other than the

model’s creators reviews the first program draft and completes the checklist, too.

Logic Model Development GuidePage 33

Exercise Three Checklist

1. The problems to be solved/or issues to be addressed by

the planned program are clearly stated

2. There is a specific, clear connection between the

identified community needs/assets and the problems to

be solved (or issues to be addressed).

3. The breadth of community needs/assets has been

identified by expert/practitioner wisdom, a needs

assessment and/or asset mapping process.

4. The desired results/changes in the community and/or

vision for the future ultimately sought by program

developers are specific.

5. Influential factors have been identified and cited from

expert/practitioner wisdom and/or a literature review.

6. Change strategies are identified and cited from

expert/practitioner wisdom and/or literature review.

7. The connection among known influential factors and

broad change strategies has been identified.

8. The assumptions held for how and why identified

change strategies should work in the community are

clear.

9. There is consensus among stakeholders that the model

accurately describes the proposed program and its

intended results.

Comments

Revisions

Not

Yet

Yes

■■ ■■

Logic Model Development GuidePage 34

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Logic Model Development

Program Planning Template – Exercise 3

• Create a free clinic staffed primarily by volunteer physicians,

nurses, and pharmacists as in Anywhere, USA

• Ask doctors to see patients for free in their own

practices/Columbia, SC

• Mytown has a history of successful volunteer programs

• The Medical Society will encourage volunteers and provide on-going

support

• The clinic can find and operate in donated space

• The hospital will support a free clinic to improve patient health and to

save money

• Chamber predicts increase

in # of small businesses

unable to offer employee

health insurance

• There is a strong

community support for

a free clinic generated by

the Uninsured Task Force

• 3 major corporate leaders

have expressed interest in

a free clinic

• Increased #’s of uninsured workers

• Local plant closings limit jobs

• Costs of uninsured ER care are rising

• Hospitals cannot fund free ER care forever

• Memorial Hospital’s Annual Report states that 28% of

uninsured, male patients are seen in ER

• United Way Assessment lists health care for uninsured as

a top priority

• Memorial & Medical Society formed Task Force on

uninsured to research solutions

• Increased access to

affordable health care for

uninsured Mytown residents

• Create a free clinic to offer

affordable health care

+ education

• Decrease # of uninsured

patients seeking care in ER

• Increase # of uninsured

patients with a medical home

Using Your Logic Model to Plan for Evaluation

hinking through program evaluation questions in terms of the logic model com-

ponents you have developed can provide the framework for your evaluation plan.

Having a framework increases your evaluation’s effectiveness by focusing on ques-

tions that have real value for your stakeholders.

• Prioritization of where investment in evaluation activities will contribute the most use-

ful information for program stakeholders.

• Description of your approach to evaluation.

There are two exercises in this chapter; Exercise 4 deals with posing evaluation questions

and Exercise 5 examines the selection of indicators of progress that link back to the basic

logic model or the theory-of-change model depending on the focus of the evaluation and

its intended primary audiences.

Exercise 4 – Posing Evaluation Questions

The Importance of “Prove” and “Improve” Questions

There are two different types of evaluation questions – formative help you to improve

your program and summative help you prove whether your program worked the way you

planned. Both kinds of evaluation questions generate information that determines the

extent to which your program has had the success you expected and provide a ground-

work for sharing with others the successes and lessons learned from your program.

Benefits of Formative and Summative Evaluation Questions

Logic Model Development GuidePage 35

Chapter 4

Formative Evaluation – Improve

Provides information that helps you improve

your program. Generates periodic reports.

Information can be shared quickly.

Focuses most on program activities, out-

puts, and short-term outcomes for the pur-

pose of monitoring progress and making

mid-course corrections when needed.

Helpful in bringing suggestions for improve-

ment to the attention of staff.

Summative Evaluation – Prove

Generates information that can be used to

demonstrate the results of your program to

funders and your community.

Focuses most on program’s intermediate-

term outcomes and impact. Although data

may be collected throughout the program,

the purpose is to determine the value and

worth of a program based on results.

Helpful in describing the quality and effec-

tiveness of your program by documenting its

impact on participants and the community.

Adapted from Bond, S.L., Boyd, S. E., & Montgomery, D.L.(1997 Taking Stock: A Practical Guide to Evaluating

Your Own Programs, Chapel Hill, NC: Horizon Research, Inc. Available online at http://www.horizon-research.com.

Looking at Evaluation from Various Vantage Points

How will you measure your success? What will those investing in your program or your

target audience want to know?

A clear logic model illustrates the purpose and content of your program and makes it eas-

ier to develop meaningful evaluation questions from a variety of program vantage points:

context, implementation and results (which includes outputs, outcomes, and impact).

What Parts of Your Program Will Be Evaluated?

Using a logic model to frame your evaluation questions.

Remember you can draw upon the basic logic model in Exercises 1 and 2 and the theory-

of-change model in Exercise 3. Feasibility studies and needs assessments serve as valuable

resources for baseline information on influences and resources collected during program

planning.

Context is how the program functions within the economic, social, and political envi-

ronment of its community and addresses questions that explore issues of program rela-

tionships and capacity. What factors might influence your ability to do the work you have

planned? These kinds of evaluation questions can help you explain some of the strengths

and weakness of your program as well as the effect of unanticipated and external influences

on it.

Logic Model Development GuidePage 36

Influences

Resources

Activities Outouts

Short-Term

Outcomes

Intermediate

Outcomes

Activities

Context

Relationships

& Capacity

Implementation

Quality &

Quantity

Outcomes

Effectiveness, Magnitude,

& Satisfaction

and/or

Summative EvaluationFormative Evaluation

What aspects of our situation

most shaped our ability to

do the work we set out to

do in our community?

What did our program

accomplish in our

community?

What is our assessment

of what resulted from our

work in the community?

What have we learned

about doing this kind

of work in a community

like ours?

Chapter 4

Outputs

Implementation assesses the extent to which activities were executed as planned, since a

program’s ability to deliver its desired results depends on whether activities result in the

quality and quantity of outputs specified. They tell the story of your program in terms of

what happened and why.

Outcomes determine the extent to which progress is being made toward the desired

changes in individuals, organizations, communities, or systems. Outcome questions seek

to document the changes that occur in your community as a result of your program.

Usually these questions generate answers about effectiveness of activities in producing

changes in magnitude or satisfaction with changes related to the issues central to your

program.

Logic Model Development GuidePage 37

Sample CONTEXT QUESTIONS: Can we secure a donated facility? With the low morale cre-

ated by high unemployment, can we secure the financial and volunteer support we need?

How many medical volunteers can we recruit? How many will be needed each evening? How

will potential patients find out about the clinic? What kind of medical care will patients need?

How can we let possible referral sources know about the clinic and its services? What sup-

plies will we need and how will we solicit suppliers for them? What is it about the free clinic

that supports its ability to reduce the numbers of patients seeking care in Memorial

Hospital’s ER?

SAMPLE IMPLEMENTATION QUESTIONS: What facility was secured? How many patients

were seen each night/month/year? What organizations most frequently referred patients to

the clinic? How did patients find out about the clinic? How many medical volunteers serve

each night/month/year? What was the value of their services? What was the most common

diagnosis? What supplies were donated? How many patients per year did the Clinic see in its

first/second/third year?

SAMPLE OUTCOME QUESTIONS: How many inappropriate, uninsured patients sought med-

ical care in Memorial’s ER in the Clinic’s first/second/third year? Was there a reduction in un-

funded ER visits? How did the number of uninsured patients compare to previous years

when the clinic was not operating? What was the cost/visit in the Free Clinic? What is the

cost/visit in Memorial’s ER? How do they compare? What were the cost savings to Memorial

Hospital? How satisfied were Clinic patients with the care they received? How satisfied were

volunteers with their service to the Clinic?

Creating Focus

Though it is rare, you may find that examining certain components of your program is

sufficient to satisfy your information needs. Most often, however, you will systematically

develop a series of evaluation questions, as shown in the Flowchart for Evaluation

Question Development.

Flowchart for Evaluation Question Development

Evaluation Focus Area

What is going to be evaluated?

List those

components from your theory and/or logic

model that you think are the most important

aspects of your program. These areas will

become the focus of your evaluation.

Audience

What key audience will have questions

about your focus areas?

For each focus

area you have identified, list the audiences

that are likely to be the most interested in

that area.

Question

What questions will your key audience have

about your program?

For each focus area

and audience that you have identified, list

the questions they might have about your

program.

Information Use

If you answer a given question, what will

that information be used for?

For each

audience and question you have identified,

list the ways and extent to which you plan to

make use of the evaluation information.

Logic Model Development GuidePage 38

The use of program theory

as a map for evaluation

doesn’t necessarily imply that

every step of every possible

theory has to be studied.

...Choices have to be made

in designing an evaluation

about which lines of

inquiry to pursue. ...The

theory provides a picture of

the whole intellectual land-

scape so that people can

make choices with a full

awareness of what they are

ignoring as well as what

they are choosing to study...

Weiss (1998)

Evaluation

Focus Area

Audience Question

Use

Evaluation

Focus Area

Audience Question

Use

Evaluation

Focus Area

Audience Question

Use

Evaluation

Focus Area

Audience Question

Use

Chapter 4

For more detail, see the Evaluation

Planning Template – Exercise 4

on p. 44.

EVALUATION

FOCUS AREA

AUDIENCE

QUESTION

INFORMATION USE

What is going to be evaluated?

For each area on which your program focuses, list the most important aspects of your

program theory and logic model. Focus your evaluation on them.

Focus Area Examples:

The benefits of asking and answering evaluation questions depend on how clear you are

about the purpose of your evaluation, who needs to know what when, and the resources

you have available to support the evaluation process.

Logic Model Development GuidePage 39

Context Examples — Evaluating relationships and capacity. How will the Free Clinic recruit

and train effective board and staff members? What is the best way to recruit, manage, retain

and recognize medical and administrative volunteers and other Clinic partners? What is the

most effective way to recruit and retain uninsured patients? How will the operation of a Free

Clinic affect Memorial Hospital’s expenses for providing uninsured medical care in its ER?

How many patients can Clinic volunteers effectively serve on a regular basis? What is the

ideal patient/volunteer ratio?

Insert focus areas into Focus Area Column of Evaluation Questions Development Template

for Evaluation Planning, Exercise 4

Implementation Examples – Assessing quality and quantity. How many major funding part-

ners does the clinic have? How are volunteers and patients scheduled? How many medical

volunteers serve Clinic patients on a regular basis? What is the value of their services? What

is the most common diagnosis at the Clinic? What is the most common diagnosis of unin-

sured patients seen in Memorial’s ER? How long do patients wait to be seen at the Clinic? Is

there a patient or volunteer waiting list?

Insert focus areas into Focus Area Column of Evaluation Questions Development

Template for Evaluation Planning, Exercise 4.

Outcomes – Measuring effectiveness, magnitude and satisfaction. Has the clinic increased

access to care for a significant number of Mytown’s uninsured citizens? How many residents

of Mytown, USA, do not have health insurance? How many patients does the Clinic serve on a

regular basis? What is that ratio? What is the cost per visit in the Clinic and Memorial’s ER?

How do the costs compare? What is the satisfaction level of Clinic patients and volunteers

with Clinic services and facilities? How many donors does the Clinic have? What is their satis-

faction with Clinic services and facilities? How effectively is the Clinic educating, engaging and

involving its partners? What organizations have officially endorsed the Clinic? What is the

board and staff’s satisfaction with clinic operations, facilities and services?

Insert focus areas into Focus Area Column of Evaluation Questions Development Template

for Evaluation Planning, Exercise 4

What Information Will Your Program’s Audiences Want?

As shown below, program audiences will be interested in a variety of different kinds of

information. Donors may want to know if their money did what you promised it would.

Patients might want to know how many patients the clinic serves and how many volun-

teers it has. Physicians donating their time and talent could be interested in the financial

value of their contributions. If you ask your audiences what they want to know, you’ll be

sure to build in ways to gather the evaluation data required.

How often do you have to gather data? Whether a question is more formative or summa-

tive in nature offers a clue on when information should be collected.

• Formative information should be periodic and reported/shared quickly to improve

your efforts.

• Summative tends to be “before and after” snapshots reported after the conclusion of the

program to document the effectiveness and lessons learned from your experience.

Logic Model Development GuidePage 40

Audience

Program

Management

and Staff

Participants

Community

Members

Public Officials

Funders

Typical Questions

Are we reaching our target population?

Are our participants satisfied with our program?

Is the program being run efficiently?

How can we improve our program?

Programming decisions, day-to-day operations

Did the program help me and people like me?

What would improve the program next time?

Is the program suited to our community

needs?

What is the program really accomplishing?

Who is the program serving?

What difference has the program made?

Is the program reaching its target population?

What do participants think about the program?

Is the program worth the cost?

Is what was promised being achieved?

Is the program working?

Is the program worth the cost?

Evaluation Use

Programming decisions,

day-to-day operations

Decisions about continu-

ing participation.

Decisions about partici-

pation and support.

Decisions about commit-

ment and support.

Knowledge about the

utility and feasibility of

the program approach.

Accountability and

improvement of future

grantmaking efforts.

Chapter 4

Involve Your Audience in Setting Priorities

Program developers often interview program funders, participants, staff, board and partners

to brainstorm a list of all possible questions for a key area identified from their program

theory or from their logic models. That list helps determine the focus of the evaluation.

Involving your audience from the beginning makes sure you gather meaningful informa-

tion in which your supporters have a real interest.

Prioritization is a critical step. No evaluation can answer all of the questions your pro-

gram’s audiences may ask. The following questions can help you narrow your number of

indicators: How many audiences are interested in this information? Could knowing the

answer to this question improve your program? Will this information assess your pro-

gram’s effectiveness?

The final focus for your evaluation is often negotiated among stakeholders. It is impor-

tant to keep your evaluation manageable. It is preferable to answer a few important

questions thoroughly than to answer several questions poorly. How well you can answer

your questions will depend on the time, money, and expertise you have at your disposal

to perform the functions required by the evaluation.

What key audiences will have questions about your evaluation focus

areas?

For each focus area that you identified in the previous step, list the audiences that are

likely to be most interested in that area. Summarize your audiences and transfer to the

Audience Column of the Evaluation Questions Development Template for Evaluation

Planning, Exercise 4.

Context – Relationships and Capacity

Implementation – Quality and Quantity

Outcomes – Effectiveness, Magnitude, and Satisfaction

Logic Model Development GuidePage 41

Example audiences: Medical professionals, Memorial Hospital Board and Staff (especially ER

staff), Medical associations, Foundations, The Chamber of Commerce, United Way, The

Technical College, uninsured residents, medical supply companies, local media, public officials.

Example audiences: Funders, In-kind donors, Volunteers, Board, Staff, Patients, Public

Officials, The media, Medical associations, Local businesses, Health care organizations.

Example audiences: Funders, In-kind donors, Medical and administrative volunteers, Board,

Staff, Patients, Public Officials, The media, Medical associations, Local businesses, Health

care organizations.

What questions will key audiences ask about your program?

For each focus area and key audience you identified in the previous step, list the questions

your stakeholders ask about your program. Insert summaries in the Question Column of

the Evaluation Questions Development Template for Evaluation Planning, Exercise 4 (on

page 44).

Sample of Key Audience Questions:

• Who are the collaborative partners for this program? What do they provide?

• What is the budget for this program?

• How many staff members does the program have?

• How many patients does the clinic serve?

• How many visits per year does the average patient have?

• What is the most common diagnosis?

• Does the clinic save the hospital money?

• How does the organization undertake and support program evaluation?

• How are medical volunteers protected from lawsuits?

• How satisfied are patients, volunteers, board and staff with the clinic’s services?

• What do experts say about the clinic?

• How many uninsured patients still seek inappropriate care in the ER? Why?

How will the evaluation’s information be used?

For each question and audience you identified in the previous step, list the ways and

extent to which you plan to make use of the evaluation information. Summarize audience

use of information. Insert in the Use Column of the Evaluation Questions Development

Template for Evaluation Planning, Exercise 4.

Context – Relationships and Capacity Examples

• Measure the level of community support.

• Assess effectiveness of community outreach.

• Assess sustainability of Clinic funding sources.

• Improve volunteer and patient recruitment methods.

• Secure additional Clinic partners.

Implementation – Quality and Quantity Examples

• Assess optimal number of volunteers and patients to schedule per session to improve

operating effectiveness while maintaining patient and volunteer satisfaction.

• Measure patient, volunteer, staff, board, donors and community satisfaction with clinic.

• Determine cost savings per visit. Share information with local medical and business

groups to encourage their support.

Logic Model Development GuidePage 42

Chapter 4

Outcomes and Impact – Examples of Effectiveness, Magnitude, and Satisfaction

• Cost savings of Clinic – use to obtain additional volunteer and financial support from

Memorial Hospital.

• Patient satisfaction survey results – use to improve patient services and satisfaction.

• Analysis of most frequent referral sources – use to present information seminars to ER

staff, social service workers and unemployment insurance clerks to increase patient refer-

rals and intakes.

• Analysis of most prevalent patient diagnoses – use to create relevant patient health edu-

cation newsletter. Patient tracking system will measure impact of education program.

Exercise 4 Checklist: After completing Exercise 4 you can use the following checklist to

assess the quality of your draft.

Logic Model Development GuidePage 43

Posing Questions Quality Criteria Yes Not Comments

Yet Revisions

1. A variety of audiences are taken

into consideration when

specifying questions.

2. Questions selected are those with

the highest priority.

3. Each question chosen gathers

useful information.

4. Each question asks only one

question (i.e. “extent of X, Y,

and Z” is not appropriate).

5. It is clear how the question

relates to the program’s logic

model.

6. The questions are specific about

what information is needed.

7. Questions capture lessons

learned about your work along

the way.

8. Questions capture lessons

learned about your program

theory along the way.

■■

Logic Model Development GuidePage 44

Evaluation

Focus Area

Relationships

Outcomes

Audience

Funders

Medical Volunteers

Patients

Staff

Funders/Donors

Volunteers

Patients

Staff

Question

Is the program cost effective?

Are volunteers & patients satisfied with Clinic

services?

What is the most common diagnosis?

How will medical volunteers be protected from

lawsuits?

Am I receiving quality care?

How long can I receive care here?

Are we reaching our target population?

How do patients find us? What’s our best promo-

tional approach?

Program Budget?

Cost/visit?

Visits/month/year?

Cost savings for Memorial Hospital?

Volunteers/year?

Patient satisfaction

Patient & volunteer satisfaction

Common DRG(?)

Use

Cost benefits/fundraising

Program promotion/fundraising

Quality assurance/Planning

Volunteer recruitment

Program improvement & planning

" "

Evaluation/program promotion

Evaluation and/or improvement

Cost benefit analysis

Annual Report/program promotion/public relations

Annual Report/program promotion/fundraising

Annual Report/volunteer recruitment

Program improvements/staff training

Logic Model Development

Evaluation Planning Template – Exercise 4

Exercise 5 – Establishing Indicators

One of the biggest challenges in developing an evaluation plan is choosing what kind of

information best answers the questions you have posed. It is important to have general

agreement across your audiences on what success will look like. Indicators are the meas-

ures you select as markers of your success.

In this last exercise you create a set of indicators. They are often used as the starting point

for designing the data collection and reporting strategies (e.g., the number of uninsured

adults nationally, statewide, in Mytown, USA, or the number of licensed physicians in

Mytown). Often organizations hire consultants or seek guidance from local experts to con-

duct their evaluations. Whether or not you want help will depend on your organization’s

level of comfort with evaluation and the evaluation expertise among your staff.

Examples and Use of Indicators.

Logic Model Development GuidePage 45

The biggest problem is usu-

ally that people are trying

to accomplish too many

results. Once they engage in

a discussion of indicators,

they start to realize how

much more clarity they

need in their activities.

I also find that it is impor-

tant that the program, not

the evaluator, is identifying

the indicators. Otherwise,

the program can easily dis-

credit the evaluation by

saying they don’t think the

indicators are important,

valid, etc.

Beverly Anderson Parsons,

WKKF Cluster Evaluator

Focus Area

Influential Factors

Resources

Activities

Outputs

Outcomes &

Impacts

Indicators

Measures of influential factors – may

require general population surveys

and/or comparison with national data

sets

Logs or reports of financial/staffing

status.

Descriptions of planned activities.

Logs or reports of actual activities.

Descriptions of participants.

Logs or reports of actual activities.

Actual products delivered.

Participant attitudes, knowledge,

skills, intentions, and/or behaviors

thought to result from your activities

How to Evaluate

Compare the nature and extent

of influences before (baseline)

and after the program.

Compare actual resources

acquired against anticipated.

Compare actual activities

provided, types of participants

reached against what was

proposed.

Compare the quality and quan-

tity of actual delivery against

expected.

Compare the measures before

and after the program

This table was adapted from A Hands-on Guide to Planning and Evaluation (1993) available from the National AIDS

Clearinghouse, Canada.

You may want to allocate resources to allow for the assistance of an external evaluation consultant to access national data-

bases or perform statistical analyses.

Many types of outcomes and impact instruments (i.e. reliable and valid surveys and questionnaires) are readily available.

The Mental Measurement Yearbook published by the Buros Institute (http://www.unl.edu/buros/) and the ERIC

Clearinghouse on Assessment and Evaluation (http://ericae.net/) are great places to start.

You may need to allocate resources to allow for the assistance of an external evaluation consultant.

Our advice is to keep your evaluation simple and straightforward. The logic model tech-

niques you have been practicing will take you a long way toward developing an evaluation

plan that is meaningful and manageable.

Determine the kinds of data you will need and design methods to gather the data (i.e.,

patient registration forms, volunteer registration forms, daily sign-in sheets, national, state

and local statistics). Sometimes, once an indicator (type of data) is selected, program

planners set a specific target to be reached as an agreed upon measure of success (for

example 25% decrease in the numbers of inappropriate ER visits).

As in the previous exercises use the space below to loosely organize your thoughts. Then,

once the exercise is completed and assessed, use the Indicator Development Template on

page 61 to record your indicators and technical assistance needs.

Filling in the Flowchart for Indicator Development

What information will be gathered to “indicate” the status of your

program and/or its participants?

Column 1: Focus Areas – From the information gathered in Exercise 4, transfer the areas

on which your evaluation will focus into column one (for example, patient health, volun-

teer participation, sustaining supporting partnerships).

Column 2: Questions – transfer from Exercise 4 the major questions related to each focus

area – big questions your key audiences want answered. Remember to keep your evalua-

tion as simple as possible.

Column 3: Indicators – Specify the indicators (types of data) against which you will

measure the success/progress of your program. It’s often helpful to record the sources of

data you plan to use as indicators (where you are likely to find or get access to these data).

Column 4: Technical Assistance – To what extent does your organization have the evalua-

tion and data management expertise needed to collect and analyze the data that related to

each indicator? List any assistance that would be helpful – universities, consultants,

national and state data experts, foundation evaluation departments, etc.

Logic Model Development GuidePage 46

Focus Area

Question Indicators

Technical Assistance

Needed

“

1 2 3 4

Chapter 4

Exercise 5 Checklist: Review what you have created using the checklist below to assess

the quality of your evaluation plan.

Logic Model Development GuidePage 47

1. The focus areas reflect the

questions asked by a variety

of audiences. Indicators

respond to the identified focus

areas and questions.

2. Indicators are SMART–

Specific, Measurable, Action-

oriented, Realistic, and Timed.

3. The cost of collecting data on

the indicators is within the

evaluation budget.

4. Source of data is known.

5. It is clear what data collection,

management, and analysis

strategies will be most

appropriate for each indicator.

6. Strategies and required

technical assistance have been

identified and are within the

evaluation budget for the

program.

7. The technical assistance

needed is available.

Establishing Indicators

Quality Criteria

Yes Not

Yet

Comments

Revisions

■■

Logic Model Development GuidePage 48

Focus Area

Relationships

Outcomes

Question

Are volunteers & patients

satisfied w/ clinic care?

Are we reaching our target

population?

How do patients find the

clinic?

Does the clinic save the

community $?

What does the clinic provide?

How has volunteering

affected doctors, nurses,

administrators and patients?

Indicators

• Patient satisfaction surveys

• Volunteer satisfaction tests

• % of clinic patients vs. % of uninsured citizens

in Mytown, USA

• # of qualified clinic patients/year

• Annual analysis of telephone referral log

• Referral question on patient intake form

• Cost/visit

• # of uninsured patients seen in hospital ER –

beginning the year before clinic opened

• Most common diagnosis

• Hospital cost/visit for common diagnosis

• Annual volunteer survey

• Patient satisfaction survey

• # of volunteers/year

• # of volunteers donating to clinic operations

Technical Assistance

Needed

Anywhere’s patient satisfaction surveys

Anywhere’s volunteer survey

Reports from Chamber of Commerce

Patient database creation

Telephone log database

Anywhere’s patient intake form

Budget figures; patient service records

Tracking database software

Strategic direction for analysis

DRG workbook/tables (hospital staff)

Input from hospital billing staff

Anywhere surveys and analysis instruments

Volunteer management database

Donor database (Raiser’s Edge?)

Logic Model Development

Indicators Development Template – Exercise 5

This Appendix provides information on print and electronic resources available to support you

in your logic model development process.

1. Logic Model Information and Examples

University of Nevada, Reno Western CAPT web site

http://www

.unr.edu/colleges/educ/captta/pr

ev/evaluate.htm

BJA Evaluation web site

http://www

.bja.ev

aluationwebsite.org

Schmitz, C. C. & Parsons, B. A. (1999). Everything you wanted to know about logic models

but were afraid to ask. Battle Creek, MI: W.K. Kellogg Foundation.

2. United Way of America’s Outcome Models

United Way of America web site

http://www

.unitedway.org/outcomes/contents.htm

Measuring program outcomes: A practical approach.

United Way of America

701 North Fairfax Street

Alexandria, VA 22314

(703) 836-7100

3. Definitions and Information on Program Theory and Evaluation

Program Theory Definitions

• A plausible and sensible model of how a program is supposed to work (Bickman, 1987, p. 5).

• The set of assumptions about the relationships between the strategy and tactics the program

has adopted and the social benefits it is expected to produce (Rossi, Freeman, & Lipsey,

1999, p.98).

• The full chain of objectives that links inputs to activities, activities to ... outputs, ... outputs to

... outcomes, and ... outcomes to ultimate goals constitutes a program’s theory

(Patton, 1997, p. 218).

• A set of interrelated assumptions, principles, and/or propositions to explain or guide social

actions (Chen, 1990, p. 40).

• An explanation of the causal links that tie program inputs to expected program outputs

(Weiss, 1998, p. 55).

• A chain of causal assumptions linking program resources, activities, intermediate outcomes,

and ultimate goals (Wholey, 1987, p. 78).

Logic Model Development GuidePage 49

Resources Appendix

Bickman, L. (Ed.). (1987). Using program theory in evaluation. New Directions for

Program Evaluation Series (no. 33). San Francisco: Jossey-Bass.

Chen, H. T. (1990). Theory driven evaluations. Newbury Park, CA: Sage.

Wholey, J. S. (Ed). (1987). Organizational excellence: Stimulating quality and communicat-

ing value. Lexington, MA: Lexington Books.

Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). (1994). Handbook of Practical

Program Evaluation. San Francisco: Jossey-Bass Publishers.

Weiss, C. H. (1998). Evaluation: Methods for studying programs and policies. (2nd Ed).

Upper Saddle River, NJ: Prentice Hall.

4. W.K. Kellogg Foundation Logic Model Examples

W.K. Kellogg Foundation Web site

http://www.wkkf.org

W.K. Kellogg Foundation Evaluation Handbook.

To order contact:

W.K. Kellogg Foundation

P.O. Box 550

Battle Creek, MI 49015

(800) 819-9997

[item number 1203]

Parsons, B. A. (1999). Making logic models more systemic. A paper presented at the Annual

Meeting of the American Evaluation Association, Orlando, FL, November 1999.

Parsons, B. A., Schmitz, Co (1999) Everything You Wanted to Know About Logic Models

But Were Afraid to Ask. A paper presented at the Annual Meeting of the American

Evaluation Association, Orlando, FL, November 1999.

5. Information about Logic Model Development and Use

The Evaluation Forum (1999). Outcomes for success.

The Evaluation Forum

1932 First Avenue, Suite 403

Seattle, WA 98101

(206) 269-0171

Logic Model Development GuidePage 50

Resources Appendix

Freddolino, P. P. (1999). The program logic model: What it is and how to teach it. A pre-

conference workshop presented at the 1999 Conference of the Michigan Association for

Evaluation, East Lansing, MI, May.

Targeting Outcomes of Programs.

http://deal.unl.edu/TOP/synopsis.htm

Innovation Network, Inc. electronic logic model development tool

http://www.inetwork.org

6. Evaluation Planning Information

W.K. Kellogg Foundation Web site

http://www.wkkf.org

W.K. Kellogg Foundation Evaluation Handbook.

To order contact:

W.K. Kellogg Foundation

P.O. Box 550

Battle Creek, MI 49015

(800) 819-9997

[item number 1203]

Taking Stock.

http://www.horizon-research.org

The Evaluation Forum (1994). A field guide to outcome-based program evaluation.

The Evaluation Forum

1932 First Avenue, Suite 403

Seattle, WA 98101

(206) 269-0171

Rossi, P. H., Freeman, H. E., & Lipsey, M. W. (1999). Evaluation: A systematic approach.

Thousand Oaks, CA: Sage.

Patton, M. Q. (1997). Utilization-focused evaluation: The new century text. Thousand

Oaks, CA: Sage.

Logic Model Development GuidePage 51

Logic Model Development GuidePage 52

This Appendix provides the worksheet templates and checklists for exercises 1-5:

Logic Model Development Program Planning and Implementation

Exercises 1 and 2 Template

Exercise 1 Checklist

Exercise 2 Checklist

Theory-of-Change Logic Model Development Planning

Exercise 3 Template

Exercise 3 Checklist

Logic Model Development Evaluation and Indicators Development

Exercise 4 Template

Exercise 4 Checklist

Exercise 5

Exercise 5 Checklist

Logic Model Development GuidePage 53

Forms Appendix

Logic Model Development GuidePage 54

RESOURCES

In order to accomplish

our set of activities we

will need the following:

ACTIVITIES

In order to address our

problem or asset we will

accomplish the following

activities:

OUTPUTS

We expect that once

accomplished these

activities will produce

the following evidence

or service delivery:

SHORT- & LONG-TERM

OUTCOMES

We expect that if accom-

plished these activities

will lead to the following

changes in 1–3 then 4–6

years:

IMPACT

We expect that if accom-

plished these activities

will lead to the following

changes in 7–10 years:

Logic Model Development

Program Implementation Template – Exercise 1 & 2

Forms Appendix

Exercise 1 Checklist

Logic Model Development GuidePage 55

Progress Toward Results Quality Criteria – 1

1. A variety of audiences are taken into consideration when

specifying credible outputs, outcomes, and impacts.

2. Target participants and/or partners are described and quanti-

fied as outputs (e.g. 100 teachers from 5 rural high schools).

3. Events, products, or services listed are described as outputs

in terms of a treatment or dose (e.g. 30 farmers will partici-

pate in at least 3 sessions of program, or curriculum will be

distributed to at least 12 agencies).

4. The intensity of the intervention or treatment is appropriate

for the type of participant targeted (e.g. higher-risk partici-

pants warrant higher intensities).

5. The duration of the intervention or treatment is appropriate

for the type of participant targeted (e.g. higher-risk partici-

pants warrant longer duration).

6. Outcomes reflect reasonable, progressive steps that partici-

pants can make toward longer-term results.

7. Outcomes address awareness, attitudes, perceptions, knowl-

edge, skills, and/ or behavior of participants.

8. Outcomes are within the scope of the program’s control or

sphere of reasonable influence.

9. It seems fair or reasonable to hold the program accountable

for the outcomes specified.

10. The outcomes are specific, measurable, action-oriented, real-

istic, and timed.

11. The outcomes are written as change statements (e.g. things

increase, decrease, or stay the same).

12. The outcomes are achievable within the funding and reporting

periods specified.

13. The impact, as specified, is not beyond the scope of the pro-

gram to achieve.

Comments/

Revisions

Not

Yet

Yes

■■ ■■

Exercise 2 Checklist

Logic Model Development GuidePage 56

Theory into Action Quality Criteria

1. Major activities needed to implement

the program are listed.

2. Activities are clearly connected to the

specified program theory.

3. Major resources needed to implement

the program are listed.

4. Resources match the type of program.

5. All activities have sufficient and

appropriate resources.

Comments/Revisions

Not

Yet

Yes

■■ ■■

Forms Appendix

Logic Model Development GuidePage 57

Strategies

Assumptions

Problem or Issue

Influential

Factors

Desired Results

(outputs, outcomes,

and impact)

Community Needs/Assets

Logic Model Development

Program Planning Template – Exercise 3

Exercise 3 Checklist

Logic Model Development GuidePage 58

Exercise Three Checklist

1. The problems to be solved/or issues to be addressed by

the planned program are clearly stated

2. There is a specific, clear connection between the

identified community needs/assets and the problems

to be solved (or issues to be addressed).

3. The breadth of community needs/assets has been

identified by expert/practitioner wisdom, a needs

assessment and/or asset mapping process.

4. The desired results/changes in the community and/or

vision for the future ultimately sought by program

developers are specific.

5. Influential factors have been identified and cited from

expert/practitioner wisdom and/or a literature review.

6. Change strategies are identified and cited from

expert/practitioner wisdom and/or literature review.

7. The connection among known influential factors and

broad change strategies has been identified.

8. The assumptions held for how and why identified

change strategies should work in the community are

clear.

9. There is consensus among stakeholders that the model

accurately describes the proposed program and its

intended results.

Comments/

Revisions

Not

Yet

Yes

■■ ■■

Forms Appendix

Logic Model Development GuidePage 59

Logic Model Development

Evaluation Planning Template – Exercise 4

Evaluation

Focus Area

Audience Question

Use

Exercise 4 Checklist

Logic Model Development GuidePage 60

Posing Questions Quality Criteria Yes Not Comments/

Yet Revisions

1. A variety of audiences are taken

into consideration when

specifying questions.

2. Questions selected are those with

the highest priority.

3. Each question chosen gathers

useful information.

4. Each question asks only one

question (i.e. “extent of X, Y,

and Z”) is not appropriate).

5. It is clear how the question

relates to the program’s logic

model.

6. The questions are specific about

what information is needed.

7. Questions capture lessons

learned about your work along

the way.

8. Questions capture lessons

learned about your program

theory along the way.

■■

Forms Appendix

Logic Model Development GuidePage 61

Focus Area

Question Indicators

Technical Assistance

Needed

Logic Model Development

Indicators Development Template – Exercise 5

Exercise 5 Checklist

Logic Model Development GuidePage 62

1. The focus areas reflect the

questions asked by a variety

of audiences. Indicators

respond to the identified focus

areas and questions.

2. Indicators are SMART–

Specific, Measurable, Action-

oriented, Realistic, and Timed.

3. The cost of collecting data on

the indicators is within the

evaluation budget.

4. Source of data is known.

5. It is clear what data collection,

management, and analysis

strategies will be most

appropriate for each indicator.

6. Strategies and required

technical assistance have been

identified and are within the

evaluation budget for the

program.

7. The technical assistance

needed is available.

Establishing Indicators

Quality Criteria

Yes Not

Yet

Comments/

Revisions

■■

Forms Appendix

EV3815

Item #1209

0305 5M SCG

Printed on Recycled Paper

One Michigan

Avenue East

Battle Creek, MI

49017-4012

USA

269-968-1611

TDD on site

Facsimile: 269-968-0413

www.wkkf.org

W.K.KELLOGG

FOUNDATION