Why Do Evaluation?
For many grant professionals, evaluation competes with finances as the least favorite part of developing proposals. It seems terribly complex, it can feel like it’s forced on you by funders, and you may experience resistance from the program people whose cooperation you need to develop a good proposal. The evaluation section is treated as a necessary evil and an afterthought -- something you throw together after the real work of your proposal is done. As a result, it is often one of the weakest parts of many people’s grants. In a competitive funding round, a strong evaluation section may make the difference between getting funded or turned down.
Yet we have doubts. Most of us aren’t trained researchers and approach the subject with some skepticism. We know it’s going to take time and energy, and it’s not clear what it gets us. Let’s look at the pros and (perceived) cons of doing evaluation.
The primary reason for doing evaluation is that it gives you reliable information to improve your program and services to your clients. Evaluation can provide data on whether a program works and why, which parts of it are effective and which need improvement, whether it is the best use of your organization’s scarce resources.
The primary reason for including evaluation in grant proposals is that funders require it. Proposals with good evaluation sections score better and are more fundable than ones with poor (or no) evaluation. Being able to cite a positive prior evaluation of your existing program increases your chances of receiving future or continuation funding.
Notice that these are not the same. Fortunately, the stick of funder requirements and the carrot of program improvement both lead to the same place. The job of the grant professional is to take advantage of the requirement and use it to improve your grant proposals and therefore your programs and possibly your organization itself.
Beyond these two fundamental reasons for doing program evaluation, there are several more advantages to your organization both during grant development and during program implementation.
In grant proposals:
- Evaluation provides a framework for improving both your grant proposal and the project you’re seeking funding for, by providing measures to make your goals and objectives more meaningful.
- Evaluation provides a way to involve key stakeholders and direct service staff in program planning, increasing buy-in in your program. A positive evaluation can help your organization attract new staff, volunteers, funders and collaborators.
- Doing your own evaluation reduces the chances of funders or other outsiders undertaking their own evaluation of your program, which might be less informed and more detrimental to your organization.
- A strong scientifically based evaluation can turn your innovative project into a research study, and could lead to it being named an “evidence based practice.”
In program operations (after you get funded):
- Evaluation gives immediate feedback, allowing you to identify and fix problems in existing programs while you still have grant funding to implement the changes.
- Evaluation gives a mechanism/vehicle and techniques for getting feedback from your client/participants about their perceptions of your organization and project, and a way of letting them know how the project is working. An evaluation showing that your program works can motivate your existing clients and attract new ones.
- Evaluation provides valuable information for your organization to use for longer term strategic planning and program improvements.
- Evaluation provides solid data for disseminating information about your program, and for others who may want to replicate it elsewhere.
Resistance to Evaluation
Despite these advantages many nonprofit organizations and their staffs are resistant to, or suspicious of, evaluation for a variety of reasons. The grantwriter must often respond to these concerns, and even advocate for evaluation, in order to be able to develop a successful proposal. Here are some common objections to evaluation and responses you can use in overcoming to them. If these are your own beliefs, you’ll need to think them through before trying to convince others.
Objection # 1) Evaluation is a way to judge us, to label our program a success or failure, and perhaps punish us (i.e. cut our funding).
- Evaluators are looking for a perfect program and if we don’t have it we’ll get marked down.
- Many people’s only experience with evaluation has been employment performance reviews and college board tests. They are reluctant to be judged and ranked.
- If we can keep our program results fuzzy, “they” won’t notice and we can keep doing what we’re doing.
- By looking at what works and doesn’t work in a program, evaluation can be used to improve your services, not to punish your organization. Evaluation also helps you learn “for whom, where and under what circumstances did it work?”
- Funders, management and governing boards are looking for results. The lack of good program evaluation is worse than finding things that need improvement.
Objection # 2) Evaluation is forced on us by the funders for their own purposes (like punishing us, see # 1.)
- Evaluation is something we wouldn’t want to do funders didn’t make us. It has no intrinsic value to our organization.
- If you design your own evaluation you can have control over it, rather than ceding the control to others (like funders).
- If you do a good evaluation, it will help your organization improve its program, improve services to your clients, meet the funder’s needs and help you attract future funding.
Objection # 3) Money spent on evaluation is diverted from direct client services where it could be helping people.
- Evaluation is expensive.
- Evaluation is always competing for the same funds as services.
- Evaluation doesn’t help our clients.
- Evaluation doesn’t have to be expensive, and sometimes has separate funding.
- A positive evaluation can help you raise more funds in the future.
- Evaluation can improve services and outcomes for your clients, making direct service dollars more effective.
Objection # 4) We’re “people people,” not “science people,” and we don’t understand or like statistics.
- Evaluation is all abstract, dry statistical stuff, not relevant to the actual work we do.
- The outside evaluator will look down on us or manipulate the data to make us look bad.
- This is complicated and we can’t learn it.
- While evaluation was once heavily statistical, program evaluation is increasingly moving to qualitative measurement which looks at people’s actual experience and the impacts of programs on their lives.
- Getting clarity about the uses of data and intentions for using findings in advance can help both the evaluator and program staff do a better job.
- A good evaluation process is designed to include program staff, and a good evaluation report is written for lay people to understand.
Objection # 5) Evaluation will take staff time away from providing direct services.
- The program staff only want to do their regular jobs and aren’t interested in improving their services.
- The best use of staff time is working directly with clients, even if the work is less effective than it could be.
- Evaluation data collection will take some of your program staff time -- how much depends on the evaluation design.
- Staff time spent on a good evaluation will improve their worklives and the lives of their clients.
Objection # 6) This is different than the way we’ve always done things and we don’t understand or like it.
- If we change, it may be uncomfortable.
- The way we do things is the best it could be, it doesn’t need improvement.
- This feeling may be the underlying basis for some of the first 5 issues.
- Doing evaluation may lead to change and may be uncomfortable, but it’s worth it if the resulting improvements are real.
- With new information, nonprofits are discovering better ways of operating.
If you embrace evaluation and incorporate it into your grant development process, it will strengthen your whole proposal. You’ll develop more compelling needs statements, create stronger goals and objectives, and write a better narratives. Your proposals will be more fundable and once funded, the grant programs will be more successful.
In fact, your whole organization will benefit, because evaluation is becoming more important not just in grant proposals but to all aspects of operation. The organizations that survive and thrive will be the ones that understand, measure and work to improve themselves and their programs. And the way that’s done is through program evaluation.