Step 1 of Six Steps to Effective Program Evaluation: Collaborate with Stakeholders
A lot of people think of program evaluation as being about measurement and data. Measurement may be a very important part of evaluation, but it’s just one of several parts of an effective evaluation—one that gets you information that you can use to strengthen and grow your program and increase your positive impact.
In this article, I’ll cover the first of six steps to an effective evaluation: collaborating with stakeholders.
Collaborating with Stakeholders: A Crucial Step to More Effective Evaluation
Collaborating with stakeholders makes your evaluation more effective in several ways:
- More meaningful results. Listening to stakeholders makes your evaluation more meaningful by helping to focus the evaluation on the outcomes and research questions that matter.
- More accurate results. We each have our own experiences and cultural backgrounds that affect our understanding of what we hear and read. Getting all stakeholders’ perspectives helps us to avoid errors based on misperceptions and misunderstandings.
- More ethical evaluation. When we’re using funds for evaluation that could be going to directly help people, making sure that the evaluation responds to all stakeholders’ perspectives and interests is the ethical thing to do. The American Evaluation Association’s Guiding Principles for Evaluators states that evaluators should include “relevant perspectives and interests of the full range of stakeholders.”
Steps to Including All Stakeholder Perspectives in Your Evaluation
Below are a few steps that you can take to include stakeholder perspectives in your evaluation, based on some techniques I’ve seen work in many evaluation projects.
Identify Stakeholder Groups
In collaboration with evaluation funders and your evaluation team, identify the groups of people who have a stake in your program and its evaluation. These categories of stakeholders are often relevant:
- People you serve/program participants. Program participants can provide valuable insights from their first-hand experience. Yet, too often, they are left out of evaluation studies.
- People next to them. Advocates, family members, and other representatives are often important to include because they also have experience interacting with your program, and their lives are also impacted by your program.
- People near you. Partnering organizations, funders, and policymakers add useful perspectives. Their insights help explain your program’s context and operating environment. Plus, including them in evaluation planning increases your chances for getting buy-in for carrying out evaluation recommendations.
- Your own people . Be sure to include a variety of perspectives from within your organization who has experience and knowledge of your program—such as managers, staff, volunteers, board members, and consultants.
Devise Strategies to Involve Stakeholders
Stakeholders can potentially participate in evaluation in many ways—as advisors, paid evaluation staff, or coevaluators/evaluators. The best approach will depend on your specific situation. Two techniques that I’ve seen work well to get stakeholder perspectives in many evaluations are evaluation advisory groups and stakeholder interviews/focus groups conducted early in the evaluation.
Forming an evaluation advisory group (also called an advisory panel) is a way to formally involve stakeholders in the evaluation. This is a group of stakeholders who agree to meet at various points in the evaluation to discuss the evaluation and to review and comment on draft evaluation materials. The meetings may be in person, online, or by conference call. Some evaluation studies have two or more advisory groups, such as a group of program staff, a group of people who use the services, and a group of policymakers.
Another way to involve stakeholders is to hold individual interviews or focus groups with stakeholders early in the evaluation. Then you can use ideas gained from those conversations to refine plans for the rest of the evaluation.
Be careful to set up stakeholder meetings so that everyone can fully participate. Several issues are often important to consider:
- Selecting a meeting location that’s convenient and accessible . For example, don’t meet five miles from the nearest bus stop when people are relying on public transportation. If you’re considering an online meeting, check that everyone has the needed technologies.
- Ensuring accessibility for people with disabilities . Choose a space that is designed to be accessible for all. Check with participants to find out any accommodations they may need.
- Providing information in a clear, easy -to-understand way. Avoid speaking in research-ese. Make evaluation-related information available in a way that is clear and understandable for all audiences.
Collaborate with Stakeholders and Incorporate Their Perspectives Throughout the Evaluation
Stakeholders can provide valuable ideas and perspectives at several points throughout planning and conducting your evaluation:
- Understanding the program. Sometimes people have different views of what effects a program is having and how. Also, stakeholder views of a program’s activities and goals may be different from the written activities and goals, such as those stated in your grant proposal or strategic plan. Asking stakeholders their perspectives on what your program is doing and achieving lets you make sure that the evaluation is based on a realistic understanding of how your program functions.
- Refining data collection strategies. Before you start sending out surveys or other data collection, talk to people and ask their thoughts about research plans. For example, you might be considering a mail survey, but find out that many of the people who you need to reach don’t check their mail, that text messaging is a better way to reach them. Or, you might learn that people would be uncomfortable participating in interviews at a government building, that they’d be more comfortable talking at a local restaurant. You can use this information to plan your research strategies to maximize response.
- Communicating evaluation results . Stakeholders can also provide ideas for effective ways to communicate the knowledge you gain from your evaluation. For example, some audiences may prefer to see information presented with a lot of charts and pictures and little text. Other stakeholders might find information presented verbally or in a newsletter article more useful. You could also ask stakeholders to review and provide feedback on draft communications, to help you ensure that the information is clear, user friendly, and useful.
Involving additional stakeholders does more than create a better evaluation. Collaborating with stakeholders also builds appreciation of your efforts and shows your dedication to the project and the larger community. People love it when we listen.
Take the first step to effective program evaluation that gives you information you can use to make a bigger difference. Incorporate these steps to involving all stakeholders’ perspectives into your next evaluation.