Secondary Menu

The Evolution of Evaluation: Part 1

The evolution of evaluation is creating quite a stir within some segments of the grant community, because vast changes have taken place. Any grant professional with an award or contract issued under the new Uniform Grant Guidance (UGG) rules and regulations should definitely take note of the differences between “then and now.”

The publication of the new UGG is the first effort of the federal government to improve how it evaluates the use of billions of taxpayer dollars since the 1970s. The revisions are designed to reduce the risk of waste, fraud, and abuse by recipients—a great goal that I’m certain we can all agree with.

Previous to the publication of the new guidelines, federal funding agencies didn’t assess the risk of awarding funds to the recipients of their dollars. Now, under the new guidelines, each federal agency awarding grants and contracts must perform a risk assessment of every applicant organization before monies are awarded.

Biggest Change is in the Philosophy of Evaluation

The change in philosophy by the entire federal funding community is extremely important to note. Under the new UGG, there is now an intensified accountability of both performance indicators and fiscal accountability.

I suggest that you read that last sentence over and over until you have it memorized and ready to recite to any of your administrators or program staff members who think almost achieving your performance indicators is acceptable. Because what was once okay is no longer okay.

For example, I know an organization that was awarded a U.S. Department of Labor grant. When, near the end of its grant, it fell short of its projected number of trainees, it was instructed to request a “no cost extension.” This extension didn’t include any additional funds. The extension required, however, that it serve the additional trainees needed to reach the contracted deliverables. So even though it was out of grant funds, it was required to continue providing the program until it delivered the promised outcomes.

Obligations of Federal Funding Agencies

Under the new UGG regulations, the duties of the federal funding agencies are greatly enhanced. Now they must provide applicants much more vigorous guidance during the application process.

Performance Measurements

Federal funding agencies must now provide grant applicants clear performance goals, indicators, and milestones. This means that applicants will no longer be able to provide vague or ambiguous projections. Everyone will be playing on an equal field because the primary deliverables will be set for all applicants.

In my opinion, this is a fantastic change. As a reviewer, I have seen proposals funded that promised retention rates of 70 percent and higher for at-risk populations. Quite impressive, one might think. But upon further examination, the applicants were combining all participant retention calculations together to achieve these impressive performances (as illustrated in the Combined Retention Rate listed in the table).

However, upon close examination, it became obvious that those actually being retained throughout the programs and successfully completing the four-year programs were minuscule in comparison (as calculated in the True Retention and Completion Rate in the table).

Table - Examples of different reporting calculations

Furthermore, from what I have read over the past few years, Congress was not amused when it examined the true success rates of many of the federally funded programs. And I’m confident that its displeasure is why federal agencies are being required to assess applicant risks and to provide uniform deliverables and calculation formulas for their programs.

Best Practices

Federal agencies are now responsible for making available the detailed information about the best practices that they have identified for their programs. While in years past it could be quite difficult to identify which strategies were the most successful in bringing about desired changes, it will be much easier for applicants to find these invaluable resources. In fact, you may have already noticed that nearly all federal program announcements include links to web addresses highlighting the best practices that they hope will be incorporated into submitted proposals.

More Bang for Their Bucks

With the federal debt increasing every minute, Congress is seeking ways to ensure that taxpayers’ monies are not only being spent on good programs, but are maximizing the outcomes achieved from every dollar spent. Who can blame it? Certainly not me, because I always want the best bargain I can negotiate when I spend my money, too.

The New Risk Assessments

The federal agencies are now required to perform risk assessments of every selected awardee before an award is offered or finalized.

  • Past Performance— This means that agencies will be assigning risk ratings based on applicants’ past performance—something that some organizations should be at least a little worried about. For example, if your organization received grant funds under any of the U.S. Department of Education’s Trio programs and didn’t perform your promised deliverables repeatedly, you could be reviewed more carefully even though the department didn’t penalize you previously.Mind you, I’m not really talking about those applicants that missed their targeted deliverables by very small amounts. I’m referring to those that promised in their narrative applications (which became part of their award package) that, in order to earn good scores from the reviewers, they would achieve degrees of change that were unrealistically high. As a previous reviewer, I have seen this myself.A few years back, the Department of Education changed its terminology in its program announcements to require “ambitious but attainable” outcomes. However, many applicants persisted in promising to reach the moon, the stars, and the sun. If your organization was one of those, you may find yourself being considered “at risk” of poor future performance.
  • Past Fiscal Responsibility— Funding agencies will be reviewing your spending habits. If you’ve spent your funds as projected and with no audit findings in the past, you’ll do fine under the new risk-assessment policy. But if you have made a habit of not drawing down grant dollars in a timely manner or postponing expenditures until the last quarter of your grant period, you could be in trouble.In particular, I have heard that agencies will be noting last-minute equipment purchases that were made so late that they cannot be used in the grant program. For example, if you had a job training grant and you bought training equipment near the end of the grant period, your trainees didn’t really benefit from the equipment, did they? This could be taken as a serious red flag and you’d better be ready with a really good justification for the delay in purchasing.
  • Posting of Suspensions & Debarments— Per 2 CFR Part 180 those entities that have been suspended or debarred from receiving federal grants or contracts will have their names publicly displayed on the Internet for all to see. Yes, your collaborators and partners will easily be able to access this information, as will every federal funding agency. The information will be available through the Federal Awardee Performance and Integrity Information System.

In Part 2, I will discuss the changing obligations of grant awardees and how program logic tables will become a much more familiar tool for illustrating the connection between promised deliverables and budgetary requests. In Part 3, I will cover common risk factors that could increase the likelihood of your agency being audited, common grant myths not to fall for, and the top audit findings that you need to avoid.

Rebecca Vermillion Shawver, MPA, GPC

About the Contributor: Rebecca Vermillion Shawver, MPA, GPC

Rebecca Vermillion Shawver possesses an MPA degree from Indiana University School of Public and Environmental Affairs at Indianapolis campus.

She is the Director of Grant Administration at Brazosport College. Her duties include assisting college administrators, faculty and staff members in the development of federal, state, corporate and private foundation grant applications; developing the conceptual aspects of proposals through the proposal review process and the analysis of statistical data; developing proposal budgets and determining personnel, equipment, and other costs to be charged to funding agencies; researching funding opportunities; conducting proposal writing and other grant-related workshops for college personnel; assisting project directors in monitoring funded proposals; and publishing a grant blog.

Rebecca is an active member of the Grant Professionals Association and has earned her GPC. Additionally, she is a member of the Council for Resource Development (CRD) for community college fund development professionals. She is a past CRD Director for Region VI and a past member of the CRD National Board. She has presented numerous workshops at multiple regional and national CRD conferences.

A native-born Hoosier, Rebecca continues to serve as a consultant for agencies located in the State of Indiana. Additionally, she is the author of What Funders Want: Developing Evaluation Plans to Support Your K-12 Grant Applications (LRP Publications).

Rebecca is a dedicated volunteer working with the AFS Intercultural Programs. As the Volunteer Chair for the AFS Cradle of Texas Team, she is responsible for implementing and coordinating the AFS high school exchange program in part of Harris and all of Brazoria County, Texas.

Having hosted twenty-two exchange students over the past twenty-five years, she and her husband are currently the proud host parents of AFS daughters from Chile and Turkey.

No comments yet.

Leave a Reply

Pin It on Pinterest