Invest in Neighborhoods Inc. Invest in Neighborhoods Inc. In writing grants for government agencies, the evaluation section of the narrative has become more and more important to the success of the proposal. The trend toward demonstration grants and the desire of the government to provide models for other communities or agencies to duplicate has caused the evaluation to become a major component of the grant application.
Government grant reviewers indicate that about 60 percent of the grant applications received have inadequate or incomplete evaluation components. Some of the evaluation problems include inadequate detail, lack of expected outcomes, inappropriate measure, lack of consultant detail, lack of an evaluator, lack of planning for staff and resources to conduct the evaluation, and poor evaluation design.
The evaluation of the project is actually the collection and use of information, to determine whether the program has been successful in meeting the goals and objectives. By gathering such information, the organization is assuming a proactive position on problem-solving and future planning. An objective evaluation is the only viable mode for assessing both the short- and long-term effectiveness of a project.
The evaluation must be planned as the goals, objectives, and activities are developed. As the objectives are established, they must be made measurable for the evaluation component. Using the program model will lead the applicant to the evaluation considerations. This model provides information for the process component -- the program of services or activity -- as well as the short- and long-term outcomes.
Information for the Efficiency Evaluation can be ascertained from a review of the Resources.
The evaluation process will begin at the onset of the project. Data must be collected throughout the project, in order to determine the effectiveness of the program. For this reason, the evaluation plan must be fully developed, including the decisions on what data to collect and how they are to be collected and evaluated.
Many organizations contract for an outside evaluator to perform the evaluation component of the project. The outside evaluator will usually assist in writing the evaluation section of the narrative and will work with the program throughout its duration, to ensure that the appropriate data are collected and made available for analysis. Many colleges and universities have faculty or graduate assistants who can perform the evaluation and provide a statistical analysis of the results. The use of an outside evaluator can add credibility to the proposal and will indicate a sincere approach to the evaluation.
Among the numerous evaluation designs, three are most widely used: process evaluation, outcomes evaluation, and efficiency evaluation. It is very common for the evaluation component of a grant proposal to combine two or more of these designs.
In process evaluation, the implementation of the project is documented. The focus here is on the process of the program, rather than the long-term effects of the program. The evaluation is an assessment of whether the project was implemented according to the planned approach (activities) through the collection of quantitative and qualitative data. In essence, the process evaluation determines whether the objectives were accomplished and what, specifically, contributed to the attainment of the objectives. For example, with the earlier objective ("60 percent of the participants will improve at least one letter grade ...") the process evaluation would document: the number of tutoring sessions that each student participated in, whether the tutoring plan corresponded with the teaching plan of the school, and other designated activities for the accomplishment of this objective.
Many evaluators use program models as part of the process evaluation: for each activity, a program model form is completed and filed under the objective. This allows the evaluator to review and evaluate all the activities that supported the accomplishment of each objective. The process evaluation can occur only if the objectives are expressed in specific, quantifiable terms. Each objective should have a corresponding set of activities or methods for accomplishing the objective.
In planning for the process evaluation, consideration must be given to how the information will be collected, who will collect it, and when it will be collected. The process evaluation allows the project director to know who will do what, at each location, and in each segment of the timeline. An ongoing data collection system that will serve as a tool for decision-making and future planning must be created and maintained.
The outcomes evaluation will attempt to determine whether the project has made a difference in the target population and what this difference is. In essence, it answers the question: Did accomplishing the objectives help to achieve the goals of the project? The data obtained in the outcomes evaluation will provide the basis for the final conclusions that are drawn. Outcomes evaluations attempt to ask probing questions. For the example given earlier, the question might be: Did the grades of the students improve? Was that improvement a result of the strong tutoring program?
In developing the evaluation plan for the outcomes evaluation, attention must be given to clearly articulating the goals of the program in both long- and short-term outcomes. Although many indices of success may be long-term, it is sometimes necessary to develop short-term measurements that can be indicative of the attainment of the goal. For example, if the goal is to reduce the dropout rate in a high-risk target population via an elementary school reading enhancement program, the real measurement is long-term because it will occur when the students reach high school graduation. However, some short-term indicators of school success and completion, such as higher grades, would be appropriate measurements of attainment of the goal.
It is important to plan for and identify the measurement tools that will be used in the outcomes evaluation. Interviews, questionnaires, observation, tests, records and statistics, and surveys are among the means of measuring the outcome of the project. Similarly, the design itself could include the use of pretests and posttests, the use of a control group, or an evaluation of change over time.
Although the planning for the outcomes evaluation must occur at the design state of the program, the actual performance of the outcomes evaluation will occur after the project has been implemented. With regard to the tutoring objective, it will be impossible to evaluate whether grades have improved until the tutoring activities have been implemented and grade reports have been compiled or testing has occurred.
The third type of evaluation, the efficiency evaluation, is used to assess whether the resources were used in the most efficient manner. A constant review of the procedures will highlight the ones that work most efficiently and will provide the best way to achieve the desired results. By means of the efficiency evaluation, the organization can review the activities and redirect them if they are not the most efficient manner of accomplishing the goals of the project. For example, it may be determined that producing and distributing 10,000 brochures was less cost-effective than purchasing one billboard on a main thoroughfare.
Whatever type of evaluation plan is used, it is most important that the evaluation become a part of the overall design of the program and that appropriate resources be budgeted to conduct the evaluation. The underlying reasons for conducting an evaluation are to determine whether a project has been successful and to develop ways for improving it and providing for program growth.
Back to Manual Table of Contents
