Expert Tips on Evaluation
Grant evaluation is a crucial component of the application and implementation process. Most funders require a plan outlining how you will measure the success of your grant goals, objectives and activities and what level of effectiveness will be fulfilled utilizing grant funding. A good evaluation plan produces objective information on the degree of accomplishments within the funded program, which in turn solidifies your commitment to the program, and the funder. Funders are accountable to their stakeholders and they utilize evaluation findings to relay the effectiveness of the program(s) they have funded.
Formulating SMART objectives will position you for a strong evaluation section, and thus, a funding-worthy project. Objectives are significant tasks needed to reach a goal or long-term strategy, as opposed to a goal, which is the outcome desired. Objectives are more specific than goals and are many times needed to reach a goal.
SMART stands for: Specific, Measurable, Achievable, Relevant and Time-bound. Let’s use SMART objectives for a project we’ll call the “After School Program.”
Specific: Make your objectives specific and concise by describing them in detail. What is your After-School Program’s design? What type of subject matter will be offered? What age groups, days and hours? Will there be a summer program?
Measurable: Objectives should be quantitatively, or descriptively measurable by creating mechanisms to track progress. Continuing to use the After-School Program example, a main objective could be to serve 75 students with a goal of improving literacy skills by 10%, or increasing the graduation rate by 10%. How do you measure success? You’ll need to consider why you want to monitor the program, exactly what will be monitored, and what the key features of effective monitoring will be. Effective monitoring is achieved when 1) the evaluation uses scientifically-valid techniques; 2) project-relevant aspects are measured; 3) evaluation is conducted consistently; 4) records are kept and 5) when monitoring is used to support or amend project actions and goals (such as a 10% growth in literacy skills or graduation rate).
Achievable: Objectives should be achievable with the project’s available resources. While they should require effort, they shouldn’t be difficult to the point where they will be unattainable. For our After-School Program, for instance, we’ll need to consider if we have an adequate facility and qualified staff to achieve the goal of increasing the literacy rate or graduation rate for students. Don’t promise 100% if you can only achieve 75%, or you will be setting yourself up for failure.
Relevant: Objectives should be aligned with larger goals / strategies. Now that the Boys and Girls Club down the street just opened an After-School Program at the YMCA, is my project still relevant? Will I be serving the same population?
Time-Bound: A very clear timeline that is tied into the activities, goals and outcomes should be completed so the call to action for the project is clear. Our After-School Project will need to be ready to roll when the school year starts and coincide with the district’s schedule.
Qualitative (Formative) and Quantitative (Summative) Research: To learn as much as possible about your target audience and obtain a favorable outcome, it’s important to incorporate qualitative and quantitative research to understand your target audience. Qualitative provides immediate feedback to improve leaning, while Quantitative evaluates what was learned.
Qualitative data collection is a method where the characteristics, properties, qualities, attributes, etc. of something is described (using data other than numbers). It deals only with data that can be observed (ie., beauty, texture, taste, smell, etc.) but is not measured. Quantitative data collection is a method by which data that can be numerically counted or expressed is collected. It is used in experiments and manipulated analysis using tables, charts, histograms and graphs. The measurements reflected include volume, area, height, length, etc. Many prefer quantitative research because it appears to be straightforward and simple – the hard numbers are more scientific / credible than insights gained from qualitative research. Quantitative research answers how many / how much and offers solutions. At the beginning of research, a hypothesis is identified, whereas qualitative doesn’t need a hypothesis to begin the process.
The goal of qualitative assessment when evaluating an education program is to monitor student learning and provide feedback to instructors. Where are students struggling? How can the problems be addressed? Qualitative assessment helps students identify their strengths and weaknesses and pinpoints areas that need work. Examples could include asking students to present their understanding of a topic in class or summarize the main points of a lecture. The goal of qualitative assessment is to evaluate student learning by comparing it to a benchmark or standard. Examples could include a final paper or project or mid-term exam. This information can be used as a guide for future courses / goals.
Using Evaluation Data in a Cycle of Improvement: Data processing is the conversion of raw data to relative information – manipulated to produce results that lead to improvement or resolution of a problem. The Data Processing Cycle includes: 1) Collection, 2) Preparation, 3) Input, 4) Processing, 5) Output, 6) Interpretation 7) Storage. Each step in the cycle must be taken in order, but the process is cyclical, as the end of one investigation results in additional questions that must be fed through the cycle again. This data is compiled into reports that are required by funders. Quarterly Reports are the most important outcome of an evaluation, which creates information that will be used to improve programs. This information will be included in monthly, quarterly and / or annual reports to the funder. It is critical the reports are precise and well-written, quickly and accurately communicating credible findings.
There are many ways to evaluate a project, which depend upon its purpose. It is critical that evaluation involves valid, sound methods for gathering information and analysis. Great expense is not needed, but depth, clarity and credibility are required.
Resource Associates has over 20 years of experience with grant writing and evaluation programs. Our Program Evaluators are subject matter experts with decades of experience. Contact us today for more information.