Title I School Improvement 1003(a) Grant Writing Workshop Building

Title I School Improvement 1003(a) Grant Writing Workshop Building a Logic Model With SMART Objectives Anduamlak Meharie Office of Research

Logic Model “A logic model is a systematic and visual way to present and share your understanding of the relationships among the resources you have to operate your program, the activities you plan, and the changes or results you hope to achieve” W.K. Kellogg Foundation Logic Model Development Guide. “A logic model’s purpose is to communicate the underlying ‘theory’ or set of assumptions or hypotheses that program proponents have about why the program will work, or about why it is a good solution to an identified problem” http://www.insites.org/documents/logmod.htm

Logic Model: Purpose Design and planning – To clearly communicate the central goal of an initiative, why it is important, and what will result from it. – To select specific and measurable objectives expected to lead to the desired result. – To identify activities and resources needed to meet selected objectives. Implementation – To identify and collect data need to effectively monitor initiative activities and improve programming. – To create a common reference point for everyone involved in the initiative. Evaluation – To help determine if activities are being implemented as planned. – To help determine if the initiative is making progress. – And, ultimately, to determine if the initiative has led to the desired result.

Logic Models: Components Goal(s) Objective(s) Activities Input/Resources Outputs Outcome(s)/Impact(s) Indicator(s)

Logic Model: Goals vs. Objectives Goal: A broad statement of intended outcomes for a program, line of business or an organization. – Goals are typically intended to guide the formation of (more specific) objectives that can be linked to goals. (McDavid and Howthrorn, 2006) Example – To decrease the dropout rate of at-risk students in grades 9-12 at Washington High School.

Logic Model: Goals vs. Objectives Objective: A statement of intended outcomes that is focused and time-specific (that is, achievable in a specified time frame). (McDavid and Howthrorn, 2006) Example – By September 2013, at least 80% of all at-risk students in grades 9-12 will be assigned an adult advocate who will be responsible for meeting the student at least twice a week to address academic and social needs, and communicate with families. – By May 2014, all core curriculum teachers at Washington High School will receive evidence- based professional development on (1) how to help struggling students, (2) differentiated instructional methods, and (3) managing student behavior.

S.M.A.R.T. Objective Specific Measurable Attainable Result-oriented and relevant Time-bound

Specific Objective Is your objective Specific? If you have a specific objective, you should be able to answer the following six questions: – What: What do you want to accomplish? – Why: What are the reasons, purpose or benefits of accomplishing the objective? – Who: Who is involved? Who are the stakeholders? – Where: Where is it going to happen? – Which: Which attributes are important? (Requirements and constraints; risk and protective factors).

Measurable Objective Is your objective measurable? If your objectives are specific and measurable, you should be able to establish concrete criteria for measuring progress toward the attainment of each. – How will you know when it is accomplished? – What indicators will you look for to measure progress and success? – Are data to measure your progress and success readily available? Or do you need to develop new measures, e.g., surveys, focus groups, etc ?

Attainable Objective Is your objective attainable? Given resources available to you, the objective must be realistic and attainable. – Do you have sufficient financial capacity to accomplish your objective? – Do you have personnel with sufficient abilities and skills to accomplish your objective (including evaluation)? – Do you have sufficient time to accomplish your objective? – What other types of resources do you need to attain your objective (i.e., technology, space, equipment, etc )?

Result-Oriented and Relevant Objective Is your objective result-oriented and relevant? Ultimately, your objectives should help you achieve your major project goal(s). – Is it worthwhile? – Is this the right time? – Does this align with other efforts/needs?

Time-Bound Objective Is your objective time-bound? Objectives should have starting points, ending points, and fixed durations. – When will you achieve this objective? – When will you undertake activities to achieve our objective? – When can you expect to see some short-term outcomes?

S.M.A.R.T. Objective: Example Goal 1: Within two years after the coaching program for at-risk youth at Washington High School is established, the dropout rate of high school students will fall below 1% at grades 10, 11, & 12. Objective 1.1: As a result of the program attendance rates at Washington High School will be at 100%. Is the objective Objective Specific? Measurable? Attainable? Result-oriented/ Relevant? Time-Bound? As a result of the program attendance rates at Benedictine High School will be at 100%. Yes Yes (X) Yes Yes (X) Yes No, revise (X) No, revise No, revise (X) No, revise No, revise (X) Attendance rates at Benedictine High School will exceed 97% one year after the apprenticeship/ coaching program is established. Yes Yes (X) Yes (X) Yes (X) Yes (X) No, revise (X) No, revise No, revise No, revise No, revise Attendance rates for at-risk youth at Benedictine High School at grades 10, 11, & 12 will exceed 97% one year after the apprenticeship/ coaching program is established. Yes (X) Yes (X) Yes (X) Yes (X) Yes (X) No, revise No, revise No, revise No, revise No, revise

S.M.A.R.T. Objective: Example Objective 1.1: As a result of the program attendance rates at Benedictine High School will be at 100%. Objective 1.1: Attendance rates for at-risk youth at Benedictine High School at grades 10, 11, & 12 will exceed 97% one year after the apprenticeship/ coaching program is established.

S.M.A.R.T. Objective: Exercise 1. Choose one of the objectives listed on your grant application. 2. Using the worksheet provided, examine your objective to ensure that it is S.M.A.R.T. with respect to the major goal(s) of your project. 3. Revise as needed until the objective meets all five (S.M.A.R.T.) criteria.

S.M.A.R.T. Objective Worksheet Goal: Objective: Is the objective Objective Specific? Measurable? Attainable? Result-oriented/ Relevant? Time-Bound? Yes Yes Yes Yes Yes No, revise No, revise No, revise No, revise No, revise Yes Yes Yes Yes Yes No, revise No, revise No, revise No, revise No, revise Yes Yes Yes Yes Yes No, revise No, revise No, revise No, revise No, revise Yes Yes Yes Yes Yes No, revise No, revise No, revise No, revise No, revise

Logic Models: Components Goal(s) Objective(s) Activities Input/Resources Outputs Outcome(s)/Impact(s) Indicator(s)

Logic Models: Components Activities: the actions that are needed to meet your objectives. Resources: inputs needed design and implement activities. Outputs: the tangible and direct results of activities. Outcome(s): specific, attainable and measurable changes that are likely to occur as a result of activities. Indicator(s): data to monitor implementation, and measure progress as well as success of initiative.

Logic Model Goal 1: Objectives Activities Inputs/Resources Outputs Outcomes Indicator/Data Source (Specific, Measurable, Achievable, Relevant, and Time-bound) (what we do to achieve objectives/goals) (human, financial, technology, etc resources needed for activities) (tangible and direct products of activities that lead to desired outcomes) (short-, medium-, and longterm desired results of activities) (indicators we look for to measure implementation, progress and success) 1.1 1.2 1.3 1.1.1 1.1.1 1.1.1 1.1.2 1.1.2 1.1.2 1.1.3 1.1.3 1.1.3 1.2.1 1.2.1 1.2.1 1.2.2 1.2.2 1.2.2 1.2.3 1.2.3 1.2.3 1.3.1 1.3.1 1.3.1 1.3.2 1.3.2 1.3.2 1.3.3 1.3.3 1.3.3

Logic Model: Example Goal 1: To decrease the dropout rate of at-risk students in grades 9-12 at Washington High School. Objectives 1.1 By September 2013, at least 80% of all at-risk students in grades 9-12 will be assigned an adult advocate who will be responsible for meeting the student at least twice a week to address academic and social needs, and communicate with families. Activities 1.1.1 Identify at-risk students Inputs/Resources -School staff, community members, and/or social workers to work as 1.1.2 Identify, recruit, and advocates train adult advocates -Monetary compensation 1.1.3 Clearly define the for trainers and advocates. roles and responsibilities of all stakeholders -Cost of training facility, (guiding document) printed materials, meals for participants. 1.1.4 Assign 10 or less atrisk students per advocate -Staff to document, monitor and evaluate implementation and progress. Outputs 1.1.1 Number of at-risk students identified by grade level Outcomes 1.1.1 By June 2013, at-risk students in grades 9-12 will indicate a greater sense of school membership and involvement. 1.1.2 Number of advocates; list of training dates, locations, 1.1.2 By June 2013, at-risk and participants; students in grades 9-12 will have reduced risky behaviors and 1.1.3 A document that absentee rate. elucidates the roles and responsibilities adult 1.1.3 By June 2014, at-risk advocates, students, students in grades 9-12 will have school staff, improved grades. parents/guardians, and Ultimate outcome (Impact): The other stakeholders sum total of all objectives- will 1.1.4 List of advocates result in a decrease in dropout and their assigned atrate of at-risk students in grades risk students; number 9-12. and average duration of meetings with each student; number and average duration of meetings with parents of at-risk students Indicator/Data Source -Student survey to measure effect on sense of school membership and involvement (maybe pre/post). -Student disciplinary and attendance record -Student academic record. -List of all at-risk students and those who participated in the adult advocate program to ascertain the % of at-risk students reached. -Participant questionnaire about the quality and usefulness of training and guiding document. -Student and parent survey to gather feedback about the quality and usefulness of the adult advocate program.

Logic Model: Exercise Insert the objective you developed earlier in the logic model template. Identify activities and inputs/resources needed to achieve the objective. Explicitly state expected results of activities in terms of outputs and outcomes. Identify indicators/data sources that will allow you to document and monitor implementation as well as measure progress and success.

Logic Model Goal 1: Objectives Activities Inputs/Resources Outputs Outcomes Indicator/Data Source (Specific, Measurable, Achievable, Relevant, and Time-bound) (what we do to achieve objectives/goals) (human, financial, technology, etc resources needed for activities) (tangible and direct products of activities that lead to desired outcomes) (short-, medium-, and longterm desired results of activities) (indicators we look for to measure implementation, progress and success) 1.1 1.2 1.3 1.1.1 1.1.1 1.1.1 1.1.2 1.1.2 1.1.2 1.1.3 1.1.3 1.1.3 1.2.1 1.2.1 1.2.1 1.2.2 1.2.2 1.2.2 1.2.3 1.2.3 1.2.3 1.3.1 1.3.1 1.3.1 1.3.2 1.3.2 1.3.2 1.3.3 1.3.3 1.3.3

Evaluation It involves looking at your program and asking – Are we doing what we said we were going to do? – Is what we’re doing working? – How do we know it’s working? – Did we accomplish what we set out to accomplish? It is part of what you put in place beforehand to help you run your programs. – It should be about looking ahead not backwards. It is part of the continuum of planning (rolling evaluation).

Evaluation It is a systematic process of asking questions, collecting information, and using the answers to Set realistic goals and fine tune strategic decisions. Measure progress and identify areas of improvement. Identify training and technical assistance needs. Be accountable and be credible to your constituents. Motivate by providing documentation of your achievements. Guide budget and resource allocation. Generate support for programs. Be an efficient organization that works toward its mission.

Formative vs. Summative Evaluation Formative evaluation (implementation and progress) How well were activities implemented (fidelity)? Was the target audience reached? What did they think about the activity? How are resources allocated and used for activities? What were other products of those activities. How are preliminary evaluation findings used to improve implementation of activities? What obstacles were encountered and how were they overcome?

Formative vs. Summative Evaluation Summative evaluation (Outcome) Are the initiative goal(s) and objectives being achieved? How fully are they being achieved? Do activities have beneficial effects on at-risk students? If so, what are they? Do activities have any adverse side effects on the atrisk students? If so, what are they? Are some at-risk students affected more by the services than others? Can the results be traced to the initiative itself? (Is a control group possible?)What other factors may have caused the results?

Evaluation: Planning What do you really want to know and for what purpose? What evidence or indicators do you need to answer your questions? Who’s in charge of evaluation? (roles and responsibilities) What time frame are you evaluating? How much time is allocated to conducting evaluation and when? What is the budget for evaluation? Commit the plan to paper!!!!

Evaluation Plan Matrix Evaluation Timeline Objective Source of Data Responsible Person (s) Instrument development Data collection Data analysis Reporting

Next Steps Examine your remaining objectives and ensure that they are all S.M.A.R.T.! Insert each objective in the logic model template Identify activities and inputs/resources needed to meet objectives. Explicitly state expected results of activities in terms of outputs and outcomes. Identify indicators/data sources that will allow you to document and monitor implementation as well as measure progress and success.

Next Steps Look at each of your completed SMART objectives and identify data sources, time lines, and assign responsible parties for collecting, analyzing, and reporting data about your project. – If necessary, assign responsibility for the design of new measures (for example, surveys). Define a time line for completion, administration, and analysis. As you assign responsibility, remember, evaluation is a team effort, not just one person’s job. Put evaluation on your upcoming meeting agendas. Always take some time to talk about where you are in your project, what data you have collected, what it is telling you and what you still need to know.

Thank You Anduamlak Meharie Coordinator – Research and Evaluation WVDE Office of Research http://wvde.state.wv.us/research ph: 304.558.2546 fax: 304.558.1613 [email protected]