Assessing Academic Programs at IPFW Overview of process, reporting

Assessing Academic Programs at IPFW Overview of process, reporting structure and support for programmatic assessment

Assessment Reporting Process Academic Program General Education Academic Program Submits Academic Department Assessment Report to College Assessment Committee College Level Assessment Committee Academic Program Prepares Assessment Summary by Course and Submits to General Education Subcommittee General Education Subcommittee College Level Assessment Committee Reviews Academic Department Assessment Report General Education Committee Reviews Course Level Assessments to Prepare Academic Assessment Report Assessment Council Assessment Council Reviews College Level Reports and Prepares Annual Report of Assessment Progress for Each College Assessment Council Reviews Academic Assessment Report Using IPFW Assessment Progress Worksheet for General Education Subcommittee

Annual Assessment Report Outline Section 1: Student Learning Outcomes for the Program (SD 15-6, Appendix D Section I) Section 2: Curricular Maps a. Map of Programmatic SLO’s to Baccalaureate Framework (Appendix D, Section II) b. Map of Programmatic SLO’s to Identified “core courses” in the curriculum (Appendix D, Section III)

Annual Assessment Report Outline Section 3: Assessment Plan a. Description of Departments Assessment Model b. Measures used c. Rubrics or evaluation metric descriptions d. Dissemination plan

Annual Assessment Report Outline Section 4: Assessment Results a. Current year assessment findings b. Proposed changes to address findings c. Prior year findings and assessment of changes d. Assessment findings for curricular changes made Section 5: Conclusions, next steps, and reporting plan

Department/Program Level Assessment Activities and Results Assessing, Making Interventions, and Reassessing to Improve Programmatic Capacity for Improving Student Learning

SLO’s, Maps, and The Assessment Plan Sections 1,2, and 3 should be fairly static (require minimal changes) over time Section 4 reporting is informed by the PLAIR Model emphasizing a process of “assess-intervene-reassess” ( Fulcher, Good, Coleman, and Smith, 2015). Sections 5 and 6 draw conclusions to demonstrate the improving quality of the program, suggest how the faculty plan on continuing to improve program support of student learning, and communicate to internal and external constituents what students are learning.

Part 2: College Level Review of Departmental Assessment Rubrics for Evaluating Academic Program Review from Appendix D of Proposed Revision to SD 98-22

Appendix D-I: Clearly Stated Student Learning Outcomes Exemplary Acceptable Developing Clarity and specificity All SLOs are stated with clarity and specificity including precise verbs and rich descriptions of the knowledge, skills and value domains expected of students upon completing the program SLOs generally contain precise verbs, rich description of the knowledge, skills and value domains expected of students. SLOs are inconsistently defined for the program, descriptions of the knowledge, skill and value domains are present but lack consistent precision. Student-Centered All SLOs are stated in studentcentered terms (i.e. what a student should know, think, or do). Most SLOs are stated in student-centered terms. Some SLOs are stated in student-centered terms. Expectation Level SLOs exceed basic expectations established by the University and other necessary approving organizations required of the submitting unit. SLOs meet the basic expectations established by the University and other necessary approving organizations required of the submitting unit. SLOs meet only a portion of the expectations established by the University or other necessary approving organizations required of the submitting unit. Not Present

Appendix D-II: Alignment with Baccalaureate Framework Exemplary IPFW Baccalaureate Framework Alignment Specific, clearly defined, studentcentered Program-Level SLOs are aligned to all foundation areas of the IPFW Baccalaureate Framework. Acceptable Generally defined studentcentered Program-Level SLOs are aligned to all foundation areas of the IPFW Baccalaureate Framework. Developing Program-Level SLOs are aligned to some foundation areas of the IPFW Baccalaureate Framework. Not Present

Appendix D-III: Programmatic Curriculum Map Exemplary Acceptable Developing Common classes or learning activities are identified for all students completing the program but most SLOs are not clearly mapped to classes or activities. Curricular Map identifies expected levels of learning for some SLOs at specific points in the curriculum. Content Alignment All SLOs are mapped to common classes or learning activities expected of all students completing the program. Most SLOs are mapped to common classes or learning activities expected of all students completing the program. Student Learning Development of SLOs (Learning Benchmarks) Curricular Map clearly identifies the progression of student learning relative to all SLOs at specific points in the curriculum. Classes and/or activities engage students in the work outlined in the SLOs. Curricular Map identifies levels of expected learning relative to most SLOs at specific points in the curriculum. Classes and/or activities engage students in the work outlined by most of the SLOs. Student Engagement Classes and/or activities do not consistently engage students in the work outlined by most of the SLOs. Not Present

Appendix D-IV: Assessment Plan Relationship between assessments and SLOs Types of Measures Established Results Data Collection and Design Integrity Evidence of Reliability of Measures Exemplary Acceptable Developing Detail is provided regarding SLOto-measure match. Specific items included on the assessment are linked to SLOs. The match is affirmed by faculty subject experts. All SLOs are assessed using at least two measures including at least one direct measure. Statements of desired results (data targets) provide useful comparisons and detailed timelines for completion. The data collection process is sound, clearly explained, and appropriately specific to be actionable. Description of how SLOs relate to assessment is general but sufficient to show alignment. Description of how SLOs relate to assessment is incomplete or too general to provide sufficient information for use in determining progress toward SLO. Most SLOs are assessed using at least one direct measure. Most SLOs are either assessed using only indirect measures or are not assessed. Statements of desired results are missing or unrealistic for completion. Methods used to ensure reliability of findings are clearly explained and consistently support drawing meaningful conclusions. Methods used to ensure reliability of findings are stated and generally support drawing meaningful conclusions. Statements of desired results provide a basic data target and a general timeline for completion. Enough information is provided to understand the data collection process with limited methodological concerns. Limited information is provided about the data collection process or includes sufficient flaws to nullify any conclusions drawn from the data. Methods to ensure reliability of findings are insufficient for drawing meaningful conclusions. Not Present

Appendix D-V: Reporting Results Presentation of Results Historical Results Interpretation of Results Exemplary Acceptable Developing Results are clearly present and directly related to SLOs. Results consistently demonstrate student achievement relative to stated SLOs. Results are derived from generally accepted practices for student learning outcomes assessment. Past iterations of results are provided for most assessments to provide context for current results. Interpretations of results are reasonable given the SLOs, desired levels of student learning and methodology employed. Multiple faculty interpreted the results including an interpretation of how classes/activities might have affected the results. Results are present and related to SLOs. Results generally demonstrate student achievement relative to stated SLOs. Results are derived from generally accepted practices for student learning outcomes assessment. Results are provided but do not clearly relate to SLOs. Results inconsistently demonstrate student achievement relative to stated SLOs. Use of generally accepted practices for student learning outcomes assessment is unclear. Limited or no iterations of prior results are provided. Past iterations of results are provided for the majority of assessments to provide context for current results. Interpretations of results are reasonable given the SLOs, desired levels of student learning and methodology employed. Multiple faculty interpreted the results. Interpretation of results does not adequately refer to stated SLOs or identify expectations for student learning relative to SLOs. The interpretation does not include multiple faculty. Not Present

Appendix D-VI: Report Dissemination and Collaboration Documents and results are shared with faculty Documents and results are shared with other stakeholders Exemplary Acceptable Developing Information is routinely provided to all faculty with multiple opportunities for collaboration to build meaningful future plans. Information is routinely provided to stakeholders (beyond faculty) with multiple opportunities for collaboration to build meaningful future plans. Information is provided to all faculty through an effective mode and with sufficient detail to be meaningful. Information is shared with stakeholders (beyond faculty) through an effective mode and with sufficient detail to be meaningful. Information is not distributed to all faculty or provides insufficient detail to be meaningful. Information is not distributed to stakeholders (beyond faculty) or provides insufficient detail to be meaningful. Not Present

College Level Report Template for Assessment Council Overview of Report Section 1: Summary of findings detailing scores of all academic departments of the college. Section 2: Summary of recommendations made to academic departments based on their assessment findings. Section 3: Summary of results of changes made or actions taken as a result of prior year findings including results of student learning and a summary of impact (positive or negative). Section 4: Conclusions providing an overall evaluation of assessment n the College and a description of changes in process planned to improve the quality of student learning.

Example of College Level Reporting Part 1a: Summary of Findings Within a Department Exemplary Department 1 Summary Acceptable Developing Summary Clarity and specificity All SLOs are stated with clarity and specificity including precise verbs and rich descriptions of the knowledge, skills and value domains expected of students upon completing the program SLOs generally contain precise verbs, rich description of the knowledge, skills and value domains expected of students. SLOs are inconsistently defined for the program, descriptions of the knowledge, skill and value domains are present but lack consistent precision. Student-Centered All SLOs are stated in studentcentered terms (i.e. what a student should know, think, or do). Most SLOs are stated in student-centered terms. Some SLOs are stated in student-centered terms. Expectation Level SLOs exceed basic expectations SLOs meet the basic expectations SLOs meet only a portion of the established by the University and established by the University and expectations established by the Acceptable other necessary approving other necessary approving University or other necessary organizations required of the organizations required of the approving organizations required submitting unit. submitting unit. of the submitting unit. Overall the quality of Student Learning Outcomes is very good. However, not all SLO’s were stated in student centered terms. For example Recommendation Exemplary Acceptable In addition, the expected level of learning for Programmatic SLO 1 and SLO 3 were stated in the Bloom lexicon at the “understand” level. Given the expected progression of student learning, we would expect students to advance to a level where they can apply and analyze. For example

Example of College Level Reporting Part 1b: Summary of Findings Across Departments Clarity and specificity Exemplary Acceptable Developing All SLOs are stated with clarity and specificity including precise verbs and rich descriptions of the knowledge, skills and value domains expected of students upon completing the program SLOs generally contain precise verbs, rich description of the knowledge, skills and value domains expected of students. SLOs are inconsistently defined for the program, descriptions of the knowledge, skill and value domains are present but lack consistent precision. College Summary College Assessment Committee provides a summary of overall College Performance for each Rubric area.

Part 3: Supporting Programmatic Assessment The Office of Assessment, IPFW Assessment Academy, and Assessment Council as the Departmental Support Team

Overview of Institutional Assessment Team Office of Assessment – Coordinates and provides support for assessment activities The Assessment Council – Shared governance focused group of faculty charged with recommending policy and providing oversight of assessment process. The IPFW Assessment Academy Leadership Team – Faculty group charged with creating and supporting learning communities focused on developing and improving assessment at IPFW.

Support Available to Academic Departments 2016-17 Workshop Series – Delivered in general sessions and available as Departmentally Focused Series IPFW Assessment Academy – Cohort based community to support departments through a full cycle of assessment Assessment Director – Provides ongoing and on demand support, resources, and leadership for programmatic and course level assessment