Building System Capacity to Implement Evidence Based Practices Project

Building System Capacity to Implement Evidence Based Practices Project Directors’ Meeting - July, 2016 - Christina Kasprzak Kathy Hebbeler Donna Spiker Megan Vinh

ECTA System Framework: Building High-Quality Early Intervention and Preschool Special Education Programs Christina Kasprzak, ECTA Center Project Directors’ Meeting - July, 2016 -

Why do we need a System Framework? Provides a common definition of a high quality system including components and elements Aligns with and supports implementation of EBPs Provides states with a framework for evaluating and improving their system Through self-assessment it provides focus for program planning

System Framework: Purpose and Audience Purpose: to guide states in evaluating their current Part C/619 system, identifying areas for improvement, and providing direction on how to develop a more effective, efficient Part C and Section 619 system that requires, supports, and encourages implementation of effective practices. Audience: the key audience is state Part C and state Section 619 coordinators and staff, with acknowledgement that other key staff and leadership in a state will need to be involved.

System Framework: Process and Partners Iterative validation process: the framework was developed through an iterative process among national and state experts in the field. Partner states: the framework was developed with 6 states (DE, ID, MN, NJ, PA, WV), so that it reflects (and is applicable to) the diversity of state systems (e.g. Lead Agency, eligibility criteria). Technical Work Group (TWG): a TWG with experts in the field advised the Center by providing early input on the elements, and later reviewed and gave input on drafts, as well as contributed resources to support states on various elements.

Partner States Delaware Susan Campbell, CarolAnn Schumann, Part C, Birth to Three Early Intervention System Verna Thompson, Section 619 Preschool Coordinator Idaho Christy Cronheim, Part C Coordinator Shannon Dunstan, Early Childhood Coordinator (Section 619 Preschool) Pennsylvania Jim Coyle, Kim Koteles & Mary Anketell, Office of Child Development and Early Learning. Bureau of Early Intervention Services Minnesota Kara Hall Tempel, Part C Coordinator, Lisa Backer, Early Childhood Special Education Supervisor (Section 619 Preschool) New Jersey Terry Harrison, Part C Coordinator Barbara Tkach, Section 619 Preschool Coordinator West Virginia Pam Roush, Part C Coordinator Ginger Huffman, Section 619 Preschool Coordinator

7 Technical Work Group Members Mary Beth Bruder, Director, Early Childhood Personnel Center (ECPC), University of CT Lori Connors-Tadros, Project Director, Center on Enhancing Early Learning Outcomes (CEELO), National Institute for Early Education Research, Rutgers University Barbara Gebbard and Jodi Whiteman, Center for Training Services and Special Projects, ZERO TO THREE Maureen Greer, Executive Director, Infant Toddler Coordinator Association (ITCA) Vivian James, 619 Preschool Coordinator, Office of Early Learning, NC Department of Public Instruction Grace Kelley, Program Specialist, South East Regional Resource Center (SERRC) Jana Martella, Co-Director, Center on Enhancing Early Learning Outcomes (CEELO), Education Development Center, Inc. Robin McWilliam, Director of the Center for Child and Family Research, Siskin Children’s Institute Cindy Oser, Director of Infant-Early Childhood Mental Health Strategy, ZERO TO THREE Ann Reale, Principal, ICF International and ELC TA Collaboration Lead, ELC TA Program Rachel Schumacher, Early Childhood Policy Consultant, R. Schumacher Consulting

Collaboration Across Centers/Orgs DaSy – the Center for IDEA Early Childhood Data Systems – Funded to develop an EC Data System Framework ECPC – Early Childhood Personnel Center – Funded to support C/619 Personnel Systems ECSWG—Early Childhood Systems Work Group COMPREHENSIVE EARLY CHILDHOOD SYSTEM-BUILDING: A Tool To Inform Discussions On Collaborative, Cross-Sector Planning

9 What does a state need to put into place to support implementation of effective practices? Governance Result Quality Standards Finance Implementation of Effective Practices Building High-Quality Systems Accountability & Quality Improvement Personnel / Workforce Data System Good outcomes for children with disabilities and their families ectacenter.org/sysframe

Components/Subcomponents of the Framework Governance Governance Vision, Mission and/or Purpose Legal Foundations Administrative Structures Leadership and Performance Management Finance Finance Finance Planning Process/ Forecasting Fiscal Data Procurement Resource Allocation, Use of Funds and Disbursement Monitoring and Accountability of Funds and Resources

Components/Sub-components cont’d Personnel/Workforce Personnel/Workforce Leadership, Coordination and Sustainability State Personnel Standards Pre-service Personnel Development In-service Personnel Development Recruitment and Retention Evaluation Data Data Systems Systems Purpose and Vision Data Governance and Management Stakeholder Engagement System Design/Development Data Use Sustainability

Components/Subcomponents cont’d Accountability Accountability & & Quality Quality Improvement Improvement Planning for Accountability and Improvement Collecting and Analyzing Performance Data Using Results for Continuous Improvement Quality Quality Standards Standards Child Level Standards Program Standards

Cross cutting themes Engaging stakeholders, including families Establishing/revising policies Promoting collaboration Using data for improvement Communicating effectively Family Leadership & Support Coordinating/Integrating across early childhood

The Center for IDEA Early Childhood Data Systems DaSy Data System Framework for Developing and Enhancing a High-Quality State Data System Donna Spiker Project Directors’ Meeting July 2016

What is DaSy?

What is DaSy? A 5-year Center funded by OSEP in 2013 to assist states with improving IDEA Part C and Part B preschool data by: Building better data systems Coordinating data systems across early childhood programs Connecting to longitudinal data systems Building the capacity of states to use data

The DaSy Data System Framework: What is it? It is the Data System component of the larger ECTA System Framework for Part C & Section 619. For more information about the ECTA System Framework, visit: http://ectacenter.org/sysframe/

Purpose of the DaSy Data System Framework and Self-Assessment Assist Part C and Section 619 state staff in developing and enhancing high-quality state data systems and in improving the quality of their IDEA data.

Broad Definition of “Data System” Data System refers to the hardware, software, and other applications that enable Part C and Section 619 programs to collect data about children, families, workforce, and/or program characteristics (e.g., program quality), as well as the analysis, reporting, and data use practices associated with those data.

DaSy Framework Addresses Three Major Uses of Data Accountability – Federal and state reporting Program Improvement – Identifying strengths and weaknesses, improving results Program Operations – Day-to-day management and operation of program activities

21

DaSy Framework Structure Quality Indicator 1.1 Subcomponen t1 Quality Indicator 1.2 6 Subcomponents 31 Quality Indicators Element of Quality 1.1a Element of Quality 1.1b Element of Quality 1.2a 170 Elements of Quality 22

The DaSy Framework

Data Use 24

Quality Indicators Planning for data use – DU1. Plan for data analysis, product development, and dissemination to address the needs of the users Analyzing and disseminating for data use – DU2. Conduct data analysis activities and implement procedures to ensure the integrity of the data – DU3. Prepare data products to promote understanding of the data and inform decision-making – DU4. Disseminate data products to users to meet their needs Using and promoting capacity for data use – DU5. State and local staff use data to inform decisions – DU6. Support the use of data at state and local levels http://dasycenter.org/resources/dasy-framework/data-use

Elements of Quality Quality Indicator DU1: Part C/619 state staff plan for data analysis, product development, and dissemination to address the needs of the state agency and other users. Elements of Quality 1. DU1a. Part C/619 state staff develop recommendations for effective data use. 2. DU1b. Part C/619 state staff identify potential data users (state and local) and periodically gather information about their specific data needs. 3. DU1c. When planning documents, products, resources, and timelines, Part C/619 state staff consider accountability and program improvement questions that drive data analysis and use.

Elements of Quality Quality Indicator DU1: Part C/619 state staff plan for data analysis, product development, and dissemination to address the needs of the state agency and other users. Elements of Quality Continued. 4. DU1d. Part C/619 state staff have a process to prioritize data requests, both for data that are readily available and queried regularly and data that require additional staff time to query, and respond to them (as appropriate) in a timely fashion. 5. DU1e. Part C/619 state staff plan for dissemination that focuses on products, methods, and timelines tailored to specific stakeholder groups. 6. DU1f. Part C/619 state staff review and revise plans for data analysis, product development, and dissemination as necessary.

The Center for IDEA Early Childhood Data Systems Framework Self-Assessment Kathy Hebbeler Project Directors’ Meeting July 2016

The Center for IDEA Early Childhood Data Systems Corresponding Self-Assessment Provides states a current status ‘snap shot’ to help them prioritize improvement efforts Quantitative self-assessment score provides states ability to measure progress over multiple points in time. Encourages participants to engage in rich conversations about the current status of the state system. 29

The Center for IDEA Early Childhood Data Systems Using the Framework Self-Assessment Excel-based tool (downloadable) Structure allows states to record current status and set priorities for improvement Complete one subcomponent at a time Different stakeholders can complete different subcomponents.

The Center for IDEA Early Childhood Data Systems Stakeholders rate the Elements of Quality 4 point scale Rated by stakeholders – Read each element Subcomponent 1 – Discuss and write down evidence – Based on the evidence, determine a rating Quality Indicator 1.1 Quality Indicator 1.2 Element of Quality 1.1a Element of Quality 1.1b Element of Quality 1.2a 31

The Center for IDEA Early Childhood Data Systems Element Rating Scale Element of Quality Rating Scale No - element not in place and not planning to work 1 No on it at this time No - element not in place but planning to work on it 2 No, planning or getting started 3 Yes, partially Yes - element partially implemented 4 Yes, fully Yes - element fully implemented 32

The Center for IDEA Early Childhood Data Systems Quality Indicator Rating Scale 1-7 point scale Stakeholders must rate all Elements of Quality in the Quality Indicator Rating is not selected by the stakeholder group but rather calculated based on the element ratings 33

The Center for IDEA Early Childhood Data Systems Quality Indicator Rating Scale Quality Indicator Rating Scale 1 None of the elements are yet planned or in place 2 Most of the elements are not yet planned or in place 3 Some elements are in place, a few may be fully implemented 4 At least half of the elements are in place; a few may be fully implemented 5 At least half of the elements are in place; some are fully implemented 6 At least half of the elements are fully implemented, the rest are partially implemented 7 All elements are fully implemented 34

The Center for IDEA Early Childhood Data Systems Suggested Process Select sub-component Identify the stakeholders for the selfassessment process Complete the self-assessment Review results to determine priorities Develop improvement plan Implement improvement plan Lather, rinses, repeat (in future) to identify progress and new areas for improvement

The Center for IDEA Early Childhood Data Systems http://dasycenter.sri.com/downloads/Framework/Framework Self-Assessment G structions 060915 Acc.pdf

The Center for IDEA Early Childhood Data Systems http://dasycenter.org/self-assessment-for-ecta-and-dasy-frameworks/

The Center for IDEA Early Childhood Data Systems http://dasycenter.org/self-assessment-for-ecta-and-dasy-frameworks/

The Center for IDEA Early Childhood Data Systems http://dasycenter.org/self-assessment-comparison-tool/

Questions 40

The Center for IDEA Early Childhood Data Systems How are States Using the System Framework and SelfAssessment? Megan Vinh, PhD Project Directors’ Meeting July, 2016

The Center for IDEA Early Childhood Data Systems Why is Building a High Quality System Important Now? Building and sustaining a high-quality system is complex and ongoing process Challenges exist that compromise a state’s ability to develop a high-quality system Shift from a compliance driven to results driven approach – State Systemic Improvement Plans (SSIPs)

The Center for IDEA Early Childhood Data Systems Barriers to Developing a High Quality System Staffing shortages Fiscal uncertainty Unclear or lack of alignment between standards Conflicting guidance and priorities Lack of accountability structures Lack of clarity around components of a high quality system 43

The Center for IDEA Early Childhood Data Systems STATE EXAMPLES

The Center for IDEA Early Childhood Data Systems

The Center for IDEA Early Childhood Data Systems Delaware This work has had powerful effects for Delaware: The results of the state cross sector system framework workgroup brought a new focus on the need for well trained professional staff in all of the inclusive environments that enroll children with disabilities. The work truly highlighted the importance of each of the system framework components for all early childhood - not just ECSE.

The Center for IDEA Early Childhood Data Systems Idaho Idaho completed the Self-Assessment for both their ECSE and K-12 special education state systems. The state is using the information gathered from the SelfAssessment to: develop an action plan for both the preschool and K-12 special education programs; and work with districts to use the Self-Assessment to self-audit and monitor progress toward implementing each of the system components at the local level.

The Center for IDEA Early Childhood Data Systems Colorado The Framework helped CO to: Integrate data between EI and ECSE; Improve data quality; Improve use of data at the state and local level; and Link transition data between EI and ECSE.

The Center for IDEA Early Childhood Data Systems Virginia Used the Framework to identify the highest priority needs for both improving fiscal management and to plan for developing a new online data system, using: Data Use, Data Governance & Management Finance Used the Framework to support the SSIP work

The Center for IDEA Early Childhood Data Systems New Jersey Used the Framework to guide an analysis of their infrastructure for strengths and areas of improvement as part of their SSIP. The Framework helped the state to: identify areas of the system components for which to develop improvement strategies, build capacity to scale-up, implement, and sustain evidence-based practices, and support interdepartmental work in the state across early childhood programs in collaborative efforts and cross-sector planning.

The Center for IDEA Early Childhood Data Systems Questions/Comments 57

The Center for IDEA Early Childhood Data Systems Discussion Questions What is the role of state systems in supporting practice at the local level? How can TA centers and others use the system framework to support states in improving their state system? 58