Connecticut’s Growth Model for the Smarter Balanced

Connecticut’s Growth Model for the Smarter Balanced Summative Assessments: English Language Arts (ELA) and Mathematics October 2016 1 CONNECTICUT STATE DEPARTMENT OF EDUCATION

Agenda What is growth? How is it different from achievement? What is Connecticut’s approach to measuring growth? What factors are considered when establishing ambitious yet achievable targets? How and when will growth be incorporated into the Next Generation Accountability System for districts and schools? 2 CONNECTICUT STATE DEPARTMENT OF EDUCATION

What is growth? How is it different from achievement? Achievement or Proficiency: A one-time snapshot measurement of a student’s academic performance Growth: Change in achievement score for the same student between two or more points in time. 3 CONNECTICUT STATE DEPARTMENT OF EDUCATION

4 Three Ways to Understand Change in Performance Achievement Change “Rough Cohort” Change Matched Student Cohort Growth How does it work? Compares student achievement across years (e.g., achievement of grade 4 students in 2014-15 is compared to the achievement of grade 4 students in 2015-16) Compares the achievement of a group of students from one grade in year 1 to a group of students in the next higher grade in year 2 (e.g., grade 3 in 2014-15 to grade 4 in 2015-16) Compares the achievement of the same student from one grade in year 1 to the next higher grade in year 2 (e.g., student in grade 3 in 2014-15 to grade 4 in 2015-16) Who is compared? Different students across different years Mostly the same students though there can be some mismatches due to student mobility, entry, and exit The same students from one year to the next no mismatches What is measured? Proficiency rate (e.g., percent at or above level 3) and/or average scale scores Proficiency rate (e.g., percent at or above level 3) and/or average scale scores The amount of growth to standard achieved by each student and groups of students What does it offer? The starting point for understanding change A “rough estimate” of growth The gold standard for growth and for understanding curricular and instructional effectiveness What is it? CONNECTICUT STATE DEPARTMENT OF EDUCATION

What is Connecticut’s approach to measuring growth? Similar to approach used with CMT growth model Criterion referenced Uses Smarter Balanced vertical scale that spans grades/years Preserves achievement level concept for interpretability Provides ambitious yet achievable individual student targets Expects all students to grow, including those performing in Levels 3 and 4 Can be aggregated for group level results Reviewed by Connecticut Technical Advisory Committee 5 CONNECTICUT STATE DEPARTMENT OF EDUCATION

What factors are considered when establishing ambitious yet achievable targets? Empirical: – What is the actual growth achieved by students performing at different segments of the vertical scale? Measurement Error: – Does the growth expectation exceed the pooled average measurement error from both year 1 and year 2 assessments? Time: – Are students on a path to higher levels of achievement in the future? 6 CONNECTICUT STATE DEPARTMENT OF EDUCATION

ELA Achievement Level Ranges and Growth Targets Grade in Yr. 1 3 4 5 6 7 8 Level 1: Not Met Level Range Target Range Target Range Target Range Target Range Target Range 1 - LOW 2 - HIGH Level 2: Approaching 3 - LOW 4 - HIGH Level 3: Met 5 - LOW 6 - HIGH Level 4: Exceeded 7 - LOW 2114-2330 2331-2366 2367-2399 2400-2431 2432-2460 2461-2489 2490-2522 82 71 70 69 68 64 60 2131-2378 2379-2415 2416-2444 2445-2472 2473-2502 2503-2532 2533-2568 82 69 69 64 58 55 49 2201-2405 2406-2441 2442-2471 2472-2501 2502-2541 2542-2581 2582-2619 69 56 55 48 43 39 30 2210-2417 2418-2456 2457-2493 2494-2530 2531-2574 2575-2617 2618-2656 73 58 53 47 44 38 33 2258-2438 2439-2478 2479-2515 2516-2551 2552-2600 2601-2648 2649-2687 69 50 49 44 40 31 20 2288-2446 2447-2486 2487-2526 2527-2566 2567-2617 2618-2667 2668-2703 8 - HIGH 2523 45/maintain 2569 34/maintain 2620 16/maintain 2657 21/maintain 2688 12/maintain 2709 7 CONNECTICUT STATE DEPARTMENT OF EDUCATION

Math Achievement Level Ranges and Growth Targets Grade in Yr. 1 3 4 5 6 7 8 Level 1: Not Met Level Range Target Range Target Range Target Range Target Range Target Range 1 - LOW 2 - HIGH Level 2: Approaching 3 - LOW 4 - HIGH Level 3: Met 5 - LOW 6 - HIGH Level 4: Exceeded 7 - LOW 2189-2351 2352-2380 2381-2408 2409-2435 2436-2468 2469-2500 2501-2526 77 61 59 60 59 57 56 2204-2381 2382-2410 2411-2447 2448-2484 2485-2516 2517-2548 2549-2574 51 38 40 44 46 47 43 2219-2419 2420-2454 2455-2491 2492-2527 2528-2553 2554-2578 2579-2605 43 46 45 44 42 41 41 2235-2434 2435-2472 2473-2512 2513-2551 2552-2580 2581-2609 2610-2639 49 41 38 36 36 36 38 2250-2438 2439-2483 2484-2525 2526-2566 2567-2600 2601-2634 2635-2664 58 35 31 31 36 37 38 2265-2455 2457-2503 2504-2544 2545-2585 2586-2619 2620-2652 2653-2685 8 - HIGH 2527 47/maintain 2575 37/maintain 2606 44/maintain 2640 31/maintain 2665 35/maintain 2686 8 CONNECTICUT STATE DEPARTMENT OF EDUCATION

Smarter Balanced Vertical Scale Score Points Hypothetical Example Growth Rate 50% (2 out of 4 students met target) Average Percentage of Target Achieved (PTA) 85% 70 60 66 60 60 60 60 60 50 42 40 36 30 20 10 0 Student 1 Met Target No PTA 70% Student 2 Met Target Yes PTA 100% Target Amount Student 3 Met Target Yes PTA 110% Actual Growth Amount Student 4 Met Target No PTA 60% 9 CONNECTICUT STATE DEPARTMENT OF EDUCATION

Two Aggregate Outcome Metrics Growth Rate Percentage of Target Achieved Measure? Percentage of students meeting their respective growth target Average percentage of growth target achieved for all students Precision? Binary (yes/no), less precise Based on scale score, more precise Continuous? No. Students nearly meeting target will be deemed not meeting target Yes. Students get “credit” for any growth up to and beyond the target Interpretability? Simple to understand More nuanced Uses? Reporting and district/school accountability Reporting only 10 CONNECTICUT STATE DEPARTMENT OF EDUCATION

How and when will growth be incorporated into the Next Generation Accountability System? Growth (Indicator 2) will be added to the system starting with the 2015-16 results. As with achievement, Growth (Indicator 2) points are awarded for All Students and High Needs groups. The points for Achievement (Indicator 1) will be halved for any school with Growth results. Growth will carry slightly more weight in the model than Achievement. In light of the discontinuance of the ELA Performance Task in February 2016, the rescored 2014-15 ELA scores that were based on the Computer-Adaptive Test (CAT) only will be used as the ELA baseline for an apples-to-apples comparison. 11 CONNECTICUT STATE DEPARTMENT OF EDUCATION

What about other factors like poverty, language ability, or disability? The CSDE is not using a value-added approach to adjust targets or evaluate growth relative to some preconceived expectation based on student characteristics of what a student can achieve or how much he/she can grow. The CSDE is not setting different targets for different students. All students at a prior achievement range will have the same growth amount expectation. 12 CONNECTICUT STATE DEPARTMENT OF EDUCATION

Not All Growth Models are Value-Added The terms “growth model” and “value-added” are often used interchangeably. But Value-Added is only one of several types of models that measure student growth. It is also the only model designed to determine which aspect of schooling (e.g., school, teacher, education program) is responsible for a students' growth. (Center for Public Education). Value-added models are focused on the effects of teachers and leaders on student score gains. They address whether students grew more or less than expected. (O’Malley, McClarty, Magda, and Burling, 2011) 13 CONNECTICUT STATE DEPARTMENT OF EDUCATION

Summary Criterion referenced: does not depend on how others do Continuous: all growth counts; no golden bands Familiar: similar to approach used with CMT Transparent: easily replicable; no “black-box” adjustments Collaborative: transparency allows for conversation/reflection Fair: excludes “partial-year” students Achievable: based on actual growth of Connecticut students Ambitious: encourages growth above target 14 CONNECTICUT STATE DEPARTMENT OF EDUCATION