WELCOME TO WEBINAR #8: TEST IRREGULARITIES This webinar will be

WELCOME TO WEBINAR #8: TEST IRREGULARITIES This webinar will be recorded and posted on the ADE Assessments webpage. Please enter your First and Last Name in the Chat for tracking purposes for the live event. We will also be capturing the chat questions. If there are questions that were frequently asked or need further clarification, ADE will compile and create an FAQ which will then be posted on the Assessments webpage under Friday Focus Webinars.

ARIZONA DEPARTMENT OF EDUCATION ASSESSMENTS FRIDAY FOCUS WEBINAR #8: TEST IRREGULARITIES LISA OLIVER DIRECTOR OF ACHIEVEMENT ASSESSMENT JANUARY 19, 2024

Overview Standardized Assessments Test Security FERPA Security Agreements Irregularities and Implications Breaches of Security Reporting and Codes Administration Tips

STANDARDIZED ASSESSMENTS Examples of Standardized Assessments: NAEP, AASA, MSAA Standardized training is required for all statewide assessments. This ensures that the test will be administered the same for all students. Assessment Administrators receive online training and the supporting documents to ensure fidelity of implementation and the validity of the assessment results as well as to help prevent, detect, and respond to irregularities in academic testing and maintain testing integrity practices for the Achievement assessments. If the test is not administered in the same ways this could result in a test irregularity or even an invalidation of a student test or tests.

VALID, FAIR, AND RELIABLE ASSESSMENT Meet comprehensive standards of validity, reliability, and fairness Reliable assessments produce stable and consistent results Conduct the assessments in a fair and ethical manner Closely adhere to security policies and test security agreement and practices Apply principles of test security to assessment material storage and handling, test administration, and electronic tools Contact the appropriate person regarding test security violations or test irregularities

FERPA! Achievement testing materials may contain Personally Identifiable Information (PII) The Achievement tests security policy also applies to the digital systems and tools used to administer and score the assessments. It is a direct test security violation to share or exchange account information, such as usernames or passwords, with anyone. This applies to all staff for: ADE Connect, PearsonAccess Next, and the Arizona Learning Management System. Secure Testing Tickets and Test Booklets Personally Identifiable Information-Only send ADE SSID

ACHIEVEMENT TESTS STAFF SECURITY AGREEMENT This Achievement Tests Staff Security Agreement must be signed by all employees of the district, charter, and schools who will have contact with Achievement test materials, administer the Achievement tests, monitor students during test administration, assist with testing, and/or enter the testing environment.

TRAINING REQUIREMENTS Achievement Assessments Tra ining Requirements o o Available in AzLMS Available on Achievement DTC webpage

HOW TO AVOID TEST IRREGULARITIES Ensure: Participation in the Test Security & Ethics module is to be completed by all staff this includes DTC, STC, TA, Tech Coordinators, proctors, and non-testing staff with access to materials or environment (including for ACT and ACT Aspire as well as AASA and AzSCI) Participation in The Test Administration Responsibilities module is to be completed by all staff this includes DTC, STC, and TA for AASA and AzSCI. Have a plan to monitor during test administration

TEST ADMINISTRATION EXPECTATIONS

STUDENT PERFORMANCE? PRACTICE! PRACTICE! PRACTICE!

TEST SECURITY

TEST SECURITY This is why we track Test Irregularities Costly Implications Student Level (Scored vs. Invalidated) Item Level (Development/Released Items) Test Level (Accountability) Technology is “locked down” Disable ALL Applications (external programs) running when TestNav launches

STUDENT LEVEL IMPLICATIONS It is unethical and shall be viewed as a violation of test security for any person to: TA Logs into TestNav as a student Share their username/password for PearsonAccess Next Allow students access to test content prior to testing, including the ELA Oral Reading Fluency (ORF) passages Provide any reference sheets to students during the Math test administration or graphic organizers during the Writing test administration Allow students to share information, tools, and/or resources during test administration Influence students’ responses by making any kind of gestures (for example, pointing to questions, Hold up fingers to signify question numbers or answer options while students are taking the test Instruct students to go back and reread/redo responses after they have finished their test since this instruction may only be given before the students take the test INVALIDATED TEST Change students’ answer choices

ITEM LEVEL IMPLICATIONS It is unethical and shall be viewed as a violation of test security for any person to: Capture images of any part of the test via any electronic device Duplicate, in any way, any part of the test Examine, read, review, disclose, or allow to be disclosed, the content of the test before, during, or after test administration Read any parts of the test to students, except as indicated in the Test Administration Directions, or as part of an approved accommodation Translate, reword, or explain any test content Read or review students’ responses, scratch paper, and other secure materials Participate in, direct, aid, counsel, assist in, encourage, or fail to report any violations of these test administration security procedures Instruct students to keep their test unit open Use remote learning software to view the screen of a student or classroom during testing

TEST LEVEL IMPLICATIONS Test level implications will impact Accountability Scores and School Data

TEST IRREGULARITIES

PLAN IN PLACE WHEN AND WHERE Administered at Arizona schools. Students must be tested in a physical building designated as a testing site by the school. Administering AASA tests on dates other than those shown without the written permission of the Assessment Unit of the. TCM, pg. 8

BREACHES OF TEST SECURITY PLEASE LOCATE YOUR TEAM’S REACTION BUTTON FOR ACTIVE PARTICIPATION

BREACHES OF TEST SECURITY Use of Unacceptable Resources If students are observed in possession of and/or use of unacceptable resources, including but not limited to unacceptable reference materials, cell phones, smart watches, or other electronic devices, the unacceptable resource must be removed. Students are provided all other tools and resources needed for AASA testing within the TestNav application. Test Administrator: Use of a cell phone and /or Walkie Talkie is an LEA decision. Please use with caution and silence the device. Communication should not interfere or impede student performance.

BREACHES OF TEST SECURITY Use of Accommodations Accommodations are provisions made in how a student accesses the test and/or demonstrates learning that do not alter the validity of the test, score interpretation, reliability, or security of the test. Modifications that alter the test construct being measured are prohibited, including changing the number of answer options, providing Text-to-Speech or a human reader during the ELA Reading test units, or providing a calculator during Math test units where a calculator is prohibited. ORInappropriate test administration for students not eligible for testing (eligible for Alternate Assessment)

BREACHES OF TEST SECURITY Testing Environment All visual aids displayed in the testing room that could assist students while testing must be removed or covered completely prior to any test session. Any motivational signs, apparel, or activities are not permitted in the testing environment. Arrange student seating so that students cannot easily see each other’s test materials. Parents, relatives, student teachers, volunteers and school helpers are not permitted to be in the testing room. District or school staff who have relatives being administered the test are not permitted to be in the testing room while their relative is testing. Test Administrators, Proctors, and any adult who will have access to the assessment must be employees of the school/district/charter.

BREACHES OF TEST SECURITY Entering and Exiting the Testing Environment Students who are working productively and require additional time must be allowed to complete the test. Once student has submitted their test, students may remain in the environment; however, they must remain silent, and they may not access content material, computer, or have access to their electronic devices. Students may not leave the testing environment to attend lunch. Group or class stretch breaks are not allowed. Students cannot be stopped from testing by the Test Administrator for a stretch break. Students who disrupt testing, refuse to participate, receive help from others, or otherwise engage in behavior not consistent with acceptable classroom behavior should be removed from the testing room as soon as possible to allow other students to continue to test undisturbed.

BREACHES OF TEST SECURITY Food in Testing Environment Students may not leave the testing environment to attend lunch. Group or class stretch breaks are not allowed. Students cannot be stopped from testing by the Test Administrator for a snack break. ACT prohibits the consumption of snacks and beverages within the testing environment. For AASA and AzSCI, please limit consumption of snacks and beverages within the testing environment. (LEA Choice)

KINDS OF TEST IRREGULARITIES All need to be documented but some will result in an invalidation of a test or tests Will most likely not be invalidated: Started test but technology prohibited reentering test to complete during same day Technology disruptions (Enter a Test Irregularity) Disruptive students (Do Not Enter a Test Irregularity) We appreciate the seriousness our District Test Coordinators take in reporting and investigating Will be invalidated: Students being administered the test without a qualified test administrator Cheating by adults or students Teachers instructing students to practice with operational test questions We get reports from DTCs and parents (whose children went home and shared what happened) Providing any reference sheets to students during testing Students using or in possession of cell phones Providing a non-allowable accommodation Reading the Reading Test Taking photo or tik-toks during test administration We investigate all reports to ADE- including anonymous reports Motivational posters up in classroom Students accessing content or web content during test session

GLITCH IN THE MATRIX TECHNOLOGY CONCERNS

COMMON ISSUES THAT ARISE DURING THE TEST AND HOW TO RESOLVE THE ISSUE Student unable to log in – Student must type their username and password exactly as listed on the student’s secure testing ticket. Verify that the student’s test session is in Ready status and student is unlocked in PearsonAccessnext. Student in “Exited” status – Student has signed out of TestNav. If appropriate, the Test Administrator should “Resume” the student’s test session in PearsonAccessnext. Student in “Completed” status – Student has completed and submitted the test unit. The test unit will not be eligible to be reopened. 3005: TestNav has detected that another application attempted to become the active window, which may compromise the test security. Should this test be resumed? PLEASE ASK QUESTIONS! Student in "Marked Complete" status – Student's test was launched but not submitted properly. At the end of each day, any test that was not submitted properly is placed in a Marked Complete status.

TECHNICAL SUPPORT Immediate Level 2 Technical Support Available by Phone or Email Pearson Customer Support 1-888-705-9421 opt # (ask to speak with Level 2)

REPORTING IRREGULARITIES PEARSONACCESS NEXT

REPORTING TEST IRREGULARITIES Submit in Test Irregularity into PAN For AASA , AZSCI , ACT Aspire. ADE will follow up with next steps, if required. Submit in PAN for ACT for online testing and paper irregularity report form for paper testing Any breach of test security, loss of materials, failure to account for materials, exposed test questions, or any other deviation from acceptable security procedures shall be reported immediately by submitting a test irregularity report in PearsonAccess Next. The discipline of staff members who violate test security is the responsibility of the district or charter. Disciplinary action may include, but is not limited to, a letter of reprimand, suspension with pay, suspension without pay, dismissal, or report of violation to the Investigations Unit of the Arizona State Board of Education. See Test Coordinator Manual

TEST IRREGULARITY REPORT 2024

TIPS:

AASA AND AZSCI SAMPLE TEST To access the AASA and AzSCI Sample Tests, the educator will go to the TestNav Arizona Sign In screen (https://home.testnav.com/). A username and password are not required. The Test Audio link and the Mic Check & Sample Tests links are at the bottom of the screen. Click on Sample Tests The AASA and AzSCI Sample Test helps students, teachers, and parents become familiar with the content, question types, and the tools in the AASA and AzSCI assessments. The sample tests are available in the computerbased format, including American Sign Language (ASL), for ELA Writing, Reading, Oral Reading Fluency (ORF), and Math test units. Students will access the test as a guest, so no personal information needs to be provided. Students should take the Sample Test prior to taking the Spring 2024 test.

SUBMIT FINAL ANSWERS Once the student clicks the green “Submit Final Answers” button, the screen “Test Submit Warning” will display to confirm that the student is about to leave this unit. Once the student selects the “Yes, Submit Final Answers” button on this warning screen, the test opportunity will end, and the student will not be able to continue to test. This unit of the test cannot be reopened once the student clicks “Yes, Submit Final Answers.”

QUESTIONS?

ASSESSMENT COMMITTEE We are inviting content area teachers, teachers of students with disabilities, teachers of EL students, and instructional coaches/administrators to provide their perspective on the items and standards set on Arizona's statewide assessments. If you are interested in serving on an Assessment Educator Committee, please complete our Committee Application found at: https://www.azed.gov/assessment/assess ment-educator-committee-application

ASSESSMENT WEBINAR SERIES - POSTED The flyer is posted on the Assessment Main page: https://www.azed.gov/sites/d efault/files/2023/07/2023-20 24%20Friday%20Focus%20W ebinar%20Schedule.pdf Reminder: participants must register in ADE’s Professional Learning and Development (APLD) system Log in information to join the session will be sent to registered participants prior to the event. Login information will not be available in APLD.

SAVE THE DATE Arizona Assessments Conference – September 17-19, 2024 https://www.azed.gov/assessment/ conference

THANK YOU! For questions, please contact us at: [email protected]