REDAC Human Factors Subcommittee NextGen Flight Deck Program Review

REDAC Human Factors Subcommittee NextGen Flight Deck Program Review NextGen Flight Deck Air/Ground Integration (111110) Regina Bolinger, BLI Manager September 8, 2016 Federal Aviation Administration

Air-Ground Integration Human Factors Description Provides human factors scientific and technical research products for Aviation Safety technical sponsors that enable their development of FAA guidelines, handbooks, orders, advisory circulars, technical standards orders and regulations. This important human factors regulatory and guidance material supports the Aircraft Certification and Flight Standards personnel who approve new flight deck displays, devices, and procedures that comprise NextGen capabilities, to ensure that pilots can conduct operations that leverage NextGen capabilities, and which together will yield increases in air transportation efficiency, capacity, and safety in the National Airspace System. FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 2

Air-Ground Integration Human Factors 7 NextGen TCRG research requirement areas: NextGen Flight Deck Human Factors Research Requirement Area Aviation Safety (AVS) Sponsor NextGen-Flight Deck Systems-flightcrew interfaces, installation, integration, and operations Aircraft Certification (AIR-134) Flight Standards – Flight Technologies and Procedures (AFS-410) NextGen: Procedures, Tasks, Skills and Training for NextGen Air Carrier Pilots and Dispatchers Flight Standards – Air Carrier Training and Voluntary Safety Programs (AFS-280) NextGen-Human Factors Guidelines for Advanced Instrument Procedure Design and Use Flight Standards – Flight Technologies and Procedures (AFS-470) NextGen-Human Error and Complex Systems Flight Standards – Air Carrier Training and Voluntary Safety Programs (AFS-280) NextGen-DataComm Human Factors R&D Flight Standards – Flight Technologies and Procedures (AFS-470) Minimum Detect and Avoid (DAA) Display and Flight Path Information Flight Standards - Unmanned Aircraft Systems (AFS-86) UAS Assessment for Enroute Contingency Operations Air Traffic Procedures (AJV-8) FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 33

NEXTGEN-FLIGHT DECK SYSTEMS-FLIGHTCREW INTERFACES, INSTALLATION, INTEGRATION, AND OPERATIONS FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 44

NextGen: Flight deck systems: flightcrew interfaces, installation, integration and operations R&D (A12C.HFNG.4) Research Requirement Description Existing policy does not address proliferation of new functions, equipment, and procedures required to implement NextGen (e.g., EVS/SVS, ADS-B, TCAS integrated with ADS-B, EFB, surface movement technologies, low visibility operations) Sponsors: Cathy Swider (AIR-134), Terry King (AFS-410) Critical Milestones Sponsor Outcome Develop and update regulatory and guidance material (14 CFR Part 121, Subparts N, O, Y (AQP); 14 CFR Parts 2X.1301, 25.1302, 2X.1329, AC 20-175). Research Accomplishments in FY16 Conduct simulation examining pilot performance with EFVS in lieu Completed report documenting considerations for display of information of required airport infrastructure for LVO/SMGCS (FY16) in the primary field of view Conduct industry survey/product review of CVS systems (FY16) Provide human factors recommendations for the design and evaluation of multi-touch controls (FY17) Examine visibility and visual aids for rollout using EFVS for approach and landing (FY17) Provide data identifying what EFVS can offer in lieu of airport infrastructure for LVO/SMGCS operations (FY17) FY16 Human Factors REDAC Fall Meeting Provided draft report documenting human factors considerations for the design and evaluation of alerts Completed final report identifying symbol stereotypes for LVO/SMGCS chart symbology Completed literature review and market survey of CVS systems. Federal Aviation Administration 5

NextGen Combined Vision Systems Research Requirement Description Sponsor Outcome The FAA needs scientifically valid data to support the development of regulatory and guidance materials that set the minimum requirements for approving CVSs and to make recommendations regarding pilot interface requirements. If the research suggests that the recommended minimum information requirements are not sufficient, the research should give an indication of what information needs to be added to the display to meet the required level of safety. In addition, this research should assist in clarifying alerting requirements for these displays and should allow recommendations to be formulated in regard to the user interface requirements for the CVS system. This research will be used to assess what CVS equipment is currently available, what CVS equipment is planned for development, how OEMs and operator’s plan to use it, what research has already been done for CVS, and what research gaps need to be addressed. It will be used to inform operational safety assessments and operational suitability decisions by AFS-400. It will be used to determine whether rulemaking is necessary and to develop operational criteria, evaluation tools and job aids, means of compliance documents, operational guidance for applicants, inspector guidance material for approving CVS operations, authorizations for conducting CVS operations, and training, recent flight experience, and proficiency requirements. Sponsor: POC: Terry King, AFS-410 Critical Milestones Conduct a literature search to identify human performance studies that were conducted using HMD. Produce a list of research citations and summarize the results of those studies. Identify research gaps. (02.07.00) Complete Conduct an industry/product review of CVS systems. Compile a list of existing systems, systems in development, systems that are either available or are being developed for civil aviation use. (02.08.00) Complete Conduct a literature search to identify human performance studies that were conducted using CVS. Produce a list of research citations and summarize the results of those studies. (02.09.00) Complete FY16 Human Factors REDAC Fall Meeting Research Accomplishments in FY16 Completed literature review and market survey of CVS systems. Completed literature review and gap analysis of HMD HF issues. Sponsor briefing in February to refine project specifications and develop research questions for out-years. Federal Aviation Administration 6

Preventing Clutter and Confusion on NextGen Flight Decks: Field of View and Alert Designs Reports Research Requirement Description Sponsor Outcome Aircraft Certification Service human factors specialists required a study of existing FAA regulatory and guidance material to identify, explain, and recommend guidance related to field of view and alert design standards in the aircraft flight deck. Researchers will identify, review, and summarize the relevant regulatory and guidance material as well as the human factors empirical research related to field of view and alerting. They will also identify the gaps that should be addressed in future updates to the FAA material. Empirical basis for regulatory guidance for flight deck layout based on field of view. The Alert Designs Report will be published as a supplemental reference for AC 25.1322-1 Sponsor: Cathy Swider , AIR-134 Critical Milestones Research Accomplishments in FY16 Annotated bibliography/spreadsheet or document of the references that were identified, 1-page summaries of each relevant research article, and the particular issue to which it applies Complete Final report – Empirical Basis for Regulatory Guidance for Flight Deck Layout Based on Field of View. Complete Evaluated and documented the alerting empirical literature and the FAA regulatory and guidance material. Draft Annotated bibliography in a Word document of the references that were identified, 1-page summaries of each relevant research article, and the particular issue to which it applies Complete Developed summaries of the literature to be included in the review (a subset of the 252 articles currently identified). Final report – Empirical Basis for Regulatory Guidance for the Design of Alerting Systems for the Flight Deck. 12/31/2016 Produced the first draft of the Alerts Report for sponsor review FY16 Human Factors REDAC Fall Meeting Field of View Considerations Report Completed Federal Aviation Administration 7

Preventing Clutter and Confusion on NextGen Flight Decks: Guidance for the Design, Evaluation, and Approval of Visual, Auditory, and Tactile Displays and Controls Research Requirement Description Sponsor Outcome Human factors specialists in the Aircraft Certification Service requested support development of guidance for the design and certification of multimodal flight deck interfaces. After a review of regulatory and guidance material related to clutter on visual and auditory displays and controls, researchers conducted a survey of current and proposed NextGen visual, auditory and tactile displays and controls. They also performed an analysis of clutter-related Aviation Safety Reporting System (ASRS) reports and a literature and regulatory material review on tactile displays and controls. Researchers also completed a simulator study on the use of spatial tactile cues for separation/collision avoidance and are currently working on a simulator study examining the effectiveness of visual and auditory feedback for touch screens. Sponsor: Cathy Swider, AIR-134 Critical Milestones Research Accomplishments in FY16 Review of regulatory and guidance material related to clutter on visual and auditory displays and controls; survey of current and proposed NextGen visual, auditory, and tactile displays and controls; analysis of clutter-related Aviation Safety Reporting System (ASRS) reports Complete Simulator study on the use of tactile spatial cues for separation/collision avoidance during closely spaced parallel approaches Complete Simulator study on the effectiveness of visual and auditory touch screen feedback Complete Draft report on ‘Preventing Clutter and Confusion on NextGen Flight Decks’ FY16 Human Factors REDAC Fall Meeting Empirical basis for regulatory guidance for flight deck tactile displays and empirical basis for regulatory guidance for visual & auditory touch screen feedback. This regulatory guidance applies to Part 23, 25, 27, 29. Simulator study on the effectiveness of visual and auditory touch screen feedback 02/28/2017 Federal Aviation Administration 8

Pilot Response to Non-RA Traffic with TCAS Research Requirement Description Sponsor Outcome Aircraft Certification Service human factors specialists are supporting the development of RTCA SC-147 to develop Minimum Operational Performance Standards for the next generation Airborne Collision Avoidance System, known as ACAS X. This system is expected to replace the existing Traffic Alert and Collision Avoidance System (TCAS), and will provide Traffic Advisories (TAs) for aircraft that are not emitting a transponder signal but are broadcasting Automatic Dependent Surveillance-Broadcast (ADS-B) out. However, Resolution Advisories (RAs) will not be provided for such ADS-B-out aircraft. Pilot performance is being measured to determine whether pilots using simulated TCAS and ACAS X systems will be likely to delay action necessary to maintain a safe distance from the traffic if they are unaware that a target of this type will never generate an RA, since pilots have experience with (and may have come to depend upon) TCAS providing TAs and RAs. Sponsor: Cathy Swider, AIR-134 Critical Milestones Results from simulation study of Pilot Response to Non-RA Generating Traffic Empirical basis for display requirements in RTCA minimum operational performance standard DO-185. Research Accomplishments in FY16 Complete Completed simulation trials Analyzed simulation and questionnaire data Delivered quick-look data analysis Delivered final report FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 9

Low Visibility Operations/ Surface Movement Guidance and Control System (LVO/SMGCS) Research Requirement Description Sponsor Outcome The purpose of this task is to identify and address human factors considerations for LVO/SMGCS operations, specifically LVO/SMGCS chart symbology. The Volpe Center will also support the FAA by gathering data on appropriate and representative chart symbology for LVO/SMGCS operations that can contribute to the FAA’s ICAO involvement on charting symbology. This research will be used to identify LVO/SMGCS chart symbology and will support FAA standardization efforts on the ICAO OPS Low Visibility Operations (LVO) Sub Group. Sponsor: POC: Bruce McGray, AFS-410 Critical Milestones Research Accomplishments in FY16 Conduct research study on LVO/SMGCS chart symbology (online research study) Complete Draft report on LVO/SMGCS chart symbology study Complete Final report on LVO/SMGCS chart symbology study Complete FY16 Human Factors REDAC Fall Meeting Completed and Delivered final report on LVO/SMGCS chart symbology study. This project is posted both internal to FAA (DOT/FAA/TC-16/25) and on the Volpe internal website. Federal Aviation Administration 10

NEXTGEN-HUMAN FACTORS GUIDELINES FOR ADVANCED INSTRUMENT PROCEDURE DESIGN AND USE FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 11

NextGen: Human Factors Guidelines for Advanced Instrument Procedure Design and Use (A12C.HFNG.2) Sponsor Outcome Research Requirement Description Identify issues and develop human factors guidelines for the design, depiction, usability, and flyability of instrument procedures and associated charts for advisory material and standards for instrument procedures and associated charting. Support updates to FAA Orders in the 8260 series, FAA Order 8900.1, AC 120-76, and others to mitigate complexity and challenges to flightcrew compliance with PBN procedures. Examine procedure and chart design factors that affect complexity, and develop design recommendations that limit complexity to mitigate challenges to flightcrew compliance with RNAV RNP procedures. Sponsors: John Swigart & Jeff Kerr (AFS-470), Kathy Abbott (AIR-100) Reduce accident rate and improve operational implementation of PBNbased airspace procedures that enable NextGen operational capabilities and improve NAS capacity and efficiency. Critical Milestones Identify and document factors affecting procedure and chart complexity (FY16) Evaluation of briefing strips for RNAV departures and arrival (FY17) FY16 Human Factors REDAC Fall Meeting Research Accomplishments in FY16 Completed report on human factors issues for the complexity of instrument procedures Federal Aviation Administration 12

Subjective Instrument Procedure Complexity Research Requirement Description The Flight Technologies and Procedures Division of the Flight Standards Service has requested human factors R&D to support updates to the FAA’s 8260 series orders that guide designers of instrument flight procedures. Research will support development of human factors guidance that will address issues concerning the design and depiction of terminal instrument flight procedures and associated aeronautical charts. Research will provide data to help the FAA consider flightcrew issues as early as possible in the design of new flight procedures to smooth their operational implementation. The data will allow Flight Standards human factors specialists to develop guidance to address hazards and risks associated with human-automation interaction in multiple aircraft types, operators, and equipment (e.g., VNAV). Sponsor Outcome Empirical basis for instrument procedure design for NextGen arrivals and departures to update FAA’s 8260 series orders to mitigate complexity in flightcrew interaction with flight deck systems. Sponsor: John Swigart & Jeff Kerr, AFS-470 and Kathy Abbott (AIR-100) Critical Milestones Report on human factors issues for instrument procedures’ complexity Complete Research Accomplishments in FY16 Draft final report – 5/16/16 Updated conference paper based on sponsor & PM input – 8/8/16 Briefing to FAA’s Metroplex Teams – 8/16/16 FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 13

NEXTGEN-PROCEDURES, TASKS, SKILLS AND TRAINING FOR NEXTGEN AIR CARRIER PILOTS AND DISPATCHERS FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 14

NextGen: Procedures, Tasks, Skills, and Training for NextGen Air Carrier Pilots and Dispatchers (A12C.HFNG.3) Research Requirement Description Proliferation of NextGen (and other) applications have created challenges for pilot monitoring, task and attention management. Identify and address changes to job tasks, personnel knowledge, skills and abilities, design of procedures and new training and proficiency standards for NextGen operations. Sponsors: Rob Burke, Doug Farrow(AFS-280) Critical Milestones Complete conceptual framework for development of standard operating procedures for airline operations. (FY16) Develop data to support guidelines for normal Sponsor Outcome Develop or update regulatory and guidance material for airline procedures and training. Research Accomplishments in FY16 Completed draft report with recommendations for updating Advisory Circular (AC) 120-71 Standard Operating Procedures for Flight Deck Crewmembers. procedures for airline operations (FY16) Develop research plan and conduct studies to validate training recommendations for flight path monitoring. (FY17) FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 15

Design of Standard Procedures Research Requirement Description Sponsor Outcome The Flight Standards Air Transportation Division last updated Advisory Circular (AC) 120-71 Standard Operating Procedures for Flight Deck Crewmembers in 2003. Numerous incidents in which flightcrews failed to adhere to standard procedures have increased regulators’ focus on these types of errors. They have also highlighted the need to update guidance material in AC 120-71 and Principal Operations Inspecter surveillance of and air carrier flightcrew conformance with standard procedures. Results will also benefit the airline industry in developing their procedures and flightcrew training , and thus will improve the safety of the flying public. Technical guidance based on the latest research on the design of flight deck procedures for the update of AC 120-71 and guidance for airline flightcrew procedure training. Sponsor: Doug Farrow, AFS-280 Critical Milestones Research Accomplishments in FY16 Draft report summarizing findings in Task 1, considerations in the design of procedures with respect to technology, humans, and the operational environment. Draft report describing general systems approach for developing and revising procedures, with particular focus on RNP operations, NextGen operations, and pre-departure procedures. Complete Draft report describing operational data collection and analysis processes and methodologies. Draft report describing methods for implementing procedures, and for evaluating the implementation of procedures. Final report on flight deck SOPs delivered to FAA. Complete FY16 Human Factors REDAC Fall Meeting Draft report delivered 6/30/16 Complete Complete 02/28/2017 Federal Aviation Administration 16

NEXTGEN-HUMAN ERROR AND COMPLEX SYSTEMS FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 17

NextGen: Human Complex Systems and Error Complex Systems (A12C.HFNG.1) Research Requirement Description NextGen is introducing additional complexity and new opportunities for errors. Examine changes in human-system coordination and potential cognitive and manual skill degradation Address the impact and role of information automation on flightcrew operations and awareness. Sponsors: Cathy Swider (AIR-134), Kathy Abbott (AIR100), Doug Farrow (AFS-280) Critical Milestones Develop research plan to mitigate cognitive skill degradation & conduct experiments (FY16) Final report identifying gaps in guidance related to system complexity. (FY16) Provide recommendations and guidelines for flightcrew task management.(FY17) FY16 Human Factors REDAC Fall Meeting Sponsor Outcome Develop and update regulatory and guidance material (14 CFR Part 121, Subparts N, O, Y (AQP); 14 CFR Parts 2X.1301, 25.1302, etc.) Research Accomplishments in FY16 Completed review of complexity-related literature/guidance material/ASRS reports. Developed online questionnaire for gathering pilot opinions on complexity, collected data, and completed the data analysis. Draft report on task management and fault management. Federal Aviation Administration 18

Complexity: Definitions, Empirical Findings and Recommendations for Training and Design Research Requirement Description Sponsor Outcome R&D is needed to improve understanding of how system complexity affects and is experienced by flight crews. Design features and operational contexts contributing to the phenomenon, its performance effects, and gaps in existing guidance documents will be identified. Results from human factors R&D efforts will support Aircraft Certification Service human factors specialists who will update regulatory and guidance material pertaining to flight deck complexity, and supports implementation of the new rule 14 CFR 25.1302 Installed systems and equipment for use by the flightcrew (May 2013). Empirical basis for regulatory guidance including 14 CFR 25.1302, for flight deck systems with regard to the complexity of the pilot interaction and recommendations for pilot training on the use of complex flight deck systems Sponsor: Paul Sigmund (TAD); Kathy Abbott (AIR-100) Critical Milestones Research Accomplishments in FY16 Review of scientific literature and regulatory/guidance material related to system complexity Complete Completed review of complexity-related literature/guidance material/ASRS reports Analysis of complexity-related ASRS and accident reports Complete Completed data collection and analysis of pilot survey of complexity issues Final state-of-the-art report on complexity (draft and revised final version based on FAA feedback) 09/30/2016 FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 19

NEXTGEN-DATACOMM HUMAN FACTORS R&D FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 20

NextGen: DataComm Human Factors R&D (A12C.HFNG.5) Research Requirement Description Identify and address human factors issues to enable DataComm implementation. Provide data to address safety and usability and support international coordination and enhancement of the CPDLC message set. Address flight crew procedures and performance, message set, and air/ground aspects. Sponsor Outcome Enable successful implementation of data communications in the NAS and the dependent NextGen operational capabilities, resulting in reduced controller workload and greater efficiency. Develop and update regulatory and guidance material for operational approval of flightcrew procedures and training, as well as recommended practices/guidance for operators. Harmonize international data communications message set and operational procedures. Sponsors: Mark Patterson (AFS-470), Tom Kraft (AVS) Critical Milestones Research Accomplishments in FY16 Provide human factors recommendations on CPDLC messages for NextGen applications. (FY16) Delivered report “Flightcrew Procedures for Controller Pilot Data Link Communications (CPDLC)” Complete analysis of voice communication errors for en route airspace to establish a wider pre implementation baseline (FY16) Delivered report “Analysis of Reportable Events in Kansas City Center” FY16 Human Factors REDAC Fall Meeting Federal Aviation Administration 21

BACKUP Federal Aviation Administration 22

NextGen Evaluation of LVO SMGCS using EFVS Without Airport Infrastructure (Taxiways) Research Requirement Description Sponsor Outcome This research program provides a means for evaluating the human factors and pilot performance aspects of using Enhanced Flight Vision Systems (EVFS) in environments where the usual low-visibility-operations support infrastructure is not available (airports not so equipped). Sponsor: POC: Bruce McGray, AFS-410 Critical Milestones Draft research plan on LVO/SMGCS using EFVS without airport infrastructure Report on LVO/SMGCS study on EFVS without airport infrastructure This research will determine the limitations of use of EFVS to support landing and taxi/surface operations. It may allow lowvisibility operations to be extended to airports that do not have specific infrastructure designed to support these operations under current regulations. It would extend the range of weather conditions in which aircraft could access unequipped airports, increase the safety with which aircraft could operate at these airports in reduced visibility, and possibly open up additional airports as “alternate” options when reduced-visibility conditions are encountered. Research Accomplishments in FY16 Complete 9/30/16 Completed evaluation of simulator platforms to support research study Completed draft specifications on research design and procedures Approved MOU with FEDEX for data collection. Federal Aviation Administration 23

Enhanced Flight Vision System (EFVS) for Operational credit to 300ft RVR / to Enable Lower than Standard Takeoff Minima Impact Assessment (Takeoff) Research Requirement Description Sponsor Outcome MITRE CASD must work collaboratively with FAA’s NextGen office and Flight Standards Technologies and procedures Division, AFS-400, to assess the impact of using EFVS for lower than standard takeoff minima to inform RTCA-213 standards development MITRE CASD must also provide technical and operational expertise/inputs to the development of a research plan led by the FAA to evaluate airport infrastructure requirements with EFVS for takeoff. This research will be used to inform new conops and operational improvements for lower than standard takeoff minima operations. It will inform RTCA-213 standards development. With regards to takeoff, it will determine the minimum airport infrastructure requirements for aircraft equipped with EFVS and could extend the range of weather conditions in which aircraft could takeoff and increase the safety with which aircraft could operate at these airports in reduced visibility. Sponsor: POC: Terry King, AFS-410 Critical Milestones Research Accomplishments in FY16 Database of Airports with Delays/Closures due to Low Visibility Complete Report on Impact Assessment of Lower than Standard Takeoff Minima for Low Visibility Operations Complete Research Plan on EFVS to Enable Lower than Standard Takeoff Minima Impact Assessment 02/15/2017 Impact Assessment Database consisting of 300 plus airports and the times and frequency of delays and closures due to low visibility. delivered to MITRE tracker on 12/22/2015. Delivered in Tableau Reader. Briefed sponsors and provided demo of impact assessment database Federal Aviation Administration 24

NextGen Synthetic Vision Technology Comparison between Head-up, Head-mounted, & Head-down Displays Research Requirement Description Sponsor Outcome HMD applications for civil aviation. This research will study the human factors aspects for certification and approval of such displaysystem technologies. Specifically, this research will identify the pilot interface and usability aspects of head-up and head-mounted display technologies, with special attention to synthetic-vision applications. Baseline reference will be to certified head-down synthetic-vision systems. The goal is to identify criteria for evaluating the equipment which are not presently contained in applicable regulations that address proper function, intuitive use, and potential hazards. This research will be used to provide certification and approval criteria and guidelines for evaluating head-up, headdown, and head-mounted displays for use in the cockpit. The regulations will be updated to address proper function, intuitive use, and potential hazards, particularly as it relates to synthetic vision applications. Sponsor: POC: Kathy Abbott, CSTA (AIR-100); Terry King, AFS-410 Critical Milestones Executive report on synthetic vision comparison on HMD/HDD/HUD differences Research Accomplishments in FY16 3/30/2017 Completed study design Awarded contract for construction of mount for head-worn device Federal Aviation Administration 25

NextGen Human Factor Issues with 3D Planar, Stereoscopic, and Holographic Displays (3D Displays) in the Flight Deck Research Requirement Description Sponsor Outcome The purpose of the current research is to examine the technologies underlying 3D planar, stereoscopic and holographic displays, to identify human factors considerations in their design and evaluation. In Phase 1, we will identify human factors considerations when using 3D displays on the flight deck by conducting a literature review and emerging market survey. In Phase 2, we will select from appropriate representative candidate products to evaluate the human factors concerns revealed in Phase 1 to develop an evaluation guide for Aircraft Certification personnel (as needed). Manufacturers (e.g., Honeywell) are developing future technologies that will require updated regulatory guidance for certification. This research will be used to identify the human factors considerations for regulatory guidance material for 3D planar, stereoscopic, and holographic displays. Sponsor: Cathy Swider, AIR-134 Critical Milestones Report on human factors issues with 3D holographic and stereoscopic displays Research Accomplishments in FY16 Complete Completed Literature Review of 3D displays Draft Report on Industry Survey completed and reviewed by sponsors. Federal Aviation Administration 26

NextGen Decision Point: Eye and Gaze Control Research Requirement Description Sponsor Outcome This research program is intended to provide human factors data and input to support the development of regulatory and guidance material for gaze controls. Specifically, this research will identify pilot interface and usability considerations for using eye gaze as a component of multimodal interactions with avionics displays. This research program is intended to provide human factors data and input to support the update of AC 20-175 with regards to the viability and HF considerations associated with using gaze control as a control technology in the cockpit. Sponsor: Cathy Swider, AIR-134 Critical Milestones Decision point Eye Gaze Control report Research Accomplishments in FY16 Complete Completed literature review and industry survey and delivered to sponsor for review. Sponsor briefing to discuss findings and changes to technical report Federal Aviation Administration 27

NextGen Flight Deck Multifunction Touch Screen Controls Research Requirement Description Sponsor Outcome Flight deck controls are being used in new ways. In particular manufacturers are using touch screens for many new applications and functions. Previously under OTA DTFAWA-10-A-80031 Task 9 Human Factors Issues with NextGen flight Deck System Controls many human factors questions surrounding multi-touch surfaced. The purpose of this work is to (1) Identify potential issues with multiple pilots inputting data/touching same display (2) Determine whether inadvertent touches and error rate will bet higher with multi-touch (3) Determine whether lack of feedback is an even larger problem for multi-touch gestures. Produce scientific information that will inform Aircraft Certification Personnel who will evaluate the use of multitouch screen controls on the flight deck and to generate recommendation for updated guidance to be incorporated into Advisory Circular 20-175 Controls for Flight Deck Systems Sponsor: Cathy Swider, AIR-134 Critical Milestones Research Accomplishments in FY16 Monthly Status Reports (23 ) Mar 31, 2018 Support Technical Conferences, Meetings, and symposia Apr 30, 2018 Draft Technical Report that includes test results and updated recommendations Apr 30, 2018 Final Technical Report with comment dispositions Jun 30, 2018 Kickoff meeting on 1/28/16 Reviewed data collection methodology 4/21/16 Federal Aviation Administration 28

Interval Management/Terminal Sequencing & Spacing Research Requirement Description Sponsor Outcome The FAA’s ANG-C1 and AIR-134 organizations, in coordination with the Surveillance Broadcast Services (SBS) Program Office, is requesting MITRE to refine the concept for IM-S integration within a future terminal environment that is informed by an empirical data collection activity. The FAA (Program Management Office (AJM), Mission Support Services (AJV), and other stakeholders) established a Working Group that is tasked to update the current IM-S Arrival Approach & Cruise (IM-S AA&C) ConOps by including more detailed definition of IM-S integration in the context of future en route and terminal environments. Sponsor: POC: Bruce McGray, AFS-410 Critical Milestones Research Accomplishments in FY16 Human Factors Human-in-the-loop (HITL) Simulation of the IM-S Integrated Operation—Simulation Plan Oct 2016 Human Factors Human-in-the-loop (HITL) Simulation of the IM-S Integrated Operation—'Quick-Look' Results Briefing Feb 2017 Human Factors Human-in-the-loop (HITL) Simulation of the IM-S Integrated Operation—Simulation Results Final Report June 2017 Conducted IM TSAS Concept Evaluation Event 2 with external participants on July 26 and 27, 2016 Federal Aviation Administration 29

Considerations for the Use of Airport Moving Maps in Low Visibility Research Requirement Description Sponsor Outcome This area of research will address design, installation and integration issues associated with use of airport moving maps in combination with Enhanced Vision System (EVS) or Synthetic Vision System (SVS) systems in NextGen to provide low visibility taxi, takeoff and approach eventually leading to operations in zero forward unaided visibility and zero cloud ceiling conditions gate to gate. This research will be used to identify gaps in requirements and research relative to using an Airport Moving Map Display in conjunction with EFVS to support LVO/SMGCS operations. Current FAA regulatory and guidance material for Airport Moving Map Display and EFVS under LVO/SMGCS do not provide an adequate foundation to enable the replacement of existing required information sources on the flight deck (i.e., LVO/SMGCS paper charts). Sponsor: POC: Terry King, AFS-410 Critical Milestones Research Accomplishments in FY16 Considerations for the use of AMM in LowVis Ops—Initial Report Complete Considerations for the use of AMM in LowVis Ops—Final Report Complete Met with ANG-C1 and AIR to review an updated version of the AMM Low Vis Ops Final Report (10/22/2015) Received additional sponsor feedback on AMM Low Vis Ops (12/10/15); comments were reviewed and updates are ongoing with performer Brought on collaboration of Volpe with MITRE for LVO ops under existing contract (FY15.HFNG.4) Federal Aviation Administration 30

Briefing Strips for Arrivals & Departures Research Requirement Description Sponsor Outcome The Flight Standards Flight Technologies and Procedures Division has identified a human factors concern for flightcrew use of area navigation ( RNAV) and required navigation performance ( RNP) Arrival and Departure Procedures. Complexity of the procedures may pose challenges for effective crew coordination. Prior studies on Instrument Approach Procedures (IAP) led to development of IAP briefing strips and identified the potential need for alternative visual depiction of information for RNAV and RNP Arrival and Departure procedures to facilitate rapid and accurate crew briefings. Empirical basis for instrument procedure design for NextGen arrivals and departures with recommendations of potential content for alternate visual depictions of information on RNAV and RNP Arrival and Departure Procedures that may support updates to FAA Orders in the 8260 series, FAA Order 8900.1 to mitigate complexity and challenges to flight crew compliance with PBN procedures. Sponsor: Mark Steinbicker, AFS-470 Critical Milestones Report summarizing need for an alternate visual depiction for RNAV/RNP arrival/departure procedures Research Accomplishments in FY16 02/28/2017 Reviewed relevant Volpe studies Initial draft questions for structured interviews with pilots/training personnel submitted for FAA review – April 2016 Federal Aviation Administration 31

Standard Operating Procedures: Flight Path Management in Air Carrier Operations Research Requirement Description Sponsor Outcome The Performance Based Aviation Rulemaking Committee (PARC) Flight Deck Automation Working Group (FltDAWG) final report titled “Operational Use of Flight Path Management Systems” published a series of recommendations in November 2013 that address a number of contributing factors to recent aviation accidents related to flight path management. The FAA has a need to characterize Flight Path Management (FPM) skills including Pilot Monitoring duties and flight crew Mode Awareness in a line operations environment. Provide research which will inform recommendations for updates to Part 121 training practices and provide a basis for recommendations formed by industry working groups including the Air Carrier Training Aviation Rulemaking Committee (ACTARC) Flight Path management work stream. Sponsor: Kathy Abbott, CSTA (AIR-100) and Robert Burke, AFS200 Critical Milestones Research Accomplishments in FY16 Standard Operating Procedures: Air Carrier Flight Training Department Interviews Draft Technical Report 1/31/2017 Standard Operating Procedures: FPM Air Carrier Line Pilot Survey Update Draft Technical Letter 3/31/2017 Standard Operating Procedures: FPM Air Carrier Line Pilot Survey Draft Technical Report Standard Operating Procedures: Flight Path Management in Air Carrier Operations - Final Report Kickoff meeting on June 27, 2016 9/30/2017 12/31/ 2017 Federal Aviation Administration 32

Flight Path Management—Manual Flight Operations Research Requirement Description Sponsor Outcome Modern air carrier transport aircraft continue to employ advanced automated system technology in an effort to decrease pilot workload and improve operational reliability and efficiency. One of the mainstays of today’s aircraft are the automated systems that provide flight path guidance and control capabilities, broadly categorized to include autopilot/flight director and auto-thrust (auto-throttle) systems, which have, in fact, improved safety and efficiency in commercial air transportation. However, evidence suggests that the manual flight operation skillset of pilots operating these highly automated aircraft might be declining. Recommend standard definition of manual flight operations utilizing unique data analytics capability including Aviation Safety Information Analysis and Sharing (ASIAS) program. Sponsor: Kathy Abbott, CSTA (AIR-100) and Robert Burke, AFS-200 Critical Milestones Research Accomplishments in FY16 Manual Flight Operations: Define and Characterize 4/30/2017 Manual Flight Operations: Operational Analysis 8/31/2017 Manual Flight Operations Flight Final Technical Report 3/31/2018 Kickoff meeting on June 27, 2016 Federal Aviation Administration 33

PARC/CAST FltDAWG Report Update Research Requirement Description Sponsor Outcome In November 2013, the Performance Based Aviation Rulemaking Committee (PARC) Flight Deck Automation Working Group (FltDAWG) published a series of recommendations that address factors that contributed to aviation accidents related to flight path management. FAA’s ANG-C1 requires an update to the accident/major incident report data and analysis contained in the original work. Provide an update containing relevant major accidents and major incidents published by investigative authorities since the original FltDAWG report. Sponsor: Kathy Abbott, CSTA (AIR-100) and Robert Burke, AFS-200 Critical Milestones Updated PARC/CAST Flight Deck Automation Working Group's Accident And Major Incident Analysis Research Accomplishments in FY16 2/28/2017 Kickoff meeting on June 27, 2016 Federal Aviation Administration 34

Task Management - Flight Deck Task Management Research Requirement Description Sponsor Outcome The Flight Standards Service has identified a need for human factors recommendations and guidance to update 14 CFR Part 121, Subparts N, O, and Y, as well as Advisory Circulars 120-54 (2006), 120-90 (2006), and 12051 (2004) to address flightcrew performance in managing the flight deck tasks in normal and non-normal situations. Human factors research is needed to examine the distribution of workload, task prioritization, and coordination in non-normal situations. The goal is to better understand task management as it relates to pilot performance and flight deck operations. This will enable researchers to assist Flight Standards in creating a framework from which to develop recommendations and guidelines to address issues in the NextGen environment, including guidelines for pilot training, operational procedures, and flight deck design. Research to support the update of regulatory and guidance material, specifically 14 CFR Part 121, Subparts N, O & Y and updates of Advisory Circulars, to include AC 120-71. Sponsor: Doug Farrow, AFS-280 and Kathy Abbott, CSTA (AIR-100) Critical Milestones Research Accomplishments in FY16 Final Task Management report 09/30/2017 Delivered preliminary Task Management report Final report describing Fault Management analysis 09/30/2017 Delivered preliminary Fault Management report Federal Aviation Administration 35

Task Management – Electronic Display Task Management Research Requirement Description Electronic Flight Bag (EFB) capabilities continue to evolve with the advance in portable electronic device technologies and applications. Advisory Circular 120-76C does not specifically address the challenge of using multiple EFB applications and the impact on flightcrew performance, although it does state, “The EFB software design should minimize flightcrew workload and head-down time.” Aircraft Certification Service human factors specialists have requested research to identify human factors considerations with the simultaneous use of multiple applications on an electronic display (including EFB) and to identify issues related to task switching, workload, distraction, and head-down time. Sponsor Outcome Produce data driven, performance based research to update human factors guidelines for revisions to AC 120-76c. Sponsor: Cathy Swider, AIR-134, Kathy Abbott CSTA (AIR-100) Critical Milestones Research Accomplishments in FY16 Draft Interview questions for pilots Complete Draft report on EFB/PED task management literature review, airline EFB training materials and preliminary findings from pilot interviews about PED usage on May 31, 2016 Complete Revised draft report on EFB/PED task management literature review, airline EFB training materials and preliminary findings from pilot interviews about PED usage on Aug 15, 2016. Complete Delivered Interview questions for pilots on April 15, 2016 Delivered draft report on EFB/PED task management literature review, airline EFB training materials and preliminary findings from pilot interviews about PED usage on May 31, 2016 Revised (#2) draft report including EFB/PED task management literature review, airline EFB training material, and pilot interviews with results, summary, proposed research plan and next steps Federal Aviation Administration 36

Data Communications Human Factors Research Requirement Description Sponsor Outcome Research pilot performance in the use of digital communications to provide recommendations for FAA policies, US and international standards, and operator best practices for flightcrew procedures and training. Examine flightcrew procedures for the use of pilot-controller data link communications (CPDLC) and recommend best practices to promote human performance. Examine pilot deviations related to loss of communications and other types of communication errors in the en route environment. Empirical basis for requirements in RTCA SC-214 minimum operational performance standards and recommended flightcrew procedures for controller-pilot data link communications. Sponsor: Mark Patterson, AFS-470 and Tom Kraft, CSTA (Aeronautical Communications) Critical Milestones Research Accomplishments in FY16 Report on DataComm SC-214 message set human factors considerations for segment 2 Complete DataComm/Digital NOTAMS Research Plan Complete Final report on loss of comm preimplementation analysis – 1.3 Complete Summary of human factors research contributions to PARC AWG and FAA policy and guidance material developments – 1.2 Complete FY16 Human Factors REDAC Fall Meeting Delivered report “Flightcrew Procedures for Controller Pilot Data Link Communications (CPDLC)” - DOT/FAA/TC‐15/56. Significantly contributed to producing RTCA DO-350 (Safety and Performance Requirements for Baseline 2 ATS Data Communication). Delivered report “Analysis of Reportable Events in Kansas City Center” Federal Aviation Administration 37

075 09 UAS Minimum Detect and Avoid Display & Flight Path Information Research Requirement Description Sponsor Outcome The FAA’s Unmanned Aircraft Systems (UAS) Program Office requested a study that would provide scientific data to inform development of regulatory and guidance materials for Aircraft Certification of UAS control stations, specifically, to establishing the minimum requirements for approving detect and avoid (DAA) system displays that are necessary to allow a UAS pilot to remain well-clear of other aircraft. The data support development of regulatory and guidance material that addresses traffic Information, alerting, and flight path guidance to provide adequate UAS pilot performance. These data will be used by RTCA SC-228 for the Minimal Operational Performance Standards for DAA. Sponsor: Stephen Plishka, AFS-86 Critical Milestones 26 -09 5 nm -25 Empirical basis for display and alerting requirements in RTCA SC-228 minimum operational performance standard for UAS detect and avoid systems. Research Accomplishments in FY16 Run simulation for UAS Detect and Avoid (WJHTC) Complete Technical Report on minimum DAA display requirements, alerting requirements, and human interface recommendations (CAMI) 12/31/2016 Run full mission simulations using recommended displays 09/30/2017 Report documenting the results of full mission simulations 09/30/2017 FY16 Human Factors REDAC Fall Meeting 16UAL 455 10 Completed initial simulations for UAS Detect and Avoid – March 2016 Completed data analysis Completed quick-look on initial study findings Federal Aviation Administration 38