Engineering Elegant Systems: Engineering at the System Level 22 June

Engineering Elegant Systems: Engineering at the System Level 22 June 2017 Michael D. Watson, Ph.D. www.nasa.gov/sls Consortium Team UAH George Washington University Iowa State Texas A&M University of Colorado at Colorado Springs (UCCS) Missouri University of S&T University of Michigan Doty Consulting Services AFRL Wright Patterson Space Launch System National Aeronautics and Space Administration

Outline Understanding Systems Engineering Discipline integration Modeling System State Variables and Goal function tree State Analysis model System Value Model Products Engineering Elegant Systems: Theory of Systems Engineering Engineering Elegant Systems: The Practice of Systems Engineering Summary 2

Understanding Systems Engineering

Motivation System Engineering of Complex Systems is not well understood System Engineering of Complex Systems is Challenging System Engineering can produce elegant solutions in some instances System Engineering can produce embarrassing failures in some instances Within NASA, System Engineering does is frequently unable to maintain complex system designs within budget, schedule, and performance constraints “How do we Fix System Engineering?” Michael D. Griffin, 61st International Astronautical Congress, Prague, Czech Republic, September 27-October 1, 2010 Successful practice in System Engineering is frequently based on the ability of the lead system engineer, rather than on the approach of system engineering in general The rules and properties that govern complex systems are not well defined in order to define system elegance 4 characteristics of system elegance proposed as: System Effectiveness System Efficiency System Robustness Minimizing Unintended Consequences 4

Consortium Research Process Multi-disciplinary research group that spans systems engineering areas Selected researchers who are product rather than process focused List of Consortium Members Michael D. Griffin, Ph.D. Air Force Research Laboratory – Wright Patterson, Multidisciplinary Science and Technology Center: Jose A. Camberos, Ph.D., Kirk L. Yerkes, Ph.D. George Washington University: Zoe Szajnfarber, Ph.D. Iowa State University: Christina L. Bloebaum, Ph.D., Michael C. Dorneich, Ph.D. Missouri University of Science & Technology: David Riggins, Ph.D. NASA Langley Research Center: Peter A. Parker, Ph.D. Texas A&M University: Richard Malak, Ph.D. Tri-Vector Corporation: Joey Shelton, Ph.D., Robert S. Ryan The University of Alabama in Huntsville: Phillip A. Farrington, Ph.D., Dawn R. Utley, Ph.D., Laird Burns, Ph.D., Paul Collopy, Ph.D., Bryan Mesmer, Ph.D., P. J. Benfield, Ph.D., Wes Colley, Ph.D. The University of Colorado – Colorado Springs: Stephen B. Johnson, Ph.D. Doty Consulting Services: John Doty, Ph.D. The University of Michigan: Panos Y. Papalambros, Ph.D. Previous Consortium Members Stevens Institute of Technology – Dinesh Verma Spaceworks – John Olds (Cost Modeling Statistics) Alabama A&M – Emeka Dunu (Supply Chain Management) George Mason – John Gero (Agent Based Modeling) Oregon State – Irem Tumer (Electrical Power Grid Robustness) Arkansas – David Jensen (Failure Categorization) Massachusetts Institute of Technology: Maria C. Yang, Ph.D. The University of Texas, Arlington: Paul Componation, Ph.D. 35 graduate students and 5 undergraduate students supported to date 5

Understanding Systems Engineering Definition – System Engineering is the engineering discipline which integrates the system functions, system environment, and the engineering disciplines necessary to produce and/or operate an elegant system. Elegant System - A system that is robust in application, fully meeting specified and adumbrated intent, is well structured, and is graceful in operation. Primary Focus System Design and Integration ‒Identify system couplings and interactions ‒Identify system uncertainties and sensitivities ‒Identify emergent properties ‒Manage the effectiveness of the system Engineering Discipline Integration ‒Manage flow of information for system development and/or operations ‒Maintain system activities within budget and schedule Supporting Activities Process application and execution 6

Systems Engineering Postulates Postulate 1: Systems engineering is product specific. Postulate 2: The Systems Engineering domain consists of subsystems, their interactions among themselves, and their interactions with the system environment Postulate 3: The function of Systems Engineering is to integrate engineering disciplines in an elegant manner Postulate 4: Systems engineering influences and is influenced by organizational structure and culture Postulate 5: Systems engineering influences and is influenced by budget, schedule, policy, and law Postulate 6: Systems engineering spans the entire system life-cycle Postulate 7: Understanding of the system evolves as the system development or operation progresses 7

System Engineering Hypotheses Hypothesis 1: If a solution exists for a specific context, then there exists at least one ideal Systems Engineering solution for that specific context Hypothesis 2: System complexity is greater than or equal to the ideal system complexity necessary to fulfill all system outputs Hypothesis 3: Key Stakeholders preferences can be accurately represented mathematically 8

Systems Engineering Principles Principle 1: Systems engineering integrates the system and the disciplines considering the budget and schedule constraints Principle 2: Complex Systems build Complex Systems Principle 3: The focus of systems engineering during the development phase is a progressively deeper understanding of the interactions, sensitivities, and behaviors of the system Sub-Principle 3(a): Requirements are specific, agreed to preferences by the developing organization Sub-Principle 3(b): Requirements and design are progressively defined as the development progresses Sub-Principle 3(c): Hierarchical structures are not sufficient to fully model system interactions and couplings Sub-Principle 3(d): A Product Breakdown Structure (PBS) provides a structure to integrate cost and schedule with system functions Principle 4: Systems engineering spans the entire system life-cycle Sub-Principle 4(a): Systems engineering obtains an understanding of the system Sub-Principle 4(b): Systems engineering models the system Sub-Principle 4(c): Systems engineering designs and analyzes the system Sub-Principle 4(d): Systems engineering tests the system Sub-Principle 4(e): Systems engineering has an essential role in the assembly and manufacturing of the system Sub-Principle 4(f): Systems engineering has an essential role during operations and decommissioning 9

Systems Engineering Principles Principle 5: Systems engineering is based on a middle range set of theories Sub-Principle 5(a): Systems engineering has a mathematical basis ‒Systems Theory Basis ‒Decision & Value Theory Basis (Decision Theory and Value Modeling Theory) ‒Model Basis ‒State Basis (System State Variables) ‒Goal Basis ‒Control Basis (Control Theory) ‒Knowledge Basis (Information Theory) ‒Predictive Basis (Statistics and Probability) Sub-Principle 5(b): Systems engineering has a physical/logical basis Sub-Principle 5(c): Systems engineering has a sociological basis Principle 6: Systems engineering maps and manages the discipline interactions within the organization Principle 7: Decision quality depends on the system knowledge represented in the decision-making process Principle 8: Both Policy and Law must be properly understood to not overly constrain or under constrain the system implementation Principle 9: Systems engineering decisions are made under uncertainty accounting for risk 10

Systems Engineering Principles Principle 10: Verification is a demonstrated understanding of all the system functions and interactions in the operational environment Ideally requirements are level and balanced in their representation of system functions and interactions In practice requirements are not balanced in their representation of system functions and interactions Principle 11: Validation is a demonstrated understanding of the system’s value to the system stakeholders Principle 12: Systems engineering solutions are constrained based on the decision timeframe for the system need 11

Sociological Concepts in Systems Engineering Specification of Ignorance is important in the advancement of the understanding of the system Consistent use of Terminology is important for Communication within the Organization Opportunity Structures Provide opportunity to mature ideas ‒Task teams, working groups, communities of practice, etc. Socially Expected Durations will exist about the project Both Manifest and Latent Social Functions exist in the organization Social Role Sets Individuals have a set of roles for their position Cultural Subsets will form i.e., disciplines can be a subset within the organization Insider and Outsider attitudes can form ‒Be Aware of the Self-Fulfilling Prophecy, Social Polarization Reconsiderations Process (i.e., Reclama Process) Provides ability to manage social ambivalence Must be able to recognize social beliefs that may be contributing to the disagreement Helps to avoid putting people in to social dysfunction or complete social anomie ‒Conformity ‒Innovation ‒Ritualism ‒Retreatism ‒Rebellion 12

Information Flow Information Flow through a program/project/activity is defined by Information Theory Organizational communication paths Board Structure Decision Making follows the First Postulate Decision Process is specific to the decision being made Tracked 3 SLS CRs, with 3 separate task team processes, all had equally rated effectiveness Margin is maintained by the Organization, not in the margin management tables Biased Information Sharing Margin Management is focused on Managing the Disciplines (informed by the System Integrating Physics) SLS Organizational Structure was defined by the LSE as a recommendation to the Chief Engineer and the Program Manager 13

Discipline Integration Models Organizational Values Value Model Goal Function Tree (GFT) Goals Organizational Structure & Mapping Value Attributes System Functions Agent Based Model (ABM) Discrete Event Simulation System Dynamics Model Allow systems engineers to: Understand information flow through the development and/or operations organization Integrate discipline information into a system level design Analyze information flow, gaps, and blind spots at the System level MagicDraw Enterprise (SysML) Matlab Matlab StateFlow JAVA Anylogic Extend

What is System Dynamics? A methodology that has been used extensively in business and engineering environments to develop better understanding of the interactions that define the operation and behavior of complex systems Based on the concept that complex systems can be subdivided into simpler, more manageable components The high-level operation of an overall system can be defined by examining and defining the behavior and interactions of the individual components. “While it is often difficult for scientists to intuitively understand the overall operations of complex systems there is often a great deal of in-depth knowledge on the operations of the structural components that constitute the system.”

Tools and Methodologies Tools and techniques have been developed using the System Dynamics methodology that make it possible to efficiently decompose complex systems and to quickly set-up and test models of system operation. Tools promote understanding through visual diagramming and modeling. disposal available? Spent Fuel Interim Storage dispose stored spent fuel max enrich rate frac pu in spent fuel dispose stored spent fuel pu burn rate spent fuel to interim storage burn f uel rate reprocess rate burn fuel rate U Being Enriched Uranium Fuel Enriched Uranium Spent Fuel Cooled Spent Fuel Material Being Reprocessed Interim Wast e Storage Final Disposal enter enrich enrich manufacture U fuel burning fuel cool spent fuel cool fuel time max man u rate man rep U man rep U rate reprocess spent fuel reprocess spent fuel? max reenrich rate frac u in spent fuel reenrich repu? Rep U disposal available? final disposal open date recycling u Enriched RepU deplete dispose stored waste waste to interim storage reenrich U enrichment fraction repu enrichment level u235 fuel ratio reprocess mox deplete repu tails fraction fract pu in mox feed fraction Depleted U Mox Fuel man mox Pu dispose stored mox reprocess mox? frac pu in spent fuel Pu 02 dispose mox rate recycling pu man mox U enrichment fraction fract pu in mox man mox rate burn mox rate burn mox pu burn rate cool mox time Irrad MOX Cooled MOX Spent Mox Interim Storage cooling irrad mox burning fuel fraction mox in mox reactors spent mox to interim st orage

Roles of System Dynamics Creates a Structured View of the Problem Space. Allows Visualization and Testing of Assumptions and Postulated Relationships. Dynamically Traces Potential Results of Modifications. Communicates Operational Understanding to Others. PRODUCTION DATA PRS Weekly Data to FACTS PRS Data D fa1 prs1 Production Readings Collect Readings pd1 pd2 pd3 pd4 pd5 PRS Data Approve Data Enter into PRS Review Data Enter Production Dataxfer to shore pd6 pd7 pd8 PRS Data C prs2 Daily Data to FACTS fa2 Monthly Data to FACTS PRS Data E prs3 fa3

Strategic Analysis System Dynamics is frequently used to support long-term strategic decision-making Modeling of strategic decision space often requires at least some tactical level modeling Art is in decomposing model to a level that captures driving relationships but is not too complex to accurately or efficiently model Stochasticity is often an important driver in system behavior Typical strategic models are run probabilistically, with major drivers represented as prob. distributions

Applications Methodology has been applied across many disciplines which involve complex systems with limited high-level operational understanding: - Business and monetary systems Power generation and distribution Information technology networks Aircraft /spacecraft design and operation Nuclear reactor operation Military operations NASA STS/ISS Transportation/Operations Analysis Develop a methodology to evaluate strategic level options for the re-supply and operation of the ISS-STS system Evaluate various potential transportation systems Analyze impact (financial and performance) on Exploration System

STS-ISS Transportation / Operation Analysis For Each Probabilistic Case Vehicle Reliability Probability of Loss of Crew for Alternative STS Failure Rates 100% 90% (1/23) 80% Cum ulative P r obability 70% J (1/76) 60% Nominal Flight Schedules Flight Rates Lockout Restrictions F M A M J J A S Quiescent Periods O N D STS P RA 1/23 (95% ) STS P RA 1/76 (Mean) 50% STS 1/123 (50% ) 40% STS 1/617 (5% ) (1/123) 30% 20% P U P U P (1/617) 10% 0% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 Year Vehicle Descriptions Upmass /Downmass Capacity Pres & Unpres Carrier Limits Altitude Adjustment Shuttle HTV ATV AAS Delivered Cargo Probabilistic Cases OSP Soyuz Progress Other Returnable Cargo Vehicle Loading Probabilistic Cases On-Orbit Parameters Desired Recoverable Fraction Cargo On-Orbit Period Vehicles to Station Return Vehicles ? Crew Time Requirements 100.0 RSA Utilization (39.45%) NASA/Other IP Utilization (60.55%) 80.0 Returned Cargo Purchase 75% of Russian Crew Time 60.0 40.0 20.0 0.0 Core Complete Core Core Core Complete Complete Complete w/Russ ian w/2 EDO with Soyuz Purchase STS Overlap 6 crew 7 crew Required Cargo Vehicle Loading Cargo in Queue Resupply Requirements (kg ) 4 5 ,0 0 0 7, 449 7, 449 2, 528 2, 690 2, 527 7, 449 7, 449 7, 450 7, 449 7, 449 7, 449 7, 448 7, 448 2, 480 2, 480 2, 480 2, 480 2, 480 2, 480 2, 480 2, 480 2, 480 5, 369 5, 604 5, 419 5, 638 5, 838 5, 779 5, 745 5, 886 1, 8 49 1, 82 9 1 , 8 26 1, 8 41 1, 84 9 1 , 8 57 1, 83 1 1 , 8 18 6, 861 6, 995 6, 993 6, 993 6, 994 6, 994 6, 993 6, 995 2011 201 2 2013 2014 2015 2016 201 7 2018 1 0 ,0 0 0 5 ,0 0 0 5, 844 1 , 8 46 6, 992 0 Yea r 20 19 40% Loading Factors Carrier Arrangements Average Tare Factors 0. 9 1 0. 9 7 0. 8 6 Ut i l i zat i on - Mi ddeck 0. 76 Ut i l i zat i on - Press ur i zed 0. 83 Ut i l i zat i on - Unpress ur i zed 0. 97 AMS 0. 9 3 30% 20% 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 Fracti on of Requi red Upmass 2, 690 2, 689 7, 257 1 5 ,0 0 0 Logi st i cs & Mai nt enance Unpres s ur i zed 50% * 7, 449 2, 471 2 0 ,0 0 0 Logi st i cs & Mai nt enance Pressur i zed 60% I SS l i f et i m e t hr ough 2 02 2; pos t - Enhanc ed Co nf i gura t i o n Car g o Re s uppl y f ro m 2 00 9 t hro ug h 20 22 * * 7 Spac e Shut t l e Fl i g ht s e ver y 2 y ear s po s t - re cov e ry ( 3 Pr es s uri ze d & 4 Unpre s s ur i z ed) ; 0. 8ATV & H TV f l i ght pe r ye ar 3 5 ,0 0 0 2 5 ,0 0 0 Crew Suppor t 70% - 4 0 ,0 0 0 3 0 ,0 0 0 ExpectedVal ues 80% 0% Non-Queuing Cargo 5 0 ,0 0 0 Baseline Case - SSV Reliabilit y 1/ 100 90% 10% Cargo Upmass Requirements 5 5 ,0 0 0 Expected Cargo 100% Probabi l i ty of Exceedi ng Fract i on of Requi red Upmass Average Utilization Hours per Week 120.0 1. 0 Propel l ant 0. 93 Wat er & Gas 0. 86 EVA Rot at i on 0. 98

ISS-STS Transportation / Operations Analysis Supplies to be Delivered u supplies in vehicle? Missions actual launch to ISS Total Flights U P1 Vol u P3 Vol u P2 Vol u carrier mass limit P on orbit time Vehicles at ISS flights u Cap P 1 u P cap u Cap P 2 u load P 2 u Supplies at ISS load U 3 u carrier mass limit U U cap u load U 1 u return flights Supplies to be Returned launch load P 3 u load P 1 u Cap U 1 u actual launch to ISS using supplies Cap U 2 u load U 2 u carrier mass limit M M cap u load M 1 u AMS cap u load AMS 1 u carrier mass limit AMS Cap M 1 u Cap AMS 1 u load M 2 u Cap M 2 u load M 3 u load AMS 2 u Cap AMS 2 u load AMS 3 u carrier mass limit water gas water gas cap u load water gas u extra 2 cap u load extra 2 u water gas 2 uCap water gas 2 uload water gas 3 u Cap water gas load u useage period returnable fraction mass loads up no accom mass loads down no accom returning supplies delivering supplies carrier mass limit extra 2 Cap extra 2 u load extra 2 2 u Cap extar 2 2 u load extra 2 3 u Supplies to be dump? Delivered Mass Returned carrier mass limit Prop Prop Cap u load Prop 1 u Cap Prop 1 u load Prop 2 u Cap Prop 2 u crew req load Prop 3 u dump pres L&M req mass limit u Cap V 2 u Cap V 1 u large unpres L&M req Vehicle Cap u PU 3 u PU 2 u PU 1 u unpres L&M req generating supplies pres util req schedule AMS Total P 1 u supply priority up supply priority up Total P 3 u Total P 2 u unpres util req supply priority up EVA Rotation req tare multiplier u tare multiplier u cargo load 1 u cargo load 2 u tare multiplier u cargo load 3 u water gas req supply priority up mid deck req prop req

Methods of System Integration Goal: Techniques to Enable Integrated System Design and Assessments by the Systems Engineer

System Models Contain an Understanding of the System Value Model State Variables Engineering Statistics Goal Function Tree (GFT) System Functions & State Variables Goals System Functions & State Variables System State Transition Model Discipline Physics Models System Integrated Physics Model (System Exergy) Multidisciplinary Design Optimization (MDO) Allow systems engineers to: Define system functions based on the system state variables Understand stakeholders expectations on system value (i.e., capabilities) Integrate discipline engineering models into a system level physics based model (e.g., system exergy) Design and Analyze system responses and behaviors at the System level MagicDraw Enterprise (SysML) Matlab Matlab StateFlow Microsoft Excell

System Design and Integration

System Physics and System Integrating Physics Goal: Utilize the key system physics to produce an elegant system design

System Integrating Physics Consortium is researching the significance of identifying and using the System Integrating Physics for Systems Engineering First Postulate: Systems Engineering is Product Specific. States that the Systems are different, and therefore, the Integrating Physics for the various Systems is different Launch Vehicles Thermodynamic System Spacecraft Integrated through the bus which is a thermodynamic system ‒Each Instrument may have a different integrating physics but integrates with the bus thermodynamically Other Thermodynamic Systems Crew Modules Fluid Systems Electrical Systems Power Plants Automobiles Aircraft Ships Not all systems are integrated by their Thermodynamics Optical Systems Logical Systems ‒Data Systems ‒Communication Systems Biological Systems System Integrating Physics provides the engineering basis for the System Model

Spacecraft Integration Model 𝟐 𝟐 2𝟐 2𝟐 𝒆,𝒏𝒈𝒊𝒆 𝒆,𝒕𝒉𝒓𝒖𝒔𝒕𝒆 4 4 𝑐,𝑓𝑖𝑛𝑎𝑙 𝑣𝑒h𝑖𝑐𝑙,𝑓𝑛𝑎𝑙 𝒗𝒆𝒉𝒊𝒄𝒍,𝒇𝒊𝒏𝒂𝒍 𝑐,𝑖𝑛𝑡𝑎𝑙 𝑣𝑒h𝑖𝑐𝑙,𝑛𝑖𝑡𝑎𝑙 𝒗𝒆𝒉𝒊𝒄𝒍,𝒏𝒊𝒕𝒂𝒍 𝑬 𝒗𝒆𝒉𝒊𝒄𝒍,𝒏𝒊𝒕𝒂𝒍 𝑬 𝒗𝒆𝒉𝒊𝒄𝒍,𝒇𝒊𝒏𝒂𝒍 𝑓𝑙𝑢𝑖𝑑 𝒑𝒓𝒐𝒆𝒍𝒂𝒏𝒕,𝒆𝒈𝒊𝒏𝒆 𝒑𝒓𝒐 ,𝒆𝒏𝒈𝒊𝒆 𝒑𝒓𝒐𝒆𝒍𝒂𝒏𝒕,𝒉𝒓𝒖𝒔𝒕𝒆 𝒑𝒓𝒐 ,𝒕𝒉𝒓𝒖𝒔𝒕𝒆 𝑟𝑎𝑑𝑖𝑡𝑜𝑟 𝑠𝑝𝑎𝑐𝑒 𝒃𝒖𝒔 𝒃𝒖𝒔 𝒅𝒆𝒔 𝒗𝒆𝒉𝒊𝒄𝒍,𝒇𝒊𝒏𝒂𝒍 𝒗𝒆𝒉𝒊𝒄𝒍 ,𝒏𝒊𝒕𝒂𝒍 𝑖𝑛𝑠𝑡𝑟𝑢𝑚𝑒𝑛𝑡 𝑒𝑙𝑐𝑡𝑟𝑖,𝑠𝑜𝑟𝑒𝑑 𝑒𝑙𝑐𝑡𝑟𝑖,𝑢𝑠𝑒𝑑 𝑡 𝒂𝒍𝒕𝒊𝒖𝒅𝒆,𝒊𝒏𝒕𝒂𝒍 𝒂𝒍𝒕𝒊𝒖𝒅𝒆,𝒇𝒊𝒏𝒂𝒍 𝑖𝑛𝑠𝑡𝑟𝑢𝑚𝑒𝑛𝑡 𝑏𝑎𝑡𝑒𝑟𝑦 𝑖𝑛𝑠𝑡𝑟𝑢𝑚𝑒𝑛𝑡𝑠 𝑽 𝑽 𝐼 𝜔 𝑽 𝐼 𝜔 𝑽 𝑮𝒙𝑴 𝑮𝒙𝑴 𝑇 𝒎 𝒉 𝒎 𝒉 𝜎𝐴𝑒(𝑇 𝑇 ) 𝑽 𝑰 cos(𝜃))𝚫𝒕 𝑿 𝑴 𝑴 1 𝑄 𝑃 𝑃 𝚫𝒕 𝟐 𝟐 𝟐𝟐 𝟐𝟐𝒓 𝒓 𝑇 ( ) ( ) ( ) ( )( ) ( ) ) 27

System State Variables Goal: Utilize system state variables to understand the interactions of the system in relation to system goals and system execution

System State Models System Stage Models represent the system as a whole in terms of the hardware and software states that the system transitions through during operation Goal Function Tree (GFT) Model “Middle Out” model of the system based on the system State Variables Shows relationship between system state functions (hardware and software) and system goals Does not contain system physical or logical relationships and is not executable System State Machine Model Models the integrated State Transitions of the system as a whole (i.e., hardware states and software states) Confirms system functions as expected ‒Checks for system hazardous, system anomalies, inconsistent state progression, missing states, improper state paths (e.g., short circuits in hardware and/or software design) ‒Confirms that the system states progress as stated in the system design Executable model of system 29

What is the “Discipline of Systems Engineering” (DSE)? A new kind of system modeled on classical engineering disciplines See 2 papers on DSE listed in backup The bases (systems, model, state/function, goal, control, value, knowledge, statistical) provide theoretical background for the introduction of a suite of models as the means by which systems engineering defines and analyzes systems The model suite implement the theoretical ideas inherent in the bases The model suite should at a minimum replicate, eventually at higher fidelity and lower cost, what systems engineering currently can achieve Lower cost depends on development of tools and automation DSE will be able to do more than what traditional systems engineering currently can achieve 30

Loop-Spiral of DSE Representations Models of Preference Concept of Operations Model Models of Intention ValueDecision Model Anomaly Investigation Goal-Function Tree Design Optimization Models of Failure Performance Models Fault Tree Failure Propagation Model State Machine Model Design Model Models of Design Physical Containment Model Rounded boxes are functions, not models. Cost & Schedule Models Organization Hierarchy Model Pattern Recognition and Anomaly Detection Physical Simulation Models Tests Models of Behavior Models of Agency 31

What is a Goal-Function Tree? A top-down hierarchical decomposition of system goals and functions --- hierarchies are models of “intention” ‒Arranged by major system phase/configuration, ‒Defines the functions the system must perform and goals the system must achieve for the system to successfully perform its mission/objectives ‒Relationship between goals and functions defined through rigorous use of state variables The GFT extends the classical systems engineering function decomposition through the use of state variables, which makes the GFT causally, physically correct

State Variable Methodology State variables are defined as inputs and outputs to functions: y(t) f(x,t) ‒x inputs to the functions f ‒f transforms the inputs into the outputs y Goals Requirements define intended range of the output state variables y Failure state (value) of output state variable y is out of intended range G 𝒚 Function f State variables enforce strong connection of the functional decomposition to the system’s physical laws and causation The state variables exist in both functional and design representations, and enable formal connection between the two 33

Nominal and Off-Nominal Goals For every nominal goal there is the possibility of an off-nominal goal, which is activated if the nominal goal is not achieved ‒ Value (state) of state vector y goes out of intended range If a new off-nominal goal is identified, this is an operational Fault Management goal, with associated off-nominal functions to detect or predict failure, and respond G 𝒚 OR Nominal Function f o re u l fai cc urs , or 𝒘 Off-Nominal Function

Why Does NASA Space Launch System Build & Use a Goal Tree? Used for Fault Management / System Health Management The “Dark Side” of Systems Engineering Does SLS detect failures properly in all SLS-caused failure scenarios that threaten the crew? ‒ i.e. failure detection coverage and failure scenario definition What is the relationship of crew-threatening behaviors to each other, to system goals, and to potential detections? What is the optimal distribution of Fault Management (FM) detections and responses? ‒ Generally, the smallest number of detections that cover the widest set of threats to goals Early in a program, detailed design and related FMEAs (failure modes and effects analyses) do not exist, but need to start FM early in design phase ‒ This must be a top-down analysis based on goals and intended functions, not just on design (SLS is only partly heritage) For a complex system, it is impossible to determine all possible ways the system can go wrong, but we can and should be able to specify completely what must go “right”! ‒ Need to base SHM/FM design in part on protecting the functions needed to succeed, regardless of how they might fail

Ascent/Abort GFT Example

GFT-based FM Analysis #1 Every unique path up the GFT from bottom to top can be associated with a failure scenario ‒This does not generate all possible failure scenarios ‒GFT paths are only nominal paths, but failures can create new off-nominal paths, such as structure breakage or electrical short-circuits ‒Scenario definition also requires a fault tree, not just a success tree, but this fault tree must be based on the same state variable methodology as the GFT Failure detection coverage is determined by identifying all paths that have at least one failure detection along the path, and conversely for any noncovered paths ‒Nuance: when the same state variable exists in more than one GFT location, it often indicates a different requirement / range constraint levied on the same state variable: example---acceleration constrained by need to protect the crew, the crew capsule structure, and the launch vehicle structure ‒When this occurs, it is possible that the threshold values set for the failure detection associated with the state variable can be set to the wrong value; it needs to be set to the tightest requirement 37

GFT-based FM Analysis #2 Optimization of failure detection & response based on the distribution and relationship of failure detections along GFT paths ‒At least one failure detection should exist along every GFT path ‒When more than one failure detection exists along a GFT path, then there is redundant detection for a given scenario. These could indicate excess redundancy, and this should be assessed. It could be that the detections are needed to cover other scenarios. ‒Failure response effectiveness is based on the race condition of the FM response speed compared to the failure effect propagation rate. The latter are related to the types of physics indicated by the state variables along the GFT paths (note not all failure paths exist in the GFT, see previous page). Examples: electrical state variables indicate electron flows with characteristic speeds; these differ from pressure and temperature state variables and their characteristic times for fluid flows. ‒Some paths have failure probabilities higher than other paths. For these it is appropriate to have detections “lower down” in the GFT to make more response time available before compromise of high-level goals. 38

Using the GFT within DSE GFT to Fault Trees Logical complement of GFT, then add new failure branches Fault trees have more branches than success trees, because failures create new “off-nominal paths” in the system More generally, failure space models are the “more complete” system models than success space models GFT-Functions & Requirements to Organizations and Organization Interfaces State variables key GFT Design Model Physical Containment Model (Configuration Items) Organization Model Nominal and Failure Scenarios for Requirement Verification Paths from bottom to top of GFTs and Fault Trees, when implemented with state variables Test procedures and observables extracted from trees 39

Traceability from Functions to Organizations; GFT to Fault Trees 40

Conclusion GFT is a central representation of systems engineering A key representation early in the design process to define intention in a physically accurate way Physical, causal accuracy enables its use for many more purposes than is possible for a classical functional decomposition GFT being successfully used today on NASA SLS to support the design of Fault Management (System Health Management, FDIR (Failure Detection, Isolation, and Response) GFT can be and will be used for many more purposes in the future Requirements traces, generation of fault trees, generation of ICDs/IRDs 41

Booster – CS Ascent GFT System Works

State Analysis Foundations Ares-Orion Communication During Aborts LADEE FM Software Development NESC Toyota Investigation ARES ARES ORION ORION After Orion Requests Auto-Safe Authority, Communication between Orion and Ares is Lost and Ares Exceeds Auto-Safe Threshold: -Ares: Abort condition exceeds the auto-safing threshold. Because Orion has the auto-safe authority, Ares sends an “auto-safe authority pass-back request” and waits for Orion to send receipt. (Same as in “nominal” scenario) Model Answers What if Scenario - Orion: Because communication is severed between Ares and Orion, Orion is not aware of the auto-safe authority request and remains in prior states. - Ares: Ares can NOT perform safing actions. SLS M&FM Analysis LADEE FM Operations

System State Machine Model The state analysis model is split into two main components: Manager software model System Plant Modeled using MATLAB Stateflow Allows the software model to look like the SysML Activity Diagrams Allows the SystembPlant to be modeled as State Machines Allows those two models to interact with each other within the MATLAB environment ‒Facilitates the ability to generate custom analysis tools Reads in command sequence to execute model 44

State Analysis Model for SLS M&FM Commands Commands From Launch Countdown Doc Control (SysML to Stateflow) 14% of R12 modeled Over 7,200 Transitions in the Vehicle and Software Over 3,500 States in the Vehicle Sensor Values Plant (State Machines) Faults Physics Values

What Comes from a State Analysis Model? Commands Open-Loop Commands from Launch Countdown Doc Control (SysML to Stateflow) Plant (State Machines) Sensor Values

State Analysis GUIs

State Analysis Results blocker; 2 time worksforme; 2 3 manager; 2 1 defect; fault management; 1 avionics; mission manager; 6 critical; cs tvc; 1 11 technical; 21 Tickets Closed; 29 invalid; 14 trivial; 10 propulsion; 18 fixed; 13 Tickets Open; 27 editorial; 28 minor; 17 major;22 16 engine; booster; 5 finding; 4

System Value Goal: Utilize system state variables to understand the interactions of the system in relation to system goals and system execution

System Value Model A System Value Model is a mathematical representation of Stakeholders Preferences (Expectations) for the system The basic structure is straight forward The sociology/psychology of representing the Preferences can be a challenge The System Value Model is the Basis of System Validation!!! The Requirements and Design Models form the basis of System Verification The System Value Model forms the basis of System Validation Constructing an SLS Value Model to compare to System Validation results Can expand to Integrated Stack with input from MPCV and GSDO System Value model also provides basis for a measure of System Robustness How many mission types are supported by the system? 50

Integration with the Capability Model Congressional Model Development Cost Capability Value Mission 1 Mission 2 SLS Attributes Mission 3

Production Function 𝑌𝑇 𝐴𝑅 [𝛿 𝐾𝑇(𝜎 1)/𝜎 (1 𝛿) 𝐿𝑇(𝜎 1)/𝜎]𝜎/(𝜎 1) Other Preferences 𝑌𝑇 Total output (Best Value from substituting 𝐾𝑇 and 𝐿𝑇) 𝜎 Elasticity of Substitution 𝛿 Distribution Parameter ‒Value between 0 and 1 𝐴𝑅 Productivity Factor 𝐾𝑇 Value of capital/services from Program 1 (SLS) ‒Societal Benefits, Resource Value, and New Information from program 1 𝐿𝑇 Value of capital/services from Program 2 (Other) ‒Societal Benefits, Resource Value, and New Information from program 2

Fundamental Objectives Value model must quantitatively answer the question: “How does the SLS contribute to NASA’s top-level objectives?” Concrete contributions of the SLS are shown below as “sub-goals”. Note that SLS contributes to the four positive objectives indirectly (for the most part) by enabling missions – the missions themselves are the actual engines of value generation. Prior work on launch vehicle value models is in the context of specific missions – these value models, while useful, are really value models for the missions, not the launch vehicle itself. TopLevel Objs. SubGoals Generate National Prestige Impress other Nations Impress Impress the Public Public Further Scientific Knowledge Enable Missions Missions Increase Military Capability Support Economic Growth Create Jobs & Support Industry Minimize Negative Impacts Minimize Environmental Damage Minimize Loss of Life A value model for the SLS should value the mission-enabling capability of the vehicle in the generic sense. Sli de -

Constructing the Value Model A generic value model maps a design, described by an envelope of top-level attributes, to a scalar value. This model can (and should) be capable of accepting uncertain attributes and propagating them through to output an uncertain value. This uncertain value can then be adjusted for risk attitude if desired, or (if decision-maker is risk-neutral) the expected value can be used as the final “score” for the design. A proposed valuation paradigm for the mission enabling capability of SLS should have two distinct components: Capability model (from system behavioral models – “what can it do?”) Value model (from preference elicitation – “how does this benefit us?”) Design variables, environmental variables, operational variables (includes uncertainties) Lower-level Attributes Value Model Capability Model (physics, software, etc.) (preferences) Top-level Attributes System Value Sli de -

Constructing the Value Model Modifications to basic value model framework for SLS: Because the SLS does not create value directly (instead enabling missions that create value), the basic value model framework must be modified. Instead of mapping capability directly to value, instead capture all relevant capabilities of the vehicle and interface them with a portfolio of individual mission value models. Design variables, environmental variables, operational variables (includes uncertainties) Lower-level Attributes Value Model Capability Model (physics, software, etc.) (preferences) Top-level Attributes System Value Sli de -

The Capability Envelope The first step in constructing the value model is to determine the appropriate abstractions for capturing the capabilities of any given launch vehicle. What are the “top-level attributes” for a launch vehicle? A top-level attribute is defined as any system attribute that directly impacts value delivery – or to put it another way, any system attribute that a potential “customer” of SLS might care about. It is important to identify which system attributes are top-level attributes. Top-level attributes (such as cost, payload mass to orbit, and weather envelope) are derived from other attributes (such as materials used, engine specific impulse, and controllability) via various behavioral models. Any given system design is represented by an N-dimensional envelope in top-level attribute space. This envelope is here called the “capability envelope”. It is then mapped to value by interfacing with mission value models. Sli de -

The Capability Envelope The capability envelope is the fundamental abstraction that allows for valuation of any given launch vehicle design. Why an envelope specifically? Systems are commonly represented with vectors in attribute space. “Envelope” implies instead a bounded region of attribute space. For many top-level attributes, a vector is sufficient to represent capability. However, certain relationships should be expressed as continuous relationships or envelopes in order to accurately model value. Definition of “capability envelope”: The Capability Envelope is the combination of all top-level attribute values and all relationships between top-level attributes that fully define any individual design concept. Sli de -

The Capability Envelope One of the most important parts of the capability envelope is the answer to the question “how much can it carry, and how far?” This is expressed as an envelope bounded by the curve representing the relationship between max payload mass and total energy imparted. Can be expressed as “C3 vs payload mass” or “DV vs payload mass”. Converting between the “C3 vs payload mass” and “DV vs payload mass” curves is trivial as long as the required DV to some reference orbit can be assumed or estimated from aerodynamic and trajectory models. OR Sli de -

The Capability Envelope Other top-level attributes included in the capability envelope: Cost (including production, integration, launch, etc.) Reliability (probability of mission success for any given launch) Payload services provided (power, thermal conditioning, etc.) Load factors imparted to payload (max Q and during dynamic events) Shock loads imparted to payload (max Q and during dynamic events) Controllability envelope (wind speeds vs fairing dims vs CoG offset) Allowable payload volumes (related to fairing dimensions) Time between launches (including assembly time, pad time, etc.) Injection accuracy (altitude and inclination) Ability to operate in loss of communication (autonomy) Important to capture relationships that may restrict variables, even if the variables may affect different aspects of value. Example: controllability envelope relates wind speeds (affects value by determining how often vehicle can launch) to fairing dimensions (affects value by determining which payloads can be carried). Sli de -

The Capability Envelope Note that the capability envelope is quite specifically the inherent capabilities of any given single design (see slide 15). The capability envelope is not a design trade-space envelope. The former is an abstraction of a single design in the space of all possible designs, while the latter is a representation of all designs that are possible. Example: the relationship “max wind speed vs fairing diameter” is part of the capability envelope (assuming multiple fairings exist for a single design), while the relationship “cost vs reliability” is not (unless you’ve designed an SLS that literally allows for trading those off on any given launch). The capability envelope is not a mission trade-space envelope. The former is the inherent capabilities of the vehicle, while the latter is related to specific mission planning. Example: the relationship “max payload mass vs max transfer energy” is part of the capability envelope, while the relationship “flight duration vs transfer energy” (as shown on a porkchop plot) is not. Sli de -

Mapping Capability to Value This is only the basic framework for the valuation model. Implementation may vary, but these core elements should remain. There is not necessarily a “correct” specific implementation of this framework – the implementation is limited in detail only by the designer’s knowledge of their preferences and/or ability to codify them. The “mission portfolio” valuation paradigm is beneficial because it allows for the consideration of the entire gamut of launch vehicle applications throughout its lifetime. The following slide presents a general illustration of how the mission portfolio maps the capability envelope to value. Mission A Mission C Mission B 20,000 m/s dV required 15,000 m/s dV required 32,000 m/s dV required Value 50000 * m Value 30000 * m Value 80000 * m Demand 25% of total Demand 60% of total Demand 15% of total

Mapping System Capability to Value & Missions Attempted “Will it work?” (Reliability) “What can it carry?” Load Factors Shock Loads Payload Volume Payload Services Injection Accuracy “How expensive is it?” Production cost Launch cost etc. Missions Succeeded Total Value Delivered by Launch Vehicle 62

Mapping Capability to Value Possible variations on this valuation framework: Mission Classifications: ‒While location is used in the example as the sole classifier for missions, it is far from being the only potentially useful classifier. ‒Missions could also be broken up by purpose (science/military/commercial), manned/unmanned, etc. – it is up to the designer to decide what is most useful. ‒The important thing is that each unique mission archetype has a characteristic C3/DV requirement, valuation function, and demand. Value Curve: ‒Instead of having a required C3/DV for each mission and value being a function of mass, value for each mission could simply be a function of C3/DV and mass. ‒This is a more complicated formulation that would be much more cumbersome to construct, but it would allow for more realistic representations of certain situations (such as scientific mission to a specific location that can accomplish more with a smaller payload and greater DV than with a larger payload and smaller DV). ‒Again, it is up to the designer to decide which formulation will best serve the project – in essence trading model fidelity for model effort. Sli de -

Mapping Capability to Value Possible variations on this valuation framework: Mission Demand: ‒Various restrictions could potentially be placed on any given mission’s demand to more accurately reflect real-world limitations – “no more than X launches of Y mission type every Z years”, etc. ‒These would allow for more realistic modeling of the change in value delivery granted by having a vehicle that can launch more frequently. ‒Example: It could be the case that no matter how often the launch vehicle can launch, you do not want to (or cannot) launch more than 2 Jupiter missions every 10 years, but would instead do many more LEO and Luna missions). General: ‒Various restrictions could potentially be placed on individual missions – “this mission requires at least X kg of payload to be attempted”, that sort of thing. ‒Any small tweaks which increase the fidelity of the model are welcome, however designers must remain conscious of the increased effort that will result. ‒It is recommended to start with a low-fidelity model (similar to that which is shown in the example formulation) and refine as needed instead of attempting to construct a highly detailed model from the start. Sli de -

Summary Discussed approach to Engineering an Elegant System Discussed Systems Engineering Framework and Principles System Integration Engineering Discipline Integration Discussed several methods and tools for conducting integrated system design and analysis System Integration ‒System Integrating Physics ‒System Design and Optimization ‒Engineering Statistics ‒State Variable Analysis ‒System Value Discipline Integration ‒Sociological Principles and Cognitive Science ‒Decision Making ‒Policy and Law Application Processes Application Systems Engineering Approach defined in two documents “Engineering Elegant Systems: Theory of Systems Engineering” “Engineering Elegant Systems: The Practice of Systems Engineering” 65

Backup 66

System Exergy Efficiency S-II Center Engine Cut-Off S-1C Stage Separation S-II Stage Separation S-1C Center Engine Cut-Off S-IVB Burn 1 Cut-Off LEO Insertion S-II Engine Mixture Ratio Shift S-IVB Burn 2 Engine Mixture Ratio Shift S-IVB Burn 2 Cut-Off Max Q S-!VB Separation 67

Crew Module Exergy Balance: ISS ECLSS 𝑽 𝟐 𝒇𝒊𝒏𝒂𝒍 𝟐 𝒊𝒏𝒊𝒕 𝒂𝒍 𝑐𝑎𝑏𝑖𝑛 𝟐 𝒊𝒏 𝟐 𝒐𝒖𝒕 𝑽 𝑇 𝑇𝑐𝑎𝑏𝑖𝑛 𝑇𝑐𝑜 𝑙𝑎𝑛𝑡 𝑽 𝑽 𝑿𝑬𝑪𝑳𝑺 𝒎𝒇𝒍𝒖𝒊𝒅 (𝒉𝑓𝑖𝑛𝑎𝑙 𝒉𝑐𝑎𝑏𝑖𝑛) 𝑇𝑐𝑎𝑏𝑖𝑛(𝒔𝑓𝑖𝑛𝑎𝑙 𝒔𝑐𝑎𝑏𝑖𝑛) 𝒎𝒇𝒍𝒖𝒊𝒅 (𝒉𝑖𝑛𝑡𝑖𝑎𝑙 𝒉𝑐𝑎𝑏𝑖𝑛) 𝑇𝑐𝑎𝑏𝑖𝑛(𝒔𝑖𝑛𝑡𝑖𝑎𝑙 𝒔𝑐𝑎𝑏𝑖𝑛) 1 𝑄𝑐𝑟𝑒𝑤 𝑄𝑇𝑀𝑆 𝑾𝑬𝑷𝑺 𝑷𝒄𝒂𝒃𝒊𝒏(𝑽𝒐𝒍𝒇𝒊𝒏𝒂𝒍 𝑽𝒐𝒍𝒊𝒏𝒊𝒕𝒂𝒍) 𝒎𝒊𝒏 (𝒉𝑖𝑛 𝒉𝑐𝑎𝑏𝑖𝑛) 𝑇𝑐𝑎𝑏𝑖𝑛(𝒔𝑖𝑛 𝒔𝑐𝑎𝑏𝑖𝑛) (𝒉𝑜𝑢𝑡 𝒉𝑐𝑎𝑏𝑖𝑛) 𝑇𝑐𝑎𝑏𝑖𝑛(𝒔𝑜𝑢𝑡 𝒔𝑐𝑎𝑏𝑖𝑛) 𝑿𝒅𝒆𝒔 𝟐 𝑝𝑟𝑜𝑐𝑒𝑠 𝟐 𝑇𝑐𝑟𝑒𝑤 𝑇𝑐𝑜 𝑙𝑎𝑛𝑡 𝟐 𝟐 𝑝𝑟𝑜𝑐𝑒𝑠 ( ( ) ( ( ) ( ) ( ) [ ( ) ( )] Calculate Efficiency 68

Relevant GFT Publications DSE Part 1: Stephen B. Johnson and John C. Day, “Theoretical Foundations of the Discipline of Systems Engineering”, AIAA SciTech/Infotech Conference, San Diego, California, 4 January 2016 Covers Motivation, Approach, Bases, and Principles DSE Part 2: Stephen B. Johnson, “The Representations and Practices of the Discipline of Systems Engineering,” Conference on Systems Engineering Research, Huntsville, Alabama, 23 March 2016 Covers Purposes, Strategies, Representations, and Practices GFT: Stephen B. Johnson, Goal-Function Tree Modeling for Systems Engineering and Fault Management. AIAA Infotech@Aerospace (I@A) Conference, 2013, Boston, MA. August 19-22, 2013. AIAA Paper 2013-4576. 69

System Engineering Supporting Activities Process Application and Execution for the Specific System

Processes Well defined in NASA/SP-2007-6105 Rev1, NASA Systems Engineering Handbook SEMP is essential to capture appropriate application of processes to the specific system Process application is specific to the system being developed ‒Tailoring is not a special exception, it is the norm