ACL Data Council Quarterly Council Meeting April 27, 2020

ACL Data Council Quarterly Council Meeting April 27, 2020

Agenda 1. Welcome and Housekeeping 2. Recap a. Council Purpose b. Council Working Groups 3. Working Groups a. Data Quality b. Data Governance Standards 4. Other Proposed Activities 5. Stakeholder Engagement and Information Dissemination 6. General Questions for the Council

1. Welcome and Housekeeping Call in via phone (not your computer) Mute your phones when you are not speaking Raise your hand and/or use the chat function Repository site to access all Council materials: http://neweditions.net/acldatacouncil Welcome

2. Recap – Purpose of the Council Support ACL’s mission by improving the coordination of ACL’s data governance, including the development of improved processes and standards for defining, collecting, reviewing, certifying, analyzing, and presenting data ACL collects through its evaluation, grant reporting, and administrative performance measures. Help OPE meet its mission to provide and promote high quality, transparent information to support sound decision-making .

2. Recap – Council Working Groups Data Quality Develop standards and guidelines for certifying ACL’s data submissions; developing advisories for ACL's data presentation site regarding possible data quality issues; as ACL accepts data through the various grantee systems, are there clear guidelines and are all of the grant/contract officers using the same standards for looking at the data and accepting it as complete? Data Governance Standards Determine which existing federal standards (e.g., the Evidence Act, Federal Data Strategy) may be applied to ACL data; developing ACL guidance that reflects the needs and capacity of the range of programs within ACL.

3. Working Group: Data Quality Topics/Products Considered Needs assessment to determine what tools are needed by ACL program staff to ensure the quality of program data Checklist with common and specific elements to guide data entry/quality SOPs for ACL staff focused on the review of program data from receipt to publication Text for funding opportunity announcements, such as standard definitions, to communicate what data will be expected Data dictionaries (i.e., data elements, business rules; definitions; examples) Case studies/highlights from early/successful adopters of good data procedures; best practice examples for other grantees (would become template)

3. Working Group: Data Quality Topics/Products Selected Develop and conduct a needs assessment with ACL staff (starting with AoA and AoD staff) As feasible, develop and/or disseminate tools (e.g., SOPs, checklists) that are identified as part of the needs assessment

3. Working Groups: Data Quality Progress to Date Held two meetings Collected existing ACL tools that staff are using or know of to ensure the quality of ACL program data Tools are listed in the working group materials background section: http://neweditions.net/acldatacouncil Developed draft needs assessment

3. Working Group: Data Quality Needs Assessment Intended Use Identify key issues ACL program staff may face that may impact the quality of the data they work with. Help determine what tools are needed by ACL program staff to enhance the quality of program data.

3. Working Group: Data Quality Needs Assessment - SAMPLE

3. Working Group: Data Quality Questions What are strategies for pilot testing the needs assessment (who, how, when)? Who from the Council would like to volunteer to review the needs assessment?

3. Working Group: Data Quality Possible Next Steps Pilot test needs assessment Finalize needs assessment Administer needs assessment to ACL staff

3. Working Group: Data Governance Standards Topics/Products Considered Formal data policy/guidance for ACL; minimum standards/policies that ACL applies that we agree on and can refer back to Guide for the practical application of federal guidance/standards (e.g., What does it mean for your job; What are the rights related to data ownership; How do you define the data you use) Action-oriented products that may become policy (e.g., playbook; worksheet with checklists) or to use for implementing policy; include broad content and tools tailored to programs Adapt existing materials/models for ACL (e.g., data maturity model)

3. Working Group: Data Governance Standards Topics/Products Selected Develop a practical annotated bibliography of relevant data governance frameworks, models, and guidance (resources) and how they apply to ACL and its programs Develop an ACL Data Governance Primer (Primer) that is based on the Department of Transportation primer and is informed by guidance from the resources listed in the annotated bibliography

3. Working Group: Data Governance Standards Progress to Date Held three meetings Identified relevant resources for the annotated bibliography Reviewed resources and developed content to highlight the resources apply to ACL and its Centers and programs

3. Working Group: Data Governance Standards Products’ Intended Use Annotated bibliography to serve as an internal living working document/tool for the working group to help drive decision-making and determine what ACL is legally required to do and what ACL staff needs to think about when making decisions; the bibliography will be updated over time as new federal standards are provided. Primer to help the ACL Centers determine where they are and where they want to be in terms of data governance standards (retrospective and prospective view); provide a set of guidelines to inform ACL data policies and practices; help guide what programs do, and build data governance into program planning.

3. Working Group: Data Governance Standards Annotated Bibliography Resources The Federal Government Data Maturity Model Federal Data Strategy 2020 Action Plan Grant Reporting Efficiency and Agreements Transparency Act of 2019 (GREAT Act) National Information Exchange Model NASA Data Strategy White Paper NIST Privacy Framework OMB Playbook P-20W Data Governance U.S. Department of Transportation Data Governance Plan

3. Working Group: Data Governance Standards Annotated Bibliography – SAMPLE Standards, Frameworks, and Models NASA Data Strategy White Paper Practical Application for ACL Importance/Purpose PURPOSE: The purpose of this document is to establish NASA’s strategy to synchronize and Has specific recommendations to address challenges that ACL faces. leverage the variety of emerging analytic work across the Agency, setting the stage for NASA to maximize momentum with these techniques for the future. Application: ACL 1. Lack of an explicit data management framework, fragmented data DESCRIPTION: This document forms the basis for NASA’s high-level data strategy that lifecycle and lack of data integration-recommendation is 1. Data addresses the diverse needs of the mission and programmatic areas. To this end, it outlines Management 2. Unified Data Lifecycle 3. Data Governance Program. notable challenges, significant opportunities and key recommendations on how the Agency 2. Data expertise gap-recommendation is a Data Fellows Program. should proceed. For each challenge described, it provides examples, opportunities and 3. Need to effectively address culture and policy issues alongside recommendations. technology-recommendation is Data Stewards (e.g., Having individual https://files.meetup.com/19117935/NASA%20-%20Data-Strategy-2015-03-17-White-Paper advocates who help implement the strategy in the individual contexts -Small.pdf will be vital to supporting that type of cultural change.). National Institute of Standards and Technology (NIST). (2020). NIST Privacy Framework: A Tool for Improving Privacy Through Enterprise Risk Management. Practical Application for ACL Programs Application: All Centers Identify Data Stewards who can support good data governance with specific data sets/data collection activities as well as among Center staff. Application: OIRM An explicit data management framework can shape how OIRM develops and implements data collection systems. Application: OPE Initiating a data fellows program open to internal and external applicants can both increase ACL data capacity and help ACL to identify new opportunities for data. Importance/Purpose Application: All Centers Provides a set of privacy protection strategies for organizations that “Privacy Risk Assessment: For example, if an wish to improve their approach to using and protecting personal individual requests access to data, the organization PURPOSE: Help organizations build better privacy foundations by bringing privacy risk into data; also provides clarification about privacy risk management may not be able to produce the data if the data parity with their broader enterprise risk portfolio concepts and the relationship between the Privacy Framework and have been distributed or encrypted in ways the NIST’s Cybersecurity Framework organization cannot access. Privacy risk DESCIPTION: The NIST Privacy Framework, created by the US Department of Commerce, assessments can help an organization understand The NIST Privacy Framework can be used to establish or improve a National Institute of Standards and Technology (NIST), follows the structure of NIST’s privacy program, applied to the System Development Life Cycle, used in a given context the values to protect, the Framework for Improving Critical Infrastructure Cybersecurity. Both frameworks are divided to create the Data Processing Ecosystem, and inform buying methods to employ, and how to balance into three parts: decisions. implementation of different types of measures.” - Core – privacy protection activities and outcomes communicated from the executive level ACL must produce data that is accessible internally to the operations level. Elements include: functions, categories, and subcategories. Application: ACL and easily disseminated to our stakeholders. Functions are further broken down into five privacy risk management areas: Identify, Beneficial for our state, contractual, and grantee partners who expect Govern, Control, Communicate, and Protect. their interactions with ACL systems will not cause adverse Profiles – profiles are broken into Current Profiles and Target Profiles, by reviewing ‘Core’ consequences to their networks. functions. After assessing and identifying the Current Profile state, Target Profiles can be During new tool development: to create the Privacy Framework, NIST created as the organizational desired state, which can be intra- or inter-organizational. conducted public workshops, requests for information (RFI), requests Current and Target Profiles are used to identify gaps and communicate privacy risk for comments (RFC), webinars, and stakeholder interaction. ACL may management. use similar methods to engage our state partners and stakeholders. Implementation Tiers – these are reference points of organization privacy risk views and the current state of risk management. There are four tiers (i.e. Partial, Risk Informed, Repeatable, and Adaptive), although all organizations need not achieve level four status. Tiers can be used to identify resources and processes needed to get an organization up to scale. https://www.nist.gov/system/files/documents/2020/01/16/NIST%20Privacy%20Framework V1.0.pdf

3. Working Group: Data Governance Standards DOT Governance Plan (i.e., the Primer): Goals 1) increased consistency and coordination between existing and new data programs; 2) elimination of redundant data collection; 3) consolidation of data sources and resources; 4) compliance with external mandates.

3. Working Group: Data Governance Standards Primer Sections 1. 2. 3. 4. 5. 6. 7. Data Governance and Stewardship Program Plan Data Governance Work Plan Data Stewardship Roster Data Governance Collaboration Data Policies Data Standards Data Procedures

3. Working Group: Data Governance Standards Primer – Additional Primers 1. Enterprise Architecture 2. Data Analytics and Storage 3. Master Reference Data a. Data program meta-data b. Data dictionary 4. Open Data

3. Working Group: Data Governance Standards Other Models NASA Data Strategy Framework Challenges Examples Opportunities Recommendations Federal Data Strategy Playbook Governance Body Data Maturity Assessment Guidance Metrics Federal Data Strategy Principles Practices Action Plan Data Governance Institute. (2014). The DGI Data Governance Framewor k. Khatri and Brown. (2010). Designing data go vernance Cordoba and Furman. (2017). How to Develop a Sim ple Data Governance Program for a SAS CI E nvironment in 90 Days 10 universal components of a Data Governance program: Rules and Rules of Engagement 1. Mission and Vision 2. Goals, Governance Metrics and Success Measures, and Funding Strategies 3. Data Rules and Definitions 4. Decision Rights 5. Accountabilities 6. Controls People and Organizational Bodies 7. Data Stakeholders 8. A Data Governance Office 9. Data Stewards Processes 10.Proactive, Reactive, and Ongoing Data Governance Processes Framework for data governance includes five interrelated decision domains: 1. Data Principles 2. Data Quality 3. Metadata 4. Data Access 5. Data Lifecycle 1. Organizational Structure 2. Processes and Decisions (i.e., develop policies and performance indicators), and 3. Operational Plan

3. Working Group: Data Governance Standards Questions What are strategies for disseminating the annotated bibliography? What are emerging data governance questions?

3. Working Group: Data Governance Standards Possible Next Steps Obtain ideas form Council members to disseminate the annotated bibliography to ACL staff (email ideas to Anne) Determine format and sections for the Primer Obtain ACL examples from Council members Draft content

Accessing the Working Group Products http://neweditions.net/acldatacouncil/data-council-products

4. Other Proposed Activities Develop sample text for funding opportunity announcements focused on expected data quality standards Conduct literature search for dissemination strategies used by other federal agencies to disseminate data findings and develop summary of findings (August-September) Develop materials to help programs use virtual data collection approaches

5. Stakeholder Engagement and Information Dissemination Who? How? When?

6. General Questions for the Council What has worked well and what needs revisiting? What are effective strategies for ensuring involvement from members who are not on working groups? Now world is so different from when we started the Council, are there other topics/issues we should consider?

Questions? If you have questions, please email: Anne Leopold at [email protected]

Thank you for your time!