Data Management Assessment Framework Assessment Elements Maturity

Data Management Assessment Framework Assessment Elements Maturity Levels Assessment Process and Supporting Tools

NCHRP 20-44(12): Research Implementation Research Objective: Encourage Dissemination and Application of the Data Assessment Practices established in NCHRP Report 814. Data Value Assessment – assess the extent data users feel data is providing value and meeting business needs Data Management Maturity Assessment – assess the agency capability to manage data assets to maximize their value 2

Why Conduct a Data Self-Assessment? Provide a foundation for data strategic plans or business plans Take stock of how well agency data is serving its intended purpose Identify opportunities for making better use of existing data Build awareness of and consensus around needed data improvements Track progress (through periodic re-assessment) Improve data management Derive full value from data investments Systematically improve efficiency and adjust the data portfolio 3

Data Management Maturity Assessment Elements Data Strategy and Governance How decisions are made – What to collect? How to manage and deliver data? Includes roles, accountability, policies, and processes Data Life-Cycle Management How data are maintained, preserved, protected, documented, and delivered Data Architecture and Integration How data is standardized and integrated to minimize duplication and inconsistencies Data Collaboration How collection and management is coordinated with internal and external users Data Quality Management How data quality is defined, validated, measured, and reported 4

Data Management Maturity Levels Maturity Level Definition 1 – Initial Processes, strategies and tools are generally ad-hoc rather than proactive or enterprise-wide; successes are due to individual efforts 2 - Developing Widespread awareness of more mature data management practices; recognition of the need to improve processes, strategies and tools 3 - Defined Processes, strategies and tools have been developed, agreed-upon and documented 4– Processes, strategies and tools are generally being Functioning implemented as defined 5 – Optimizing Strategies, processes and tools are routinely evaluated and improved 5

Assessment Work Plan Overview Individual and Group Assessment Complete individual assessments Summarize individual results Hold group consensus building meeting Discuss potential improvements Action Planning Prioritize gaps Identify priority improvement actions Evaluation and Closeout Support next steps Identify opportunities to improve future research implementations 6

Data Mgmt. Maturity Individual Assessment Tool Sub-element description Sub-Element Maturity Levels Check boxes to assess Current and Desired State Note: a Level 5 is very mature, do not expect to regularly achieve that – even as a desired state! Note: assign the level that best represents the current or desired practice Worksheets by Element Support with comments and potential improvements 7

Data Mgmt. Maturity Consensus Building Tool Summarizes individual responses and allows group consensus to be recorded. Group comments and potential improvements can be captured 8

Assessment Summary (Radar Chart) 1.1 Strategy and Direction 5.2 Data Quality Assurance and 1.2 Imprv. Roles and Accountability 5.1 Data Quality Meas. and Reporting 4.3 Public Data Sharing 5 4 1.3 Policies and Processes 1.4 Data Asset Inventory and Value 3 4.2 External Agency Collaboration 2 1.5 Relationships and Data Customers 1 4.1 Internal Agency Collaboration 1.6 Data Mgmt. Workforce Capabilities 0 3.4 Temporal Data Management 2.1 Data Collection and Updating 3.3 Data Consistency and Integration 2.2 Data Access Control 3.2 Geospatial Data Management 2.3 Data Findability and Documentation 3.1 Location Referencing 2.4 Data Backups and Archiving 2.6 Data Delivery 2.5 Data Change Mgmt 9

Data Management Assessment – Action Planning 1.1 Strategy and Direction SubCurrent Desired Element Level Level 2 4 Consensus Comments Consensus Potential Improvements Business Impacts of Taking No Action Possible Improvements Targeted Priority Improvements Targeted Improvement Descriptions Build from existing draft, supplemented by Finalize Data assessment results. Collect feedback from Strategic Plan Data Innovation Team, and then from Executive Management before finalizing. Provide guidance for what data is defined as Define of "enterprise" interest. This should be tied Enterprise to strategic goals and key data uses and can Data include external data (e.g. open data Concepts sources). Current exercise is Uncoordinated, supporting decentralized establishment of data decisions Data Governance leadership vision Bodies and direction Unproductive, through strategic lower-value Data Governance plan. Desire to Finalize Data data and Stewardship High facilitate Strategic Plan investments Policies integration and Provide Highpartnership across Duplication of Data Business Level Data Provide training to support understanding of data programs data and efforts Plans Management data management roles and responsibilities. towards common Training agency goal and priority. Columns with data entry entirely by the Group Consensus Building Tool. Columns with data entry pre-populated by the Group Consensus Building Tool from the entries on the Element-Specific Worksheets. Columns with data entered by Facilitator during Improvement Identification and Action Planning Discussions. 10

Action Detail Presentation Slide: Improvement 6 Example Action Planning Template Establish and Improve Remote Sensing Data Standards Description and support – ensuring clear understanding of the activity and its execution. Description: Develop minimum, recommended, and optional data collection elements and requirements for various data types and data products. Support: Support with appropriate documentation (e.g. data dictionaries and standards) Evaluate opportunities to leverage/integrate with GIS, CADD and other related data Consider Common Data Repository scoping outcomes and BIM Vision and Strategy Details that can be established during detailed planning. Implementation Lead TBD Benefit of Taking Action Maximize data value and support data integration. Reduce data misuse and misunderstanding Target Completion TBD 11

Detailed Element Descriptions Element 1: Data Strategy and Governance Element 2: Data Lifecycle Management Element 3: Architecture and Integration Element 4: Data Collaboration Element 5: Data Quality Management

Element 1: Data Strategy and Governance How data decisions are made – what to collect? How to manage and deliver data? Addresses roles, accountability, policies and processes. Related Sub-Elements 1.1 – Strategy and Direction 1.2 – Roles and Accountability 1.3 – Policies and Procedures 1.4 – Data Asset Inventory and Value 1.5 – Relationships and Data Customers 1.6 – Data Management Workforce Capabilities 13

Element 1: Data Strategy and Governance 1.1 Strategy and Direction Leadership commitment and strategic planning to maximize value of data to meet agency goals. Focus: Extent to which the agency leadership has demonstrated commitment to managing data as a strategic asset. Improvement: Establishment of data governance structures Set communications and planning activities to align data investments and business needs Benefits of Advancement Move from highly decentralized to more deliberate, coordinated decisions Shift limited resources from lower-value to higher-value data investments Better answer questions such as: ‒ Are we collecting the right data? ‒ Are we managing our data effectively? 14

Element 1: Data Strategy and Governance 1.2 Roles and Accountability Clear roles, accountability and decision making authority for data quality value and appropriate use. Focus: Extent to which roles and accountability for data stewardship have been agreed upon, defined, documented, and assigned Benefits of Advancement Increase clarity and accountability regarding data ownership / stewardship Staff training and resources are aligned with stewardship expectations Proactively and efficiently provide the right data, with the right quality, in the right form Improvement: Scope of agency data governance structures – who decides what? Define specific roles and capabilities Provide appropriate resources and training, and monitor for improvement 15

Element 1: Data Strategy and Governance 1.3 Policies and Procedures Adoption of principles, policies and business processes for managing data as a strategic agency asset. Focus: Extent to which there are clear policies and procedures about how data is to be managed Improvement: Define strategic vision, goals and objectives for data management and capture executive endorsement Document, implement, monitor, support, and continually improve data management policies and practices Benefits of Advancement Increasingly standardize how the agency treats its data assets to result in: ‒ Higher quality data ‒ More effective use of data ‒ Clear decision-making processes around data 16

Element 1: Data Strategy and Governance 1.4 Data Asset Inventory and Value Tracking of agency data assets and their value added. Focus: Extent to which the agency’s data, and its uses and value, are documented Improvement: Create and maintain a data inventory/catalog and expand to include the data’s users, uses, costs, etc. Examine the data assets to identify duplicative datasets and data collection and management methods for improvement Benefits of Advancement Consistently document data and track how it is used Provide a basis for articulating the value of different data to the agency Weigh data collection and maintenance costs against value added Identify areas of duplication and opportunities for consolidation 17

Element 1: Data Strategy and Governance 1.5 Relationships with Data Customers Connections between data producers and users. Focus: Extent to which data program managers have established channels of communication with their data users Improvement: Formalize processes for data user engagement Routinely engage through a variety of forums and formats Establish written agreements with customers regarding data provision/use Benefits of Advancement Strengthen relationships and grow in customer understanding Gather feedback on data uses as well as data quality, availability, and usability Establish and formalize expectations for data provision Mitigate risks for data misuse 18

Element 1: Data Strategy and Governance 1.6 Data Management Workforce Capabilities Attracting, building and sustaining a workforce with the knowledge, skills and abilities to meet changing data management and analysis requirements. Focus: Extent to which the work force has the right mix of skills and the extent to which the agency can sustain skills through transitions Benefits of Advancement Ability to systematically identify risks and discuss associated strategies Reduce risks of disruption to data activities due to workforce related issues Provide orderly and efficient transitions of responsibilities in key positions Improvement: Recognize workforce risks Document and implement increasingly robust mitigation strategies 19

Element 2: Data Life-Cycle Management How data are maintained, preserved, protected, documented, and delivered. Related Sub-Elements 2.1 – Data Updating 2.2 – Data Access Control 2.3 – Data Findability 2.4 – Data Backups and Archiving 2.5 – Data Change Management 2.6 – Data Delivery 20

Element 2: Data Lifecycle Management 2.1 Data Updating Well-defined and coordinated data update cycles. Focus: Extent to which data update methods and cycles have been defined and documented for key datasets Improvement: Define enterprise collection and update cycles, best practices, and business rules Consistently implement practices and follow update cycles and business rules Periodically review and improve Benefits of Advancement Increase awareness and understanding of data collection and update cycles Ensure consistent approaches for adding or editing key data entities (e.g. routes, projects, employees) Embed common data business rules into agency applications and business processes Understand long term maintenance costs and quality requirements 21

Element 2: Data Lifecycle Management 2.2 Data Access Control Well-defined policies and guidelines for managing access to data sets. Focus: Extent to which agency manages access to data sets in order to protect sensitive information and maintain data integrity Improvement: Document and implement standards (beyond enterprise system access controls) to address risks to data access and sharing Periodically review and improve Benefits of Advancement Move from ad-hoc to standard, formal processes Support compliance with information security regulation Prevent data corruption due to unauthorized or unmanaged changes Provide consistent criteria and methods to support information sharing, while protecting sensitive datasets or elements 22

Element 2: Data Lifecycle Management 2.3 Data Findability and Documentation Availability of data catalogs and dictionaries that enable discovery and understanding of available agency data assets. Focus: Extent to which agency ensures that potential data users can discover what data are available and understand the potential applicability of the dataset to a given need Benefits of Advancement Data becomes more findable Additional value derived from existing data Duplicative collections are reduced Reduce risk of data misuse Improvement: Set minimum documentation standards (e.g. catalogs and dictionaries) Implement and monitor standards, ensuring documentation is kept up-to-date Periodically review and improve based on needs, feedback, new technology, and best practices 23

Element 2: Data Lifecycle Management 2.4 Data Backups and Archiving Guidelines and procedures for protection and long term preservation of data assets. Focus: Extent to which active data sets are backed up, and inactive data sets are archived for future use as needed Improvement: Recognize need for regular data backup and archiving and establish policy and procedures Monitor implementation and test results meet business needs Periodically review and improve Benefits of Advancement Move from ad-hoc to routine, reliable backup and archive processes Understand frequency and needs, and establish accountability Reduce risks of data loss due to hardware failures or other sources of data corruption Better meet business needs, ensuring backups and archives serve intended their intended purpose 24

Element 2: Data Lifecycle Management 2.5 Data Change Management Processes to minimize unanticipated downstream impacts of data changes. Focus: Extent to which procedures are in place to manage the changes to data structures in one data set or system that may impact other systems or reports Improvement: Recognize need for standard change management processes and apply to key data elements Provide integrated/automated analysis and tools to support change management Periodically review and improve Benefits of Advancement Avoid unintended consequences of changes to data structures, definitions, unique identifiers, etc. Provide efficient, proactive approaches to anticipate downstream changes Implement changes in controlled, automated, and coordinated manner Avoid creating barriers to data integration 25

Element 2: Data Lifecycle Management 2.6 Data Delivery Delivery of data to users in a variety of convenient, useful and useable forms. Focus: Extent to which data are delivered to end users in convenient forms that are suited to best meet business needs. Improvement: Implement enterprise solutions for data access, reporting, visualization, and analysis Move towards data self-service solutions supported by a flexible architecture Benefits of Advancement Support a wide range of potential data uses Provide delivery formats that best serve users Ensure flexibility to meet changing requirements Squeeze more value out of data investment by promoting use and re-use Reduce time consuming data manipulation and custom report development 26

Element 3: Data Architecture and Integration How data is standardized and integrated to minimize duplication and inconsistencies. Related Sub-Elements 3.1 – Location Referencing 3.2 – Geospatial Data Management 3.3 – Data Consistency and Integration 3.4 – Temporal Data Management 27

Element 3: Data Architecture and Integration 3.1 Location Referencing Common location referencing methods across agency data sets. Focus: Extent to which standardized methods for location referencing (including linear references) are available for road-related datasets Benefits of Advancement Ability to reliably map and efficiently integrate data in different datasets or systems Improve LRS management, quality, usefulness, and adoption Improvement: Develop and implement agency standard LRS Support LRS improvement and integration with quality standards, and processes to propagate changes to other datasets 28

Element 3: Data Architecture and Integration 3.2 Geospatial Data Management Standardize approach to collection and management of geospatial data. Focus: Extent to which there are standard approaches to collect, manage, and integrate spatial data Improvement: Implement agency geospatial data planning processes and integrate it with broader IT and data management planning Provide, implement, and support spatial data collection, storage, management, sharing, and integration standards Benefits of Advancement Increase coordination of GIS with “mainstream” data management functions Ensure GIS is integrated with other agency business data Maximize business value by promoting usefulness of data Reduce need for time consuming, repetitive data integration tasks 29

Element 3: Data Architecture and Integration 3.3 Data Consistency and Integration Standards and practices to ensure use of consistent coding and common linkages so that different data sets can be combined to meet business information needs. Focus: Extent to which database creation and application development are managed to minimize duplication and ensure integration Improvement: Plan for data integration and linkage, identifying and defining key master and reference data, authoritative source systems, etc. Implement and support plans, and routinely assess opportunities to move towards the desired state Benefits of Advancement Improve data integration proactively Establish a clear architecture and vision Minimize data duplication Improve data management efficiency Reduce risk of conflicting data or inconsistent reporting in different systems or from different business units 30

Element 3: Data Architecture and Integration 3.4 Temporal Data Management Standardization of date-time data elements to enable trend analysis and integration across data sets that are collected and updated on varying cycles. Focus: Extent to which standardizing of temporal data elements supports user needs Improvement: Define standards, guidelines, and strategies regarding data and time-related data elements Implement in a manner supporting temporal/trend analysis, snapshots, or integration needs Provide tools to automate processes and streamline implementation of new use cases Benefits of Advancement Support easy conversion between key date/time related standards (e.g. Calendar vs. Fiscal Year) Support snapshots to represent point-intime conditions needed for specific business purposes Streamline integration of data that are collected on varying cycles to support advanced data analytics 31

Element 4: Data Collaboration How collection and management is coordinated with internal and external users. Related Sub-Elements 4.1 – Internal Agency Collaboration 4.2 – External Agency Collaboration 4.3 – Public Data Sharing Policy and Guidance 32

Element 4: Data Collaboration 4.1 Internal Agency Collaboration Collaboration across agency business units to leverage opportunities for efficiencies in data collection and management. Focus: Extent to which there is collaboration and coordination across organizational units for data collection and management Improvement: Recognize needs for collaboration and establish and implement standard processes Apply standards to promote effective collaboration and generate efficiencies Periodically review to integrate new technologies and new data sources Benefits of Advancement Increase coordination and sharing Build data partnerships Pursue new data collection technologies meeting multiple user needs Maximize value of enterprise reporting platforms Reduce duplication and prevent proliferation of overlapping datasets 33

Element 4: Data Collaboration 4.2 External Agency Collaboration Partnerships with external entities to share data and avoid duplication. Focus: Extent to which agency seeks out externally available data to meet data needs Benefits of Advancement Save staff time through standard processes Fulfill data requests more efficiently Provide a richer pool of data at lower cost than possible through internal collection Improvement: Recognize potential value in partnering and proactively identify opportunities Formalize data partnership process and documentation Routinely reassess and improve 34

Element 4: Data Collaboration 4.3 Public Data Sharing Policy and Guidance Policies and guidance for sharing agency data with the general public (e.g. open data portal). Focus: Extent to which public data sharing policy and models are in place and meeting data producer and consumer needs Improvement: Develop and implement data sharing polices, guidance, and models Provide training Gather feedback from data producers, managers, and users to improve Benefits of Advancement Save staff time through standard processes Fulfill data requests more efficiently Meet open data policy goals and/or requirements Expand data value by providing easy data access to agency partners and general public 35

Element 5: Data Quality Management How data quality is defined, validated, measured, and reported. Related Sub-Elements 5.1 – Data Quality Measurement and Reporting 5.2 – Data Quality Assurance and Improvement 36

Element 5: Data Quality 5.1 Data Quality Measurement and Reporting Metrics and reporting to ensure user understanding of current data quality. Focus: Extent to which quality metric are defined and user to inform users about currency, accuracy, coverage, and completeness Improvement: Develop and implement quality measurement and reporting policy and guidance Proactively identify new areas where common data quality metrics would be beneficial Benefits of Advancement Consistently measure and report on data quality Provide a basis for initiating data quality improvement efforts Support informed use of data based on current quality Increase trust in data 37

Element 5: Data Quality 5.2 Data Quality Assurance and Improvement Practices for improving quality of existing data and assuring quality of newly acquired data. Focus: Extent to which a systematic and proactive approach to data quality assurance and improvement is in place Benefits of Advancement Standardize data quality control and assurance processes Increase capabilities to automatically detect quality issues and/or cleanse data Produce reliable information meeting decision-support needs Improvement: Develop and implement quality assurance and improvement policy, and guidance Document data rules Support data quality management with tools and training and periodically review to processes improve 38