S-DWH Manual Gary Brown, ONS

S-DWH Manual Gary Brown, ONS

Overview Prologue Props Actors Stage Finale Epilogue

Prologue ESSNet on S-DWH ran from 04/10/2010 to 03/10/2013 (Netherlands, Estonia, Italy, Lithuania, Portugal, Sweden, UK) CoC on S-DWH ran from 04/10/2013 to 04/10/2014 (Netherlands, Estonia, Italy, Lithuania, Portugal, Finland, UK) CoE on S-DWH running since 05/10/2014 (Netherlands, Estonia, Italy, Lithuania, Portugal, Finland, UK)

Props ESSNet on S-DWH produced 41 reports, including: – Handbook – WP reports (architecture, metadata and methodology) CoC & CoE on S-DWH produced 15 reports, including: – Glossary – Draft Manual (not on CROS-portal)

Actors Five main acts o o o o o Introduction – Antonio Architecture – Sonia, Valerij Metadata – Tauno, Colin Methodology – Gary New chapter – Antti . the narrator – Harry

Stage The state of (the) play o o o o o Introduction –original Architecture – updated Metadata – updated Methodology – updated New chapter – updated

Finale A comprehensive, weighty ( 150 pages), user manual for an S-DWH

Finale A comprehensive, weighty ( 150 pages), user manual for an S-DWH S-DWH

Finale A comprehensive, weighty ( 150 pages), user manual for an S-DWH o aligned with GSBPM, GSIM, SDMX . GSDWH (one day!) o positioned firmly in NSI context due to the practical experience of the authors . not external consultants o to be continued . during the remaining CoG contract(s), and any additional FP funding, the manual will be constantly updated with new developments and in response to user feedback

Epilogue Difficulty working at desks across NSIs & meeting every 3 months Much easier when we get together eg Newport – Antti & Gary splitting the methodology chapter, Lisbon – project team updating the template Still work to do – today! 1. Naming the new chapter – write name on sheet

“HOW” (OPERATING INSTRUCTIONS) NEW NAME: Glue to link architecture, metadata, data & methods.

Epilogue Difficulty working at desks across NSIs & meeting every 3 months Much easier when we get together eg Newport – me and Antti splitting the methodology chapter, Lisbon – project team updating the template Still work to do – today! 1. Naming the new chapter – write name on sheet 2. Restructuring the manual – re-order named sheets

“HO W” OD H T ME GY O L O RE U T TEC I H RC A T I (OP ER ATI NG ARCHITECTURE METADATA ME TAD ATA INS TRU CTI ON S)

Questions?

METHODOLOGY HOW Data cleaning Editing Validation Imputation Data linkage Linkage methods Data linkage processing Estimation Single source estimation Combined source estimation Outliers Further estimation Statistical disclosure control ESSnet on Statistical Disclosure Control ESSnet on Data Integration Methodology of Modern Business Statistics (Memobust) Revisions Current state and pre-conditions Methodological description of the statistical data process Quality requirements for the statistical data IT tools Requirements for the metadata Building blocks – The input datasets Use of administrative data sources The Business Register and the statistical DWH Statistical units and population Business processes of the layerd S-DWH Functionalities Management processes of the S-DWH Type of Analysts Data linking process Correcting information in the population frame and feedback to SBR Functional Architecture FD Strategic functionalities FD Operational Functionalities IT ARCHITECTURE METADATA S-DWH Architecture S-DWH Business Architecture S-DWH Information Systems Architecture Metadata-driven system Layered approach of a full active S-DWH S-DWH Technology Architecture Using SDMX in Statistical Data Warehouse What is Statistical Data and Metadata eXchange (SDMX)? SDMX and the GSBPM Using SDMX in different layers Data centric workflows Possible work flow scenarios Modular approach First step “Framework” Metadata and data basic definitions Metadata category dimensions Metadata subsets Metadata structures Business Architecture: metadata Metadata of the S-DWH layers Metadata model Metadata model, general references Mapping Metadata Generic Models on Metadata subset Metadata System Metadata functionality groups Metadata functionalities by layers Metadata and SDMX The SDMX standard Structural metadata Reference Metadata Content Oriented Guidelines SDMX metadata within the S-DWH layers