The Impact of Audit Technology on Audit Outcomes:

The Impact of Audit Technology on Audit Outcomes: Technology-Based Audit Techniques’ Impact on Internal Auditing Univ.-Prof. Dr. Marc Eulerich, CIA University Duisburg-Essen 28th of July 2021

M. Eulerich A. Masli J. Pickerd Marc Eulerich, Adi Masli, Jeffrey S. Pickerd and David A. Wood 30.06.23 D.A.Wood

Introduction “to sustain and build on our successes of the past two decades, internal auditors will need to pivot yet again to address the changing needs driven by high-tech disruptions that fundamentally impact how work gets done. To successfully adapt, internal auditors will need to embrace technology like never before.” (2019) Audit technology is becoming more important for auditors to embrace! Richard Chambers, President of the IIA Protiviti reveals that 3 out of 4 Internal Audit Functions worldwide are undertaking some form of digital transformation initiatives (Protiviti 2019). Internal auditing is facing a paradigm shift that requires an adaptation of current developments in information technology and data analytics (e.g. Chan & Vasarhelyi, 2011). Nevertheless, research has provided relatively little support that auditors’ use of technology enhances important audit outcomes. 30.06.23 Technology-based Audit Techniques for Internal Auditing 3

Prior Literature Multiple benefits of TBAT/Data Analytics for the IAF have been broadly discussed by practitioners, academics and the IIA (e.g. PWC, 2018; KPMG, 2018; Singh et al., 2017; Coderre, 2015; Koskivaara, 2007; Trompeter & Wright, 2010; Yoon et al., 2015; Cardinaels/Eulerich/Sofla, 2021): Enhancement of the timeliness of information Support of the audit committee/ Meeting management’s regulatory objectives Reducing the cost of assurance. Promoting a more efficient use of resources. Increased reliability of audit outcomes / generated audit evidence, especially since audits with TBAT are able to cover the entire data set and not just a specific (small) sample of the whole population (e.g. Brown et al., 2007; Chan and Vasarhelyi, 2011; Kearns et al., 2011, Jans & Hosseinpour, 2018) There have been multiple calls for research on how audit technology influences audit outcomes (e.g. Christ, Eulerich, Krane and Wood, 2021; Alles and Gray 2016; Earley 2015; Wang and Cuthbertson 2015; Janvrin and Wood 2016; Moffitt, Richardson, Snow, Wiesner, and Wood 2016; Austin, Carpenter, Christ, and Nielsen 2020) . 30.06.23 Technology-based Audit Techniques for Internal Auditing 4

Hypotheses and Research Question So it is obvious Technology improves Audits.?!?! Prior Research found mixed results when it comes to the outcome Technology (e.g. a software solution) is not automatically helpful, so it depends also on the specific auditor and the audit It is still not clear what the „real effects“ on efficiency and effectiveness are There is no evidence about differences/similarities when it comes to the use of technology in different steps of the audit process H1: The use of technology-based audit techniques (TBATs) and the maturity of the data analytics usage are associated with increased auditor effectiveness. H2: The use of technology-based audit techniques (TBATs) and the maturity of the data analytics are associated with an increase in the efficiency of the audit. RQ1: Where in the phases of the audit are the use of technology-based audit techniques (TBATs) and the maturity of the data analytics usage helpful in improving the effectiveness and efficiency of the audit? 30.06.23 Technology-based Audit Techniques for Internal Auditing 5

Method and Analytical Approach Data and sample selection: Data Collection: at 2 large IIA conferences in May and November 2019 Participants: 297 different internal auditors. No. of Audits covered: 531 different audits. Pilot Test: With 2 internal auditors and the German IIA. Questionnaire: 19 questions Analysis: Multiple Regression with two measures for effectiveness and one measure for efficiency! 30.06.23 Technology-based Audit Techniques for Internal Auditing 6

Method and Analytical Approach Empirical Approach: Audit Effectiveness: Audit Effective b0 b1 Overall TBAT (Overall DAT) bj Controls Engagement e Audit Effective b0 b1 TBAT (DAT)-at various stages bj Controls Engagement e DV is measured as No. of Recommendation / Risk Factors Found Audit Efficiency: Auditing Days b0 b1 Overall TBAT(DAT) bj Controls Engagement e Auditing Days h0 h1 TBAT (DAT)-at various stages hj Controls Engagement e 30.06.23 Technology-based Audit Techniques for Internal Auditing 7

Method and Analytical Approach: Variables Dependent Variables Auditing Days Risk Factors Found Recommendations Main Independent Variables TBAT—Planning Schedule Number of days spent on the audit (days in preparation fieldwork reporting) Number of significant risk factors found as a result of the audit divided by total number of auditing days Number of Recommendations produced by Internal Audit TBAT—Planning Audit Process TBAT—Gathering Evidence TBAT—Analyses TBAT—Reporting Respondent’s rating of the audit engagement’s use of TBATs for planning the audit schedule (1 Very Low to 5 Very High) for planning the audit processes and procedures for gathering audit evidence for analyses of audit findings for reporting of audit findings TBAT—Follow Up for following up on audit findings Additional Dummys for - Audit Object (Subsidiary, Plant, Project, Process) - Focus (Financial Audit, Operational audit etc.) - Assurance Focus Control Variables for: - Team Size - Audit Size - Risk/Performance of the Auditee - 30.06.23 Technology-based Audit Techniques for Internal Auditing 8

Results: Overall Use of TBAT and Data Analytics on Internal Audit Outcomes Overall Use of TBAT and Data Analytics on Internal Audit Outcomes (1) (2) (3) (4) (5) (6) Risk Factors Recommend. Auditing Days Risk Factors Recommend. Auditing Days coef/p-value coef/p-value coef/p-value coef/p-value coef/p-value coef/p-value Overall TBAT ( /- ) Overall DAT ( /-) Subsidiary Plant Project Process Financial Focus Operational Focus Compliance Focus Managerial Focus IT Focus Assurance Team Size Audit Size Risk before Performance before Report to Board/AC Publicly Listed 0.045*** 0.041** -5.493*** 0.053 0.077 0.015 -0.037 0.045 -0.028 0.030 -0.028 -0.029 0.018 -0.001 -0.032 0.023 -0.002 -0.061* -0.028 0.021 0.199* -0.051 -0.137** 0.119** -0.003 0.143** 0.037 -0.004 0.046 -0.005 -0.047 0.058** 0.007 0.003 -0.043 10.038 7.810 21.031** 9.958* -2.012 0.368 -3.695 4.054 0.664 -1.752 9.355** 9.248*** 2.139 0.857 2.479 -2.718 0.037*** 0.054 0.071 0.007 -0.039 0.052 -0.024 0.029 -0.034 -0.018 0.030 -0.001 -0.029 0.019 -0.002 -0.055* -0.024 0.029* 0.028 0.201* -0.059 -0.137** 0.127** 0.011 0.139** 0.030 0.003 0.066 -0.005 -0.048* 0.053* 0.008 0.010 -0.040 -2.507** 9.531 6.516 21.425** 10.457* -3.358 -0.988 -3.983 4.489 -0.645 -2.407 9.328** 8.820*** 2.368 0.728 1.554 -2.776 Finance Services Industry -0.110*** -0.258*** 6.891** -0.102*** -0.249*** 6.881** 0.166 0.316* -18.914 0.186 0.349** -25.833 351 409 434 354 410 435 Intercept Number of observations Adjusted R2 0.081 0.167 0.288 0.075 0.159 0.267 The p-values are in parentheses. P-values are two-tailed, except for the hypothesized effects in bold (one-tailed). *** p 0.01, ** p 0.05, * p 0.1 See Appendix A for variable definitions. 30.06.23 Technology-based Audit Techniques for Internal Auditing 9

Results: Risk Factors Found by Internal Audit Risk Factors Found by Internal Audit Panel A. Usage of TBAT TBAT—Planning Schedule TBAT—Planning Audit Process TBAT—Gathering Evidence TBAT—Analyses TBAT—Reporting (1) coef/pvalue 0.014* (0.098) (2) coef/pvalue (3) coef/pvalue (4) coef/pvalue (5) coef/pvalue (6) coef/pvalue 0.036*** (0.002) 0.017** (0.044) 0.023** (0.014) 0.046*** (0.000) TBAT—Follow Up 0.036*** (0.001) Control Variables Included YES YES YES YES YES YES Number of observations 360 359 361 359 364 359 Adjusted R2 0.056 0.081 0.059 0.067 0.101 0.091 The p-values (one-tailed) are in parentheses. *** p 0.01, ** p 0.05, * p 0.1 See Appendix A for variable definitions. 30.06.23 Technology-based Audit Techniques for Internal Auditing 10

Results: Recommendations by internal audit TABLE 5 Recommendations by internal audit Panel A: Usage of TBAT TBAT—Planning Schedule (1) coef/pvalue 0.018 (0.157) TBAT—Planning Audit Process (2) coef/pvalue (3) coef/pvalue (4) coef/pvalue (5) coef/pvalue 0.029** (0.044) TBAT—Gathering Evidence 0.020 (0.102) TBAT—Analyses 0.015 (0.152) TBAT—Reporting 0.045*** (0.007) TBAT—Follow Up Control Variables Included Number of observations Adjusted R2 30.06.23 (6) coef/pvalue YES 418 0.159 YES 417 0.167 YES 419 0.161 YES 417 0.160 YES 422 0.176 0.034** (0.010) YES 417 0.169 Technology-based Audit Techniques for Internal Auditing 11

Results: Recommendations by internal audit TABLE 6 Auditing days worked by internal audit Panel A: Usage of TBAT (1) coef/pvalue TBAT—Planning Schedule -3.279*** (0.000) TBAT—Planning Audit Process (2) coef/pvalue (3) coef/pvalue (4) coef/pvalue (5) coef/pvalue -3.490*** (0.001) TBAT—Gathering Evidence -2.424*** (0.004) TBAT—Analyses -3.747*** (0.000) TBAT—Reporting -5.199*** (0.000) TBAT—Follow Up Control Variables Included Number of observations Adjusted R2 30.06.23 (6) coef/pvalue YES 443 0.295 YES 442 0.295 YES 444 0.289 YES 442 0.302 YES 447 0.313 -3.762*** (0.000) YES 442 0.284 Technology-based Audit Techniques for Internal Auditing 12

Discussion, Conclusion and Limitations Discussion & Conclusion: TBATs / Mature Data Analytics have: a positive influence on # of Risk Factors / Findings found a positive influence on # of Recommendations made a negative influence on # of Days spent for the engagement In other words, technology and data analytics increase the effectiveness and efficiency of an audit. Looking at the different audit phases: TBAT (DA) in all (except of one) phases increase the # risk factors. TBAT (DA) in all (except of one) phases descrease the # of days spent. TBAT and DA have a different impact on the audit depending on the phase of the audit process. One of our main limitations: Our study is based on self-reporting and likert scales, not measuring the ”real” use of technology! 30.06.23 Technology-based Audit Techniques for Internal Auditing 13

Chair of Internal Auditing Univ.-Prof. Dr. Marc Eulerich, CIA [email protected] 49 203/379-2849 www.ircg.msm.uni-due.de Universität Duisburg-Essen Mercator School of Management Lehrstuhl für Interne Revision Lotharstraße 65, D-47057 Duisburg Marc Eulerich @MarcEulerich www.internalauditing.de Prof. Dr. Marc Eulerich Working Papers at SSRN.com ResearchGate: Marc Eulerich

IIA’s Definition of relevant technology Technology-based Audit Techniques: Any automated audit tool, such as generalized audit software, test data generators, computerized audit programs, specialized audit utilities, and computer- assisted audit techniques (CAATs). We use the terms technology-based audit techniques and CAATs analogously.” Continuous Auditing — the combination of technology- enabled ongoing risk and control assessments. CA is designed to enable the internal auditor to report on subject matter within a much shorter timeframe than under the traditional retrospective approach. Continuous Monitoring — a management process that monitors on an ongoing basis whether internal controls are operating effectively (PA 2320-4: Continuous Assurance). 30.06.23 Technology-based Audit Techniques for Internal Auditing 15

Prior Literature The potential benefits of TBAT for the IAF have been broadly voiced by practitioners and academics (e.g. PWC, 2018; KPMG, 2018; Singh et al., 2017; Coderre, 2015; Koskivaara, 2007; Trompeter & Wright, 2010; Yoon et al., 2015). Benefits of continuous auditing from an academic perspective: Enhancement of the timeliness of information Support of the audit committee Meeting management’s regulatory objectives Increased reliability of audit outcomes / generated audit evidence, especially since audits with TBAT are able to cover the entire data set and not just a specific (small) sample of the whole population (e.g. Brown et al., 2007; Chan and Vasarhelyi, 2011; Kearns et al., 2011, Jans & Hosseinpour, 2018) Benefits of CA from IIA’s perspective: Promoting a more efficient use of organizational resources. Reducing the cost of assessing and providing assurance over the adequacy of internal controls. Providing an ongoing evaluation of risks and controls. Providing timely reporting of gaps and weaknesses, enhancing the opportunity for prompt corrective action. (e.g. GTAG 3) 30.06.23 Technology-based Audit Techniques for Internal Auditing 16

Supplemental analyses: Internal auditors’ perceptions of the value of TBAT in their job function Panel A: Perceptions of internal auditors about the benefits of TBAT Outcome Obs Identify new risks 258 Identify significant risks 260 Indentify internal control weakness 259 Preparing for audit 259 Gathering evidence 257 Making IA work easier 260 Increase job satisfaction 260 Improve effectiveness of IA work 259 Improve efficiency of IA work 261 Mean 3.21 3.22 3.56 3.46 3.49 3.31 3.14 3.86 3.92 p-value for Mean 3 (neutral) 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 Panel B: Comparing responses from companies that have high vs. low usage of TBAT Outcome Identify new risks Identify significant risks Indentify internal control weakness Preparing for audit Gathering evidence Making IA work easier Increase job satisfaction Improve effectiveness of IA work Improve efficiency of IA work High TBAT 3.39 3.44 3.81 3.7 3.74 3.49 3.22 4.11 4.21 Low TBAT 2.88 2.71 3.19 3.08 3.03 2.95 2.95 3.47 3.34 Difference 0.51 0.73 0.62 0.62 0.71 0.54 0.27 0.64 0.87 p-value for Difference 0.01 0.01 0.01 0.01 0.01 0.01 0.11 0.01 0.01

Supplemental analyses: CAEs’ perceptions of the value of TBAT in internal audit job function Panel A: Perceptions of CAEs about the benefits of TBAT Outcome Identify new risks Identify significant risks Obs 313 312 Mean 3.13 3.12 Indentify internal control weakness Preparing for audit Gathering evidence Making IA work easier Increase job satisfaction Improve effectiveness of IA work Improve efficiency of IA work 316 314 314 311 311 315 315 3.39 3.22 3.37 3.21 3.09 3.61 3.54 0.01 0.01 0.01 0.01 0.13 0.01 0.01 High TBAT 3.53 3.55 Low TBAT 2.88 2.88 Difference 0.65 0.67 p-value for Difference 0.01 0.01 3.78 3.61 3.83 3.57 3.43 4.16 3.99 3.08 2.99 3.05 2.89 2.80 3.25 3.19 0.70 0.62 0.78 0.68 0.63 0.91 0.80 0.01 0.01 0.01 0.01 0.01 0.01 0.01 p-value for Mean 3 (neutral) 0.03 0.05 Panel B: Comparing responses from companies that have high vs. low usage of TBAT Outcome Identify new risks Identify significant risks Indentify internal control weakness Preparing for audit Gathering evidence Making IA work easier Increase job satisfaction Improve effectiveness of IA work Improve efficiency of IA work

Regression Results: Stakeholder Relianceby Audited Unit coef/pcoef/pcoef/pcoef/pvalue value value value Technology Audit Techniques Technology Audit Techniques 0.137** (0.012) - Cont. Auditing 0.109** Technology Audit Techniques - Cont. Auditing Technology Audit Techniques (0.016) Technology Audit Techniques 0.091** - Cont. Monitoring Technology Audit Techniques - Cont. Monitoring (0.046) Maturity Data Analysis Technologies 0.023 Maturity Data Analysis Technologies Relianceby Top Management coef/pcoef/pcoef/pcoef/pvalue value value value Technology Audit Techniques 0.149*** Technology Audit Techniques (0.007) Technology Audit Techniques - Cont. Auditing Technology Audit0.127*** Techniques - Cont. Auditing (0.005) Technology Audit Techniques - Cont. Monitoring Technology Audit Techniques - Cont. Monitoring Maturity Data Analysis Technologies Relianceby Supervisory Board coef/pcoef/pcoef/pcoef/pvalue value value value 0.200*** (0.003) 0.212*** (0.000) 0.167*** (0.003) 0.120 (0.144) coef/pvalue 0.075 (0.308) Relianceby External Auditor coef/pcoef/pcoef/pvalue value value 0.155*** (0.006) 0.109** 0.114* (0.057) (0.021) Maturity Data Analysis Technologies 0.118* (0.059) Technology Audit Techniques Technology Audit Techniques - Cont. Auditing Technology Audit Techniques - Cont. Monitoring coef/pvalue 0.193*** (0.001) J ob Resources coef/pcoef/pvalue value 0.038 (0.639) coef/pvalue 0.116** (0.015) 0.046 (0.338) 30.06.23 Maturity DataTechniques Analysis Technologies Technology-based Audit for Internal Auditing 0.104 19 (0.162)