What can we learn from the way that the Queensland Audit Office uses data analytics in their audits? Dyfrig Williams had an early morning phone conversation to find out.

As I previously mentioned on my post on the Wales’ Audit Office’s Cutting Edge Audit project, I’m looking to identify ways of making better use of data and technology in order to transform the way that we work. I’d come across this article on how Queensland Audit Office are using audit data analytics to gain greater insight, and through my well-travelled colleague Tom Haslam I managed to organise an early morning conversation between myself and my colleagues (Steve Lisle and Nigel Blewitt) and Daniele Bird, the Assistant Auditor-General for Performance Audit in Queensland, Ben Jiang the Assistant Director for Audit Analytics and David Toma, their Assistant Director for Performance Audit Services.

What is the Queensland Audit Office doing?

Back in 2014, they decided to look at opportunities to use more sophisticated data visualisation tools in order to get a better insight into their audited bodies. They trialled and tested Qlikview, which can be accessed via the web or desktop software. They set up a data analytics team in 2015, all of whom were former auditors branching out into the world of data analytics. The team undertook initial training, but their development from there was centred on self-learning. The team first planned to work on Performance Audit. However the focus moved to Financial Audit, as the recurring datasets meant they could work more efficiently and effectively, compared to the very different types of datasets that are needed for each performance study.

The importance of relationships

Our seminars on early closure of accounts in 2015 and 2016 have made it clear to me that both the auditor and the audited body have to work together closely in order for audit to work more effectively. Whilst the Queensland Audit Office had built processes and automatic systems to collect data, they still faced questions from clients about security and what they were going to do with the data. They built an internal Frequently Asked Questions page so that audit team leaders could re-assure organisations and build trust, and they also took the burden of change away from the audited bodies by enabling them to dump the unformatted data with the office. The analytics team did the work to transform and clean up the data, and they’re now able to offer unique insights as they gather data from such a wide range of bodies. They can quickly identify and access an overview of issues, as well as benchmark clients’ performance and show how they compare to their peers (which you can see in the slide below). They also use the tools to undertake exercises like Benford’s Analysis, which clearly identifies outliers. This is particularly useful in identifying fraud.

[caption id="attachment_3358" align="alignnone" width="1022"] How Queensland Audit Office gains insight to the value of bad debt[/caption]

Performance Audit

The Data Analytics Team is now also supporting Performance Audit staff. The team is using text mining, for example to see what teachers are identifying as development goals. This has enabled them to produce Wordclouds and text summaries of the top 10 statements. They also use these tools to sweep the text of Hansards.

The team is also looking at how it can help staff to choose performance audit topics. It’s running sentiment analysis tools on Twitter feeds in order to support the strategic planning process. This helps them to understand the issues that are out there in the wider world and what people are saying about them. Although the technology has been built for Twitter, it could also be used for networks like LinkedIn, which would enable auditors to access different users and perspectives. They’ve also built social media analysis tools that rank tweets. It grades them and enables auditors to focus resources on important messages.

Creating a positive working environment

Daniele, Ben and David acknowledged the importance of leadership from the top in creating the right environment for the work of the Data Analysis Team to flourish. They’ve received high level support from the previous Auditor General and the current Acting Auditor General. This is something that we’ve seen from our own Auditor General in the development of the Cutting Edge Audit project, and hopefully the findings from our own work can help to provide a base for us to develop our own ways of working. It was really interesting to hear how the organisation is continuously looking to develop their work to build on current practices. They’re currently looking at whether moving from Qlikview to Qlik Sense could provide further advantages.

The work of the Data Analytics Team has achieved efficiencies, mainly around saving time that can be reinvested in future work, but they’re also looking to save more to do more. The rapidly changing environment and limited public finances presents a challenge for every public service. Queensland Audit Office have faced that challenge head on and are working in a different way to provide a better service. Not only that, but their staff’s work is now more meaningful and interesting, and auditors are further developing their critical thinking skills. I’ll be thinking about our Good Practice Exchange principles as we reflect on our conversation with the Queensland Audit Office, so that we think about how we might adapt this approach in order to ensure that our work, like the services we audit, are delivering the best possible outcomes for the people of Wales.