April 26, 2012 By paul priestly
Data Quality Blog – A Never Ending Story
Where’s a spell-checker when you need it?
I was recently invited to provide one or more data quality anecdotes for a chance to win some glorious prize. I was hooked, more for the opportunity to contribute an anecdote and less for the prize. I was also amused that there’s still such widespread appeal for these embarrassing events and, more worryingly, that there are still plenty of embarrassing events happening.
I decided that a blog focussing on data quality stories would provide some insight into how data quality issues can impact business and why it’s important we don’t just accept poor data but we actively work to fix the problems (and stop these lifelong embarrassing moments).
This blog is the first in a series that will focus in a light-hearted way on DQ anecdotes and stories, to examine how such errors occur and what can be done to prevent them in the future.
To set the scene let’s look at what the term Data Quality means. My colleague Veronica Coyne recently identified the dimensions of Data Quality, in her blog ‘What Does Data Quality Mean?’ (http://altis.com.au/what-does-data-quality-mean/) Veronica agreed with the importance of the dimensions identified by Larry English (http://www.infoimpact.com/) these are:
- Existence and Completeness
- Validity (conforms to business rules)
- Accuracy (correctly represents characteristic of real world object or event)
- Precision (values are correct to the right level of detail)
- Non-duplication (each record appears only once in a data store)
- Equivalence (of redundant or distributed data)
- Timeliness (for all knowledge workers)
- Currency (data is up-to-date)
- Presentation Clarity (data is presented in a way that clearly conveys the truth)
- Definition Conformance (the value of an attribute is consistent with the attribute’s definition, e.g in type and format).
- Plus her own useful additions:
- Auditable and
The purpose of Veronica’s message was that measuring data is necessary to managing it and for that you need to have clearly defined measures or in this case data quality dimensions. My purpose is to look at what happens when you get it wrong and then focus on the next step “What do we do about it?”
These blogs are to help highlight the DQ dimension(s) embedded and show their potential impact on your business.
A tip for the uninitiated here: A DQ occurrence is hardly ever likely to have a single outcome. The chances are you get it wrong once you get it wrong over and over.
Data Quality Felix
Here’s my first example – ok it’s not real, it’s a TV commercial, but you can easily see how real this example could be:
Remember this ad from a couple of years back? A certain large bank had Tony Barber accompanied by an obvious senior manager cum trophy holder; photographers, media hack and cheer-squad. They knock on the door of some house and tell the unsuspecting door-opener that he was being presented with an award because the bank was the umpteenth winner of Home Lender of the Year. The joke was they had the wrong person – the guy they want is next door. Poor data quality – you betcha! Did the management at the bank know they were making a joke out of poor data quality? Seems to have eluded their management – or did it?
I wondered how the experience of Altis consultants could contribute a number of real-life anecdotes. When you work in the DW/BI space long enough you can’t help but capture interesting stories along the way. That said, you may only need to be a new entrant to pick up a story or two, because the issue’s not going away. It’s almost as if (poor) data quality is an implied cost of doing business and organisations have given up trying to address it and simply, even happily, accept the situation.
To set the scene, here are some references to locations where you can see various data quality results posted.
Now it’s over to our Altis story-tellers to come forth with their tales. Stay tuned.
About the author:
Paul Priestly is a Senior Consultant with Altis Consulting and has over 20 years’ experience in Information Management and IT project management within public and private sector organisations nationally and internationally. Paul specialises in strategic initiatives for DW/BI, data quality, MDM and data security.