Data Quality

Data, Data, & More Data

As discussed in the prior post, we live in a world where an enormous amount of data is being generated all the time. With the constant creation of data, companies are utilizing it to make informed business decisions and more. It isn't enough to have data, what is important is to have quality data to make useful decisions or the decisions made with poor data are not useful to a company.

 

Data Quality

For any industry, the quality of data impacts every decision made along the spectrum of their business processes. Therefore, the demand for accurate and reliable data has never been more important. There are techniques to analyze and assess data to determine the quality of data stored by a company. Some of those techniques consist of data profiling, integrity checks, and business checks.

There are also properties of quality that can affect analyses or database modeling. Each property, as seen in Figure 1, helps a company determine the state their data is in and what might need to be improved before utilizing. Companies may find it hard to implement a data quality initiative, but the potential benefits of having quality data definitely outweigh the hassle. In the article "3 Reasons Why Data Quality Should Be Your Top Priority This Year," states cost, compliance, and decision making as the three reasons to implement a data quality initiative. The cost of poor quality data may cost organizations six hundred billion dollars annually. Poor data quality is also the leading cause of the failure of IT projects as well as one of the driving factors behind customer attrition. The second reason for having quality data is compliance, if a company is unable to access reliable data or missing data it may cause compliance violations which produces reputation issues. The third reason which has already been stated many times is decision making. Having good data quality means having accurate and timely information as well as the ability to prioritize and ensure the best use of resources. If a company has poor data quality in this day and age of big data there will be much to lose.

I can attest to the importance of quality data because the company I work for began reviewing our data that would be loaded into our data warehouse and it was evident that we had some issues. In order to obtain quality data, we began by preventing, detecting, and repairing within our information system that stores our data which is exactly what "Data Quality and Record Linkage Techniques" book discusses to obtain high-quality data. We have continued with these techniques and have been able to overcome the issues and make adjustments to our data entry processes in order to better our data. By making these improvements, the decisions we make are more informed which in turn has saved us on costs, helped us determine the projects to prioritize, and ensured compliance within the industry. For any company that takes advantage of taking the time to collect and store quality data will see profits, customer satisfaction, and a good reputation that will attract new customers.

Resources

1. Eichhorn, Gadi. 2014 February 19. "3 Reasons Why Data Quality Should Be Your Top Priority This Year." Realise Data Systems. http://www.realisedatasystems.com/3-reasons-why-data-quality-should-be-your-top-priority-this-year/
2. Herzog, Th.N., Scheuren, F.J., Winkler, W.E. 2007. "Data Quality and Record Linkage Techniques." https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CDAQFjAAahUKEwiG8aboxJPJAhUT-GMKHfcCANw&url=http%3A%2F%2Fwww.springer.com%2Fcda%2Fcontent%2Fdocument%2Fcda_downloaddocument%2F9780387695020-c1.pdf%3FSGWID%3D0-0-45-381701-p173712918&usg=AFQjCNEFM9g86heuzgYu9MNciSZkjkkf9A 
3. Ram, Sudha. 2013. "Data Quality Analysis." OIS-MIS587, slides 1-18.