A recent project reminded me of the real world scenario where masterdata is never clean. We are being told that the textbox is a free text and city can be a street, district or even province. Of course, this creates a challenging downstream impact for your data analytics.
Masterdata POV (Point of View)
Decide the POV of masterdata. One system masterdata could be another system transactional data and vice versa.
Use a reference point and communicate a common language of masterdata.
Knowing your end game will decide what masterdata to be collected with a relevant POV.
Tips to clean your masterdata
If more than 50% of your masterdata data needs cleansing, it is worthwhile to drop this masterdata,
Know what to clean and not to clean for the sake of cleaning.
A clean masterdata exhibits consistent patterns while an unclean one is a total chaos.
Know your domain well to clean effectively!
Cleaning masterdata is a iterative process. You get better and resilient with practice. A good data sense is also advantageous. Good luck cleaning and may the force be with you!
Have you ever heard of Data Gap? This is a common issue from legacy application and processes. Data Gap are missing data in a required dataset. They are hidden risks to the Organisation and incurred unnecessary cost. In my latest project, data gap is the single biggest contributor in the support ticket. So, what and how can we do to remove Data Gap?
Signs of Data Gap
Increased support tickets with data updates as resolutions
Additional fields in manual process vs existing fields from integration.
Mismatch of data fields between source and target system.
Added requirement of data translation and exception logic used in integration and reporting.
Ways to handle Data Gap
Harmonisation of data for source and target system.
Business process reengineering of existing SOP (Standard Operating Procedures).
Removal of dead fields and update requirements to automate missing data retrieval
Change management of users to identify and manage Data Gap
There are an increasing trend of data analytics as Cloud technologies have enabled cheaper access to obtain data. Many organisations are scrambling to create and hire data analytics resources. As many are aware, data is always a two edge sword. How should you be managing data correctly? I will show you a simple checklist to evaluate your data awareness.
Favourable Data Checklist
Do you have data quality best practices in place?
Do you have data sensitive resources?
Do you conduct data historical review consistently to review trends and business performance?
Unfavourable Data Checklist
Data is as good as what you can record.
Dirty data is as good as no data.
Data have no meaning when the blind is leading the analysis.
In short, do not jump into the data bandwagon blindly. If you are misleaded by data, you are worst than having no data. Always implement good processes and quality practices to ensure good data quality. Data analytics can only be amplified and fully utilised with good business and IT alignment.
Data is becoming a commodity as storage become cheaper and cheaper. Many information are now digitised and uploaded to Cloud. Many social media are encouraging content creation with various monetisation incentives. If you own the data, how do you sell it?
Ownership of Data
The ownership of data is something of a dispute as there is no clear legal stand as it differs from country to country. If data is really yours and transferable, you shall be able move it from place to place like physical thing. Unfortunately, data is intangible and the true owner often belongs to the platform you created. Common platforms are social media who harvest your data for their commercial usage. This is with the fact that most platforms provided free services for “acquiring” your data with very fine print of consent.
Data Broker and Data Banking
Since it is established that data mainly do not belong to you, how do you secure your data privacy and rights? Data Broker and Data Banking will emerge in future. Like currency, data is a value of trade and can be transferred and stored. These entities will act as intermediary to help you secure data. Imagine a futuristic version of your Cloud Drive that can seamlessly extract and secured your social media data and transfer anywhere.
A current version of Data Marketplace is the stock market. When there are Data Broker and Data Banking, suppliers and consumers will come together in a Data Marketplace. This is place where Data can be bought and sold. This is not new as many informal channels existed for such transactions. We will be looking forward to such future where data owners are can sell and be compensated with their data.