Data can never be perfect…

February 29, 2008 § Leave a comment

[Re-posted on my current blog: Data can never be perfect… ]

This is a re-post of a comment I made on Doug Henschen’s article about data governance and the suprime crisis.

Doug’s position (quoting from a lot of data governance vendors) is: “A first step toward avoiding such calamities… is an integrated, overarching data governance program that addresses data security, data privacy and data quality so that risks can be better understood and outcomes anticipated.”

Basically, if the banks had better data they would have made better decisions and not got themselves into this mess.

The problem is not a lack of governance but an unshakeable belief in the data and risk models. Interested readers should look at “Did Black-Scholes Cause the Housing Bubble?” in Portfolio.

I’m a data guy, but every executive needs to understand that data is merely a map and the map is not the territory. If an explorer has a map that does not match the territory they can see, they would do well to question the map, rather than ask the territory to change.

The credit score is simply another map. There is evidence that they were significantly weakened by new financial products over the last 7 years. Again, see “Credit Scores: Not-So-Magic Numbers” for details.

Data quality, data governance, etc. are all **super** important. However, as data professionals we need to build systems that incorporate common sense, human based checks and balances. Trusting too much in software will eventually get you fired or indicted for criminal negligence.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

What’s this?

You are currently reading Data can never be perfect… at Perfect Information.


%d bloggers like this: