The statistician George Box claimed 30 years ago "All models are wrong, but some are useful."
The enterprise approach to data analysis is broken. Over the years, I am increasingly hearing from business and IT leaders that their enterprise data warehouses, data marts and other collection of data repositories and analytic systems are not providing the value they should.
An overwhelmingly evident reason for the disappointment is that almost all data warehousing systems as we know them require the “modeling” of data. It is an act to define how to fit data into a relational database system and requires such enormous amounts of organizational buy-in with inherent compromises, that it always promises to succeed for some but guarantees to fail for others.
While George Box was referring to statistical models, the principles of relational data modeling are the same. A model is an assumption of data behavior. It takes time to agree to what the behavior is. As the business environment changes so does the data behavior, which takes more time to resolve.
Most importantly, data has value at the time of creation and that value decays over time. I’d like to think the value of data decays exponentially and that the rate of decay is increasing. Enterprises which can capture, process, store, analyze and act upon this data in its highest moment of value have a distinct competitive advantage.
Half Life of Data is a series of thoughts to recognize that organizations have to start moving past 40 year old techniques of data management and analytics if they have to compete in the digital economy. Every so often we get to witness a profound change in the way things are done. Someone conceives a radically different mousetrap which actually works better. I believe that “data mousetrap” has arrived.
