March 7, 2012 4:23 PM
Data economics
A number of commentators, including Accenture,
rightly point out that the advent of so-called "Big Data" technologies
doesn’t change the need for our trusty relational databases and our
well-curated data warehouses.
Quite the contrary - the increasing availability and usability of unstructured (sensor, text, audio, image, etc.) data makes the enterprise’s existing investments in managing its structured, typically transactional data even more valuable and will drive continued investments in those platforms.
Now that we all agree that enterprise structured data and newer sources of data, that are often less structured, will coexist and be integrated, we can focus on the real disruptive innovation that these big data technologies share: linear scalability.
While most of the talk around big data technologies focuses on
the "big", that focus distracts us from the key disruptive innovation:
linear scalability. With Hadoop, Cassandra, recently Amazon’s DynamoDB,
and many other big data technologies, the first terabyte of storage and
processing power costs the same as the last. Therefore, the cost of
storing and using data increases as a linear function of the quantity
and the number of uses of the data.
Most (but importantly, not all) relational databases scale geometrically, which is to say the last terabyte of processing and storage capacity is exponentially more expensive than the first. Vendors like Greenplum and Asterdata are bringing linear scalability to the relational world, and Microsoft is bringing it to SQL Server on Azure.
But the true disruption really isn’t about database vendors’ market share - the pie is getting bigger, people. From a data management perspective, a linear scaling world couldn’t be more different than a geometrically scaling one. In a geometrically scaling world, you have to prioritise the use of your data resources and only the most valuable applications and reports get access to the operational and enterprise data architectures that support them. If you have ever witnessed (or been) an analytics person asking a DBA about accessing data, you understand the implication of this - ad hoc exploration of data to inform strategic decisions is painful because direct access is impossible and getting exports into analytical data marts is slow and discouraging.
Linear scalability turns data management from an art of managing
scarcity to a science of maximising use. The linear-scaling world lets
you encourage the use of data. Think about that in the context of our
typical conversation with the DBA, and you will begin to realise how
revolutionary the notion of promoting data’s use in decision making
really is.
Additional use of data - by granting additional users’ access,
integrating additional applications, or providing value-added data
services - can be accommodated by simply adding proportionally more
commodity nodes. The incremental cost of the additional use doesn’t
increase, so you can keep adding users, and as long as they meet a
minimal threshold of value, they are worthwhile.
That means every organisation now has the technical opportunity and business imperative to embark on a continual journey of implementing business processes, learning about them through exploration and analysis of the data, and achieving new and better outcomes by exploiting and using that insight. Enterprises will move to linear-scaling data platforms for both structured and unstructured data to support the continued growth of data assets and data use.
These new uses are not limited to the internal (data-driven)
organisation. Companies can now seek opportunities to monetise their
data by providing it to partners and customers, because syndicating
outside uses of it also enjoys linear cost to deliver increasing scale.
What if our health plans and grocery loyalty programs started sharing
data?
On the bright side, personalised dietary modification suggestions are
probably not far behind. On the other hand, do I really want the
doctor disapprovingly telling me how many pints of Ben and Jerry’s I’ve
had in the last decade as I’m in recovery from an emergency heart
bypass?
By John Akred, senior manager, Accenture Technology Labs
Subscribe to this blog
Email
Permalink
Print
Contact Us
