While Teradata chief technical officer Stephen Brobst thinks "Big Data is a terrible name" because it's not just about the "bigness" of the data but rather "a different kind of data" (text, social content, rich media and so on), he says the technology is crossing the chasm (a reference to Geoffrey Moore's 1991 book Crossing the Chasm) into the mainstream market.
Big Data has four key characteristics, he said: volume, variety, velocity and (as an important success factor) value. Mr Brobst said there is a lot of "technology tyre kicking" going on, whereas projects should be aimed at solving business problems.
Up until mid-2012, Big Data was the preserve of Internet companies (eg, Google, Amazon). Now, he said, it is being taken up by 'early mainstream' businesses such as Fortune 1000 companies, banks, telcos, retailers and healthcare organisations.
Not surprisingly, Teradata can help with that.
In a presentation given in Melbourne today, Mr Brobst explained how T-Mobile's US operation had used Teradata products to explore and subsequently exploit large amounts of both traditional and non-traditional data in an attempt to better understand the customer experience with the goal of increasing market share.
How did T-Mobile approach this task? Find out on page 2.
The company used Teradata's Aster technology (which makes MapReduce readily accessible via SQL rather than requiring programming expertise) to combine operational data from its cellular network with three billion text records from the contact centre system.
This allowed the identification of 'account activity sessions' - series of actions occurring within 60 minute intervals - and then recurring patterns among sessions.
This allowed T-Mobile to spot patterns of interactions that lead to outcomes such as churn (bad) or upgrades (good).
For example, when customers received warnings about the amount of data they had used, they were more likely to upgrade to a plan that suited their current level of use as opposed to reducing their data usage if they called the contact centre than if they visited T-Mobile's web site (both contact options were presented in the warning message).
So the company now encourages customers to call, and has redesigned that part of its web site.
Patterns were similarly found that revealed dissatisfaction caused by poor initial experiences, and identified situations were intervention at the right time could head off subsequent account closures.
Mr Brobst said Teradata's integrated systems were an important consideration for companies such as T-Mobile, which used Teradata's enterprise data warehouse systems for production analytics, Aster-based products for "data R&D" (where "you're not always going to find something"), and Hadoop-based products to keep "all data forever" at low cost with the most appropriate data being promoted to the data warehouse via an ETL (extract, transform and load) process.
Looking to the fairly near future, he said "people are figuring out how to get value from [Big Data]," and predicted the technology would increasingly show up in application areas such as fraud detection, 'smart cities' and healthcare. There is "a huge opportunity to make healthcare more effective than it is today," he suggested, but reiterated that success comes from focussing on problems rather than the technology itself.