Big Data Market Segment LS
Big Data Market Segment RS
Wednesday, 19 February 2020 13:16

Avoid data swamps by concentrating on data products: Teradata CTO

Teradata CTO Stephen Brobst Teradata CTO Stephen Brobst

Silicon Valley has stopped talking about big data, according to Teradata CTO Stephen Brobst.

The industry has realised that executing big data in isolation failed to deliver value, and that organisations instead need to think in terms of their data ecosystem.

A data strategy needs to encompass unstructured data, big data, and all the other types of data, he said. And a next-generation data strategy involves "the integration of... all of the useful data."

Where analytics processing commonly took place on premises, it is increasingly being done in the cloud, although government, financial and health data tends to remain on premises.

Few organisations have as yet gone all-in on the cloud, although cloud-first is a common strategy. Data gravity is thus moving towards the cloud, but an analytics strategy requires access to data that is generated and stored in both places.

Completely replicating data across two platforms is an expensive process, so Brobst favours either "surgical replication" of precisely the data required in both places, or the use of Teradata QueryGrid, which integrates multiple file systems and moves data on demand.

More generally, Teradata's Vantage platform separates compute and storage, allowing a single copy of data to be used with multiple analytics engines. "You get a much cleaner implementation" whether the data is stored in a Teradata file system, Hadoop, or a cloud-native object store, he told iTWire.

There has been a "stampede into native object stores," he said, because they are economically attractive and deliver good availability and throughput. But they are not suitable where low latency is required – in that case Teradata's technology (available on premises or in the cloud) is more appropriate.

Vantage's flexibility in this regard means organisations can optimise for specific requirements.

"Getting the architecture right is really important," he warned. Throwing everything into a data lake and leaving it for someone else to worry about later just creates a data swamp or a "data dumping ground."

Brobst pointed to Gartner research showing that 90% of data lakes have delivered no value because they lacked both architectural principles and governance, and other research that found 80% of data science work fails to go into production.

Organisations need strategies to promote data products (ie, models and insights that provide real business value), he suggested.


As part of our Lead Machine Methodology we will help you get more leads, more customers and more business. Let us help you develop your digital marketing campaign

Digital Marketing is ideal in these tough times and it can replace face to face marketing with person to person marketing via the phone conference calls and webinars

Significant opportunity pipelines can be developed and continually topped up with the help of Digital Marketing so that deals can be made and deals can be closed

- Newsletter adverts in dynamic GIF slideshow formats

- News site adverts from small to large sizes also as dynamic GIF slideshow formats

- Guest Editorial - get your message out there and put your CEO in the spotlight

- Promotional News and Content - displayed on the homepage and all pages

- Leverage our proven event promotion methodology - The Lead Machine gets you leads

Contact Andrew our digital campaign designer on 0412 390 000 or via email



Security requirements such as confidentiality, integrity and authentication have become mandatory in most industries.

Data encryption methods previously used only by military and intelligence services have become common practice in all data transfer networks across all platforms, in all industries where information is sensitive and vital (financial and government institutions, critical infrastructure, data centres, and service providers).

Get the full details on Layer-1 encryption solutions straight from PacketLight’s optical networks experts.

This white paper titled, “When 1% of the Light Equals 100% of the Information” is a must read for anyone within the fiber optics, cybersecurity or related industry sectors.

To access click Download here.


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.



Recent Comments