The challenge of doing business in real-time is that it means different things to different people, and the expectation of how quickly things need to occur continues to shift over time.

Everyone understands that latency matters. There's a good understanding generally of the impact that higher ping times have on access to applications and data.

Similarly, organisations understand how data – and faster access to it – can fundamentally change the value proposition they create for customers.

For these organisations, it's no longer sufficient to look at past application usage or purchase patterns as a predictor of future customer behaviour.

Instead, organisations are cognisant they now operate in a "real-time economy," where what matters most is what I – as the user or consumer – am looking at now, combined and enriched with the historical context of my past transactions..

For organisations, it's about making online, not offline, predictions. It's about having the ability to learn and execute while the user is still active and engaged. From a technology perspective, it's the ability to continually learn by updating machine learning models in real-time and incorporating new incoming data.

That's the essence of the real-time challenge that organisations are faced with enabling today.

Time matters

Executing the processing and predictive work while the user is online means that time is always going to be of the essence. More to the point, there's still a lot of 'fuzziness' around how real-time an organisation's capabilities have to be in order to execute on that.

Does it need to occur in seconds, milliseconds, microseconds or nanoseconds?

The short answer is 'yes,' but with a big 'but' as it will largely depend on business SLA. Even within early use cases, such as fraud detection and prevention, personalised offers and gaming, there are differences in what constitutes acceptable, real-time performance.

And that gets to the essence of the challenge around real-time: there's a continuum of options as to the time it takes to execute the process.

Where an organisation starts on that continuum is not an indication of where it'll stay. The goalposts will continually move, and organisations will need to keep pace. Real-time is a journey and may take a few steps to accomplish.

As expectations of what really constitutes real-time shift, it's critical that organisations build real-time applications on a platform that enables them to keep pace with this constant compression of the acceptable time window available for processing and predictive analysis.

It's the only way to ensure they remain real-time and that the applications satisfy the value it creates for the customers.

How we got here

The migration from batch processing and offline behavioural analysis to real-time analysis – while a customer is active online – is a trend that's been a while in the making but is coming of age due to a confluence of factors.

The 'art of the possible' is now orders of magnitude different than it was a decade ago. It wasn't long ago that the recommendation from Netflix was the North Star of what was possible. While the engine is still highly effective – with some reports suggesting it drives 80% of stream time – improvements continue to the engine and its underlying machine learning algorithms.

Yet, other large technology-based organisations have since emerged with engines of their own that significantly advance the state-of-the-art. TikTok is an example of this progression; the data science behind its operations is seen as a key reason for its meteoric rise, and more importantly, why its user base is so obsessive.

What this demonstrates is that the power of the underlying platform is a key factor in the drive to real-time, as well as the ability for more organisations to enable real-time approaches in their operations.

Organisations know data is gold. They understand how data can fundamentally change the value they create for customers and, in turn, change customers' behaviours. However, only a fraction of fresh streaming data is being used today. They also know that the era of real-time is possible because there's now enough compute and storage capacity available to power these ambitions.

Additionally, real-time is made possible by advances on the customers' end as well. Other key enablers for accessing data and services in real-time are the constant march to more powerful edge computing devices, as well as the emergence of powerful, low-latency, highly resilient networks like 5G.

Where real-time goes next

The other reason organisations have gone real-time is because of the emergence of more platform-like solutions to execute this strategic direction.

To enable organisations to build real-time solutions, a set of core capabilities or services are required. These include a way to ingest real-time event and messaging, a way to store and have quick access to the data once ingested, streaming analytics, and real-time machine learning.

The challenge organisations face is that it's still non-trivial to stitch these capabilities together. Not every organisation is the size of a Netflix or TikTok, with the engineering resources to build and continuously improve a real-time application or capability.

That's why application enablement is emerging as a key building block for organisations in this space. To build a real-time business or application, organisations need the support of a modern data platform that has the components and underlying compute to realise their real-time ambitions.