Bixby was created to provide a more personalised experience to enhance the way people interact with their phones and the world around them.
Bixby is not an ordinary voice assistant, but rather an intuitive and comprehensive interface that leverages contextual awareness to learn users’ habits and respond accordingly.
Samsung says the aim is to grow from an interface for your smartphone to an interface for your life.
Initially, it offers four main feature – voice, vision, reminders and home.
Research shows that users wanted to engage on their terms so Bixby wakes instantly by pressing a dedicated button on the side of the S8 or the Bixby icon on the lock screen. But once it is awake you can mix it up by voice, touch, type and more interchangeably – it is called multi-modality.
Bixby uses natural language understanding and adapts to the unique speaking style of the user. If it can’t get the context, if will ask for more information so it can complete the task, rather than giving up and presenting often useless search results.
At launch, it will work on selected apps and system commands. For example, if a user wants to send pictures to a friend, they can select images in their Gallery by tapping them, then tell Bixby to send the selected photos to their contact.
Bixby can see what you see through the S8 camera but more importantly, it can apply context and recognition to the image. For example, Bixby sees an image of a building and uses the image to determine that the structure is a famous art museum and can then provide related information, such as the museum’s address, phone number, and website.
It can also leverage Vision to get a translation of say, a French menu into the user's native language or learn more information about the bottle of wine they would like to order.
Vision will also facilitate shopping to identify an item or QR code and locate it online.
Bixby knows what the user is doing (via the calendar) and can add context to reminders like to leave earlier for the airport due to traffic.
Or you can ask it to remind you to buy eggs, milk and bread at the shops and it will do so based on time and location and regular shops.
The S8 also uses Bixby reminder cards along the Edge style screen.
Home is about you and your life and it's where Bixby becomes more of an aide-de-camp
Home integrates with select native Samsung apps to provide a look at all the activity across the Galaxy S8 through a customisable card-like interface. For example, Home’s contact card shows frequent and missed call contacts, while music-related cards provide options such as playing paused content. Facebook and other social media cards, meanwhile, display trending content and news cards announce the latest headlines.
But Home doesn’t just relay information; it puts it into context, paying attention to the user’s schedule, routine and personality each time it’s used. In this way, Bixby can anticipate what the user needs and respond accordingly.
Say someone often uses Uber to commute to work. Bixby will start to place an Uber card at the top of Home in the morning, with a fare estimate and request button. When it’s time for bed, Home might display a user’s alarm and health summary of daily activity. The more users access Bixby, the more it evolves and adapts.
"While not reinventing the smartphone category, Samsung raises the bar with its new flagship S8 devices," Thomas Husson, vice-president and principal analyst at Forrester, said. "The new design, iris scan, as well as top-of-the-line hardware specifications coupled with new Gear VR headset and 360 cameras — coupled with an upcoming massive communications plan — will help Samsung benefit from about five months sales opportunity before the launch of the new iPhone.
"The most interesting element of the announcement is the integration of the Bixby assistant. No doubt that in the coming years, voice-based intelligent agents will reshape how we interact with devices, brands and our environment – the way touchscreens became new interfaces in less than a decade.
"The Galaxy S8 is just a first step in integrating Bixby into the broader IoT and smart home ecosystem for Samsung. In that space, Samsung will face fierce competition from broader digital platform ecosystems from Apple, Amazon or Google. To be successful in the long term, Samsung will have to work way more closely with third-party brands and developers and even to extend Bixby beyond the Samsung portfolio of devices."
Samsung chief technology officer Injong Rhee discusses Bixby in depth in the video below.