Wednesday, 02 March 2016 10:08

Why updating Android without vendor help is a nightmare


Over the last few years, Google's Android mobile operating system has taken over the market leaving all others in the dust. But along with that has come a spate of security issues that often make headlines.

The operating system runs on billions of smartphones and tablets, made by all sorts of companies. It primarily targets the ARM platform. A count can only be kept when a device is activated and many users choose not to do so.

Android is based on the Linux kernel which is released under the GNU General Public Licence version 2 and is free software; modifications can be made but if the modified binary kernel is distributed then the source needs to be made available too. All other components are released under the Apache licence, which means that there is no obligation to divulge any changes; in short, these can be locked away.

As manufacturers have taken up Android, each has customised it in order to try and make it attractive to the consumer. The most extreme example is Cyanogenmod.

Thus, although Google releases updated firmware when there are major security issues, mobile users have to wait until their vendor uses the fixes to create new firmware and make it available. This rarely happens and creates something of a security nightmare.

Creating and releasing new firmware with security fixes incorporated is a costly and lengthy exercise; a company need specialised staff and also needs to carry out testing before release. Given that lots of companies which have Android devices are small entities, they tend to avoid issuing updates, concentrating instead on the next generation of their device to keep up with the competition.

To get an understanding of the difficulties involved in keeping Android secure, iTWire spoke to Matthias Brugger (seen above) and Andreas Faerber, both engineers at SUSE Linux, the former in Spain and the latter at SUSE headquarters in Nuremberg.

iTWire: First, can you explain how ARM is different from other platforms. As I understand, you have to port things to every different processor unlike, say with x86, where one port will work on all x86 processors.

Matthias Brugger and Andreas Faerber: x86 processors have a long history of standardisation. An x86 processor is mostly made of the CPU and an interface to the BIOS/UEFI firmware on the motherboard. From there the operating system can gather the information about the hardware components present in the system. In comparison, ARM processors, especially in the Android/embedded market are System-on-Chips (SoCs). This means, apart from the CPU the SoC provides most to all peripheral components on one die (e.g. network chip, USB chip, display chip, etc).

Apart from that, although ARM has standard components for the core system (e.g. interrupt controller, timers etc), most ARM SoCs implement their custom components here, which needs their own drivers in the kernel. The reason is because many companies which produce ARM-based processors have a long history in the mobile phone market in the pre-smartphone era. They have core components which were developed years ago and they want to re-utilise them, as they have a deep knowledge on how they work and their custom software drivers.

Third, especially in the embedded world, the ARM processors don't use the Unified Extensible Firmware Interface (UEFI) to communicate with the firmware. To tell the Linux kernel which components are present, the bootloader passes a device tree blob to the kernel. So there is no dynamic way to tell the kernel which components are present. This is changing a lot and the Linux community are putting a big effort in standardising the firmware interface (EFI, PSCI, ACPI).

iTWire: Secondly, as I understand it, you cannot update one part of the Android firmware as you would with, say, an openSUSE installation. You have to issue the complete firmware. Why is this?

Matthias Brugger and Andreas Faerber: In general an Android system is made of different partitions, which fulfill different functions. The bootloader, the boot image (kernel and rootfs), the Android core system, the recovery partition and the user data are some of the most important parts. Depending on the manufacturer of the SoC there may be more partitions.

In theory, you can update these partitions individually, but in many cases the software ecosystem of Android is more tightly coupled than, let's say, an x86 system. This happens due to the fact that the SoC vendors and original design manufacturer (ODM) see their system as one building block, so they may do custom changes to, for example, the interface between bootloader and kernel, and kernel and userspace.

iTWire: What are the technical issues involved in upgrading the kernel on an Android device without vendor support?

Matthias Brugger: First of all, it could be quite difficult to get the kernel sources and get them compiled. The kernel source code has been heavily modified by the SoC vendor and maybe by the ODM as well. Most ARM SoC vendors have heavily changed the Android build system which doesn't make it trivial to compile the kernel.

As already mentioned, an Android-based kernel has been touched by many hands. From Google, to add the necessary Android parts, from the SoC vendor to make it work on their SoCs and by the ODM, which integrates external components (e.g. the touch panel driver). Depending on the last two, the code quality can vary a lot, and it might be quite difficult to find out, which parts of the mainline kernel sources are really used by the kernel in question. So any update from the mainline kernel to one of the production kernels can be quite difficult to handle.

Next step is to find out how to update the boot image which contains your kernel. Basically you can flash the partitions using fastboot, but many mobile phones in production don't have fastboot enabled, as this might be a security concern. Normally the SoC vendors have their own solution to flash through the first stage bootloader. But these tools are not always published by the SoC vendors. You may even find yourself in need of a Windows machine to be able to flash your Linux kernel to the board.

Another problem you might encounter is the difficulty in finding the bootloader sources for your device. If you are working on the bootloader/kernel level you need a serial console or even a Joint Tag Action Group specified access to debug the system. In your commercial device, there might be no easy solution to get access to these pins.

iTWire: In this story the bug in question also affects ARM systems. Now in this case, the people who found the bug claim it will affect all ARM devices with a 3.8 kernel and later. How is that possible, when each ARM processor stands literally on its own?

Matthias Brugger: The Linux kernel is split in several parts. There is a common part which is independent of the underlying hardware architecture. This is sometimes called the kernel core which includes, for example, the process scheduler, the file systems or the network protocol implementation. Apart from that parts exist which include architecture specific code (e.g. for ARM or x86) and drivers for the hardware devices used by the kernel. The subsystem where the kernel bug was found is part of the generic part of the kernel. Therefore it only depends on the configuration of the kernel if the access to keys in the kernel is enabled or not, to make the system vulnerable or not.

Andreas Faerber: It is worth pointing out that there is a difference between updating the kernel with some bug fix and upgrading the kernel to a newer version. Matthias' explanations mainly focused on updating the kernel at all.

In the open Linux server ecosystem, companies are expected to integrate their changes for hardware enablement into the mainline Linux kernel. They get reviewed by kernel maintainers and interested people and, once merged, undergo regular testing by various parties.

Using SUSE as an example: When we do a product release, changes from the latest mainline kernels may get backported to the chosen base kernel version. Our security team actively monitors security vulnerability disclosures during product lifetime and arrange for updated packages through further backports, in addition to the usual customer-driven support services.

Many manufacturers of Android devices take a shortcut and do it the opposite way - the SoC vendor takes an Android kernel (based on an older mainline kernel release), does the hardware enablement work based on that forked version, hands it on to its ODM customers and they, in turn, do the final device enablement, again based on the old version.

The deliverable from this workflow is often referred to as Board Support Package (BSP). The changes made this way never go through the review processes of the mainline kernel, and there is no upgrade path for users at all, i.e. there is vendor lock-in.

If you want to upgrade, the enablement patches need to be forward-ported first. This is either done by the companies themselves if they decide to  offer a new Android version for the existing hardware, or otherwise by a community of hobbyist users - and if you're unlucky you need to do the work yourself. Nothing technically prevents the vendors from delivering security back-ports for their old kernel, but in practice most vendors don't seem to bother about security issues.

Linux kernel developers are aware of these issues and some of us SUSE engineers have been lobbying for SoC vendors and development board vendors to work with the  mainline kernel community to improve the situation. At Embedded World Conference 2014 we published a paper comparing the BSP situation to our openSUSE Linux distribution workflows and at LinuxCon Japan 2015 I held a presentation about mainlining ARM board and SoC support.

One big obstacle for using mainline Linux kernels on arbitrary Android devices is the lack of accelerated graphics drivers for several popular graphics IP vendors. So, likely (but not even guaranteed) with the mainline kernel you can get simple framebuffer graphics, if you're lucky also some 2D acceleration, but any 3D acceleration for OpenGL ES and related APIs is unlikely. Depending on whether we're talking about a phone or tablet device with display that can be a show-stopper to upgrading.

Subscribe to ITWIRE UPDATE Newsletter here


It's all about Webinars.

Marketing budgets are now focused on Webinars combined with Lead Generation.

If you wish to promote a Webinar we recommend at least a 3 to 4 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site and prominent Newsletter promotion and Promotional News & Editorial. Plus a video interview of the key speaker on iTWire TV which will be used in Promotional Posts on the iTWire Home Page.

Now we are coming out of Lockdown iTWire will be focussed to assisting with your webinatrs and campaigns and assassistance via part payments and extended terms, a Webinar Business Booster Pack and other supportive programs. We can also create your adverts and written content plus coordinate your video interview.

We look forward to discussing your campaign goals with you. Please click the button below.



iTWire TV offers a unique value to the Tech Sector by providing a range of video interviews, news, views and reviews, and also provides the opportunity for vendors to promote your company and your marketing messages.

We work with you to develop the message and conduct the interview or product review in a safe and collaborative way. Unlike other Tech YouTube channels, we create a story around your message and post that on the homepage of ITWire, linking to your message.

In addition, your interview post message can be displayed in up to 7 different post displays on our the site to drive traffic and readers to your video content and downloads. This can be a significant Lead Generation opportunity for your business.

We also provide 3 videos in one recording/sitting if you require so that you have a series of videos to promote to your customers. Your sales team can add your emails to sales collateral and to the footer of their sales and marketing emails.

See the latest in Tech News, Views, Interviews, Reviews, Product Promos and Events. Plus funny videos from our readers and customers.


Sam Varghese

Sam Varghese has been writing for iTWire since 2006, a year after the site came into existence. For nearly a decade thereafter, he wrote mostly about free and open source software, based on his own use of this genre of software. Since May 2016, he has been writing across many areas of technology. He has been a journalist for nearly 40 years in India (Indian Express and Deccan Herald), the UAE (Khaleej Times) and Australia (Daily Commercial News (now defunct) and The Age). His personal blog is titled Irregular Expression.

Share News tips for the iTWire Journalists? Your tip will be anonymous




Guest Opinion

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News