Behind Dynamic System Updates in Android Q: How Google is using Project Treble to improve future Android releases

Alongside the release of Android 8.0 Oreo, Google unveiled Project Treble: a major rearchitecting in the way the Android OS framework and the vendor HALs and Linux kernel communicate. Treble is a major initiative designed to reduce Android platform version and security patch fragmentation, and all Android-branded devices launching with Android Pie are required to support Project Treble. OEMs and vendors test Treble compatibility by booting a Generic System Image (GSI)—a pure stock build of Android from AOSP—and passing the Vendor Test Suite (VTS) and Compatibility Test Suite-on-Generic System Image (CTS-on-GSI). The GSI has proven useful in not only allowing software engineers working for OEMs test Treble compatibility, but it has also opened the door for a large custom ROM community on XDA. For the Android Q release, Google wants to make GSIs useful for another group: app developers.

Since the first stable release and source code drop of any given Android platform release usually comes in August, developers who would want to test the next Android release on a real device typically need access to a Google smartphone if they don’t want to wait for the update to reach their own hardware. However, Google worked with OEMs to bring an Android P beta to several devices last year, and they’ve followed up on that this year with an Android Q beta. Alongside an official Android Q beta, Google this year also released an official Q beta GSI so any developer with a Project Treble-compatible device can install the latest Q release without having to wait months for the build to reach their devices. This new way of testing the next Android release gives developers more opportunities, and thus more time, to test their apps against major changes in Android.

Unfortunately, the current method of installing a GSI can be difficult. It requires unlocking the bootloader—which means wiping all user data and/or voiding the warranty—and flashing an image via the fastboot protocol. It’s not a quick and simple process for an app developer to do, if their device even allows for unlocking the bootloader. That’s why, for the past several months, Google has worked on a new way to boot GSIs. Enter a new feature called Dynamic System Update, or DSU.

(This feature was previously developed under the names “Live Image,” “Dynamic Android,” and “Android on Tap,” so don’t be surprised if Google calls it something else in a few weeks or months.)

Dynamic System Updates in Android Q

The goal of the DSU feature is to allow a developer to boot into a GSI without interfering with the current installation. That means the bootloader doesn’t need to be unlocked and the user data doesn’t need to be wiped. The installation process is also greatly simplified as Google has provided a command line interface via ADB and an app that can be controlled via intents. Here’s what it looks like to boot a GSI using DSU:

In this video*, a Google Pixel 3 XL running Android Q beta 3 reboots into a GSI. In this environment, an app developer can install and test their app for Q API compatibility. When they’re done testing, they can simply reboot back into the regular Q beta 3 software on the device. You’re basically dual booting a GSI so you can safely test your app!

*We recorded this video at Google I/O 2019 when DSU wasn’t publicly available yet, so the Q beta 3 build on the filmed Pixel 3 XL was slightly modified by Google to include DSU support. Devices running Q beta 4 and later are eligible to support DSU if they meet the requirements below.

Requirements for Dynamic System Updates

Getting what’s essentially dual booting up and running wasn’t an easy task for Google. Major changes had to be made to the way partitions are managed on the Pixel 3, Google’s testbed for DSU. Thus, the first major requirement for DSU support is that the device supports dynamic partitions. Dynamic partitions involve one real partition of storage that’s divided into resizable logical partitions like system, vendor, odm, oem, product, etc. During the installation of a GSI, space is reserved for new system and userdata partitions by taking unused blocks from the existing userdata partition. Since these new partitions can be several gigabytes in size, DSU support only makes sense with logical partitions otherwise a device would need to permanently reserve several gigabytes of storage space for GSI installations.

Other requirements include a ramdisk, which decides whether to boot to recovery, system, or a logical partition, and a metadata partition to store the metadata of the GSI. In general, the building blocks for DSU support are Android Q launch requirements, according to Project Treble lead Iliyan Malchev. We’re not sure if everything that’s needed to support DSU is an Android Q launch requirement, but we can presume that most if not all devices launching with Android Q can support DSU even if Google does not currently require them to. So far, only the Pixel 3, Pixel 3 XL, Pixel 3a, and Pixel 3a XL have dynamic partitions, and of these devices, only the Pixel 3 and Pixel 3 XL support DSU in Android Q beta 4. Although DSU support isn’t required, Google hopes that OEMs enable the feature anyway because it simplifies securely testing for Treble compatibility. For example, an OEM software engineer can put a GSI on an SD card so they can quickly boot on multiple devices to test Treble compatibility.

Security for Dynamic System Updates

Since DSU introduces essentially a second operating system into the mix, Google needs to make sure that this new installation can’t be tampered with to break the integrity of the device. Thus, the same basic security protections for the original installation are in place for the GSI installation: Android Verified Boot and SELinux policies. Furthermore, only apps with the INSTALL_DYNAMIC_SYSTEM signature|privileged permission can initiate a GSI installation, while apps with the MANAGE_DYNAMIC_SYSTEM signature permission can enable/disable or wipe a GSI installation. This means that only trusted, system-level apps can work with DSU.

To ensure that the original user data is protected, Google has added an extra protection mechanism in Android Q. Called “Checkpoint,” the feature protects against user data destruction by restoring checkpointed partitions to their original state. Checkpoints are useful for not just DSU, though. They’re also used to protect against botched Project Mainline APEX module and A/B OTA updates. (Devices with A/B partitions already have rollback protection, but those rollbacks require factory data resets while user data checkpoints do not.)

Installing a GSI

If your device supports DSU like the Pixel 3 series, then it’s easy to install a GSI. You first have to make sure that the Dynamic System feature flag is enabled through one of two ways:

1. If you’re on a userdebug build, enable the settings_dynamic_android flag in Settings > System > Developer options > Feature flags.
2. If you’re on a user build, run the following adb shell command:

   setprop persist.sys.fflag.override.settings_dynamic_system 1
   

Then, download the latest Android Q beta GSI from Google or your device’s OEM. (DSU only allows installing GSIs signed by Google or an OEM.) Once downloaded, use simg2img to convert the sparse image to a raw image. Use gzip to pack the raw image, and then copy the resulting archive to a location on your device’s external storage (e.g. /data/media/0/Download) or an actual external storage medium (like a physical SD card). Lastly, launch the DynamicSystemInstallationService app with the right intent to begin installation:

adb shell am start-activity \ -n com.android.dynsystem/com.android.dynsystem.VerificationActivity \ -a android.os.image.action.START_INSTALL \ -d file:///storage/emulated/0/Download/system_raw.gz \ --el KEY_SYSTEM_SIZE $(du -b system_raw.img|cut -f1) \ --el KEY_USERDATA_SIZE 8589934592

Once you click restart, you’ll boot into the GSI. The usability of the device in the GSI depends on how well your device’s OEM implemented Treble (or rather, how little they violated Treble compatibility.) Some devices will work better with GSIs than others, but in general, don’t expect to use this installation as a daily driver. You’re meant to test your app then get out by rebooting. If you want to stay in the GSI installation for further testing, then you can use the gsi_tool shell command.

The full GSI installation instructions for DSU can be found here. Bugs can be filed on the Google Issue Tracker, Reddit, or Stack Overflow.

The reason behind Dynamic System Updates

When I spoke to Iliyan Malchev at Google I/O, he reiterated what Hung-ying Tyan from the Treble team said about early GSI access at November’s Android Dev Summit. Google made DSU to solicit feedback from as wide an audience as possible. The goal is to improve the quality of the GSI, which in turn improves the quality of the future Android release since a GSI is the purest form of Android. Google is currently the only company that tests next-version GSI compatibility (for example, how well the Android Q system image works on top of the Android P vendor implementation), but as more people flash GSIs and give feedback, OEMs can fix Treble compatibility violations so GSIs will work even better in the future. Iliyan says there’s strong interest from OEMs and vendors like Qualcomm in reusing vendor images from the previous Android release with the next-version system image. Initiatives like DSU help Google and OEMs plug the gap in coverage from automated tests like VTS and CTS-on-GSI. Thus, Google gets more beta testers to give feedback on the next Android release, while also hearing about Treble compatibility violations so OEMs can improve their work.

The addition of Dynamic System Updates in Android Q is welcome, but it’s not going to be the dual boot solution some of you are hoping for. As mentioned previously, only system images signed by Google or OEMs can be installed. When I asked Iliyan about the possibility of extending DSU to support an ecosystem of alternative Android systems, he said that it’s technically possible to do so as DSU is simply a channel to deliver system images. Any OEM can use it however they want so long as the end result is Android compliant. Google hasn’t created an alternative to the OTA system here, and DSU isn’t intended to be used for true dual booting. Regardless, the work that Google has done on Treble is making Android more modular, so I wouldn’t be surprised if native dual booting becomes reality down the road.

The post Behind Dynamic System Updates in Android Q: How Google is using Project Treble to improve future Android releases appeared first on xda-developers.