Hackbench – Find Best Performing Kernel for Android

If you are looking for the best performing Kernel for Android, you can do that easily with Hackbench, a benchmarking and stressing tool designed for the Linux Task Scheduler.

If you are using custom kernels on your Android devices, most probably you came across the situation where you had to choose among a large number of different custom kernels. In the search for the best performing Android kernel, most users just pick the kernel with the largest number of features. Others rely on the usage experience for their decision.

However, a kernel overblown with custom features and backports does not guarantee it will run better. Furthermore, choosing a kernel based on the usage experience alone is a highly subjective practice and results lack any scientific evidence. No wonder why most custom kernels receive mixed reviews from users that report based mainly on their usage experience.

The need for a more scientific approach pushed developers to create several benchmarking Apps for Android (Antutu, GeekBench, and PCMark are the most notable). As these applications are built on top of the Android software stack, they generally put the whole Operating System to the test and not only the Linux kernel. Any external factors interfering with the results cannot be eliminated, which explains why results can vary significantly between tests on the exact same device software and hardware.

The best way to test the Linux kernel’s performance is to use a tool that talks directly to it, without any Java code or system management layer in-between. The ideal candidate for that would be a command line tool. Today we will discuss how we can use a benchmarking tool created especially for the Linux kernel, in order to compare different custom kernels and find the best performing one. This tool is known as hackbench and is a benchmarking and stressing tool designed for the Linux Task Scheduler. It dates back to 2009, but it is still a handy tool for kernel developers today.

As you should probably already found out, we are focusing on the Task Scheduler performance in this guide. Task Scheduler is one of the most important parts of an Operating System’s kernel, as it is responsible for managing and running all processes on the CPU while maintaining a usable system. Minor changes to its code can have a significant impact on system performance. Its logic is quite complicated, and even a single line of wrong code can be destructive. Furthermore, its performance is prone to changes in other kernel subsystems. There are tools available for testing GPU, Network and other major subsystems of the Linux kernel, but that should be discussed in a separate article.

Hackbench was created for use by developers, to identify the impact of code changes to the Linux Task Scheduler. However, it appears to fit quite well in our context too: Different custom kernel developers apply different patches to their kernels. In addition to that, there are usually more than one approaches to a single problem on Android kernels, each one having a different impact on kernel performance. As a result, there is great variability between custom kernels performance.

If you want to know how to compare Android kernels from the low-level, read below.

Don’t miss: Boost Performance on Android by Tweaking the Kernel’s Task Scheduler

Prerequisites

Hackbench was designed around the x86 architecture (desktop PCs). It runs fine on Android platforms as well, but there are no pre-compiled binaries available. So, we will have to build it on our device first. This is actually a really simple process, as everything can take place inside Termux. Termux is a powerful Terminal App for Android which also provides a small standalone Linux environment inside Android.

• Install Termux
• Open Termux, it will automatically download a small software stack that it needs in order to run. (Make sure you are connected to the Internet when you first start the App).
• Download and Install needed packages:
  Make sure you have around 150 MB of free space on your internal storage. Then, give the following command and hit enter. Termux will take care of everything:

  pkg install -y git clang libllvm

Everything is now ready for the next step, which is downloading the source code and building the tool.

Build and Install Hackbench

• Inside Termux, enter the following command:

  git clone https://github.com/geekydoc/hackbench

  This command will download the hackbench source code through git and put it inside a folder named “hackbench”.

• Next, give the following two commands to build the binary:

  cd hackbench

  gcc -Wall -O2 hackbench.c -o hackbench -lpthread

  The last command might take some time to finish, depending on your device capabilities.

• Lastly, give the following command to move the binary inside the local environment’s executables directory (it makes it easier to execute commands later):

  mv hackbench ../../usr/bin/

Running Hackbench

Being a command line tool, hackbench accepts a number of arguments at run-time. You can get a help prompt for hackbench by giving the following command:

hackbench -h

Hackbench creates a number of process or thread pairs, which communicate between them using UNIX Sockets or Pipes. It then calculates the time needed for all entities to send and receive data. As we can see in the screenshot above, we can enable the usage of pipes instead of sockets (the default) using the -pipe switch. We can choose between process or thread mode by using the respective switch in the command line. The number of process or thread groups (10 by default) is also configurable. Lastly, there is support for changing the number of loops, ie the times that message-exchange operations will repeat before ending the benchmark and calculating total time.

So, to make a benchmark using pipes and threads, with 20 pairs of threads and 2000 loops, we must give the following command:

hackbench -pipe 20 thread 2000

The above command will give an output in the form:

Time: 154.113

This is the total time in seconds needed to complete the benchmark. The lower the time, the lower the scheduler wakeup-latency (the time between a task asks to run and the time it actually runs on the CPU) and the better the kernel performance.

Tips

• Make sure that you test both pipes and sockets and both process and thread modes on each Android kernel. Usually, there are performance differences between them.
• Try to run longer benchmarks (higher number of loops). This will require more processing and will make differences between kernels more clear.
• Try to have your Android in an idle state when running the benchmark. Hackbench will run in parallel with the Operating System, meaning that other system tasks will interfere with it. Clear your recent App Switcher, close any running Apps, disable Wifi and set Airplane Mode to ON. This will make sure that the least possible tasks are interfering with the test.
• Run each test at least 3 times and calculate the mean value. This improves statistical significance and helps reduce interference from other tasks. The mean value of 3 consecutive tests will be ( t1 + t2 + t3 ) / 3 , where t1,t2,t3 are the test results.

Must read: Past and Future of the Linux Kernel on Mobile Devices

Example Benchmark

The following is a real-life benchmark, that took place during kernel development of our testing device (a Qualcomm msm8610 based smartphone). We tested two kernels, the first using the stock Task Scheduler and the other having a scheduler debug feature enabled, HRTICK. In simple words, this feature tries to improve scheduling the wake-up latency by using a separate, high-resolution (nanoseconds) timer, along with the standard scheduler timer. However, this feature might not work well on all platforms, since workloads vary between devices (eg. a desktop PC and a smartphone). Indeed, results show that enabling this feature on our device makes the Task Scheduler slower:

Stock Kernel

Time: 27.845

Kernel with HRTICK enabled

Time: 32.495

Results displayed above are the mean values of 5 benchmarks conducted on each kernel respectively. Command line arguments were the following:

hackbench -pipe 20 thread 1000

For simplicity, we do not include results using sockets or process mode, but the stock kernel performed better in all situations.

To sum up, hackbench is a neat tool to measure your kernel’s performance and compare it with other available custom kernels. If you decide to give hackbench a try, make sure you share your results with us in the comments section below!