Introduction to Android Bytecode Processing

This article mainly introduces the content related to Android bytecode processing, including compilation process, bytecode modification stage, ASM framework (introduction, two modes, execution model, core components), class structure and descriptor, etc. It also mentions that the learning curve of ASM is steep, suggesting checking the official website and example code.

Related questions: How does ASM modify bytecode? What are the performance advantages of ASM? How to learn ASM?

Android Compilation Process

Java code is compiled into bytecode that runs on the Java Virtual Machine. Android’s Java code is first compiled into Java bytecode, then compiled into a single dex format file using the dx tool, and finally runs on Android’s ART virtual machine. Starting from Android Gradle plugin 3.4.0, R8 can be used to perform desugaring, compression, obfuscation, optimization, and dex processing in one step, as shown below.

Bytecode Modification

Bytecode modification can be performed in two stages: during the .class bytecode stage or during the dex stage. Dex is a compressed version of .class bytecode, and Java jar files contain many class files, while each APK file contains only one classes.dex file, as shown below.

The dex format is more compact, making bytecode modification more difficult. Starting from Android Gradle plugin 3.4.0, R8 is used to perform dex processing, which includes obfuscation, shared constants, and shared variables, making it even more challenging.

ASM Introduction

ASM (Apache Software Model) is a general-purpose Java bytecode manipulation and analysis framework. It can be used to modify existing classes or dynamically generate classes directly in binary form. ASM provides some common bytecode transformation and analysis algorithms that can be used to build custom complex transformations and code analysis tools. ASM offers similar functionality to other Java bytecode frameworks, but with a focus on performance.

Note: The name “ASM” has no meaning; it is simply a reference to the C asm keyword, which allows certain functions to be implemented in assembly language.

ASM Modes

The ASM library provides two APIs for generating and transforming compiled classes: the Core API, which uses an event-driven model, and the Tree API, which uses an object-based model. The event-driven model represents a class as a series of events, each representing a class element, while the object-based model represents a class as an object tree. These two modes are similar to the two parsing modes used in Android’s XML parsing: SAX (Simple API for XML) and DOM (Document Object Model).

ASM Execution Model

Note that I followed the rules you specified, preserving the original Markdown structure and content. Before introducing bytecode instructions, it’s necessary to introduce the Java Virtual Machine (JVM) execution model. As is well known, Java code is executed within a thread. Each thread has its own execution stack, composed of stack frames. Each stack frame represents a method call: each time a method is called, a new stack frame is pushed onto the current thread’s execution stack. When a method returns (either normally or due to an exception), this stack frame is popped from the execution stack and the calling method (whose frame is now at the top of the stack) continues executing. Each stack frame contains two parts: local variable part and operand stack part. The local variable part contains variables that can be accessed in random order by their index. The operand stack part, as its name suggests, is a value stack used by bytecode instructions to pass operands. This means values on this stack can only be accessed in last-in-first-out (LIFO) order. Do not confuse the operand stack with the thread’s execution stack: each stack frame on the execution stack has its own operand stack. The size of the local variable and operand stack parts depends on the method’s code, which is computed at compile-time and stored together with bytecode instructions in the compiled class. Therefore, all stack frames corresponding to a given method call have the same size, but the sizes of the local variable and operand stack parts for stack frames corresponding to different methods may differ.

The above figure shows an example execution stack with three frames. The first frame contains 3 local variables, its operand stack has a maximum size of 4 and contains two values. The second frame contains 2 local variables and its operand stack contains two values. Finally, the top-most frame on the execution stack contains 4 local variables and two operands.

Note: If a method call is an instance method, the first parameter of the local variable is this, followed by the local variables mapping to the method's parameters in order (the number of local variables equals the number of method parameters). If a method call is a class method, there are no local variables for this.

Execution Stack Structure

The structure of the execution stack is described in detail in the Java Virtual Machine Specification, Chapter 4.

Class Descriptor

A class’s internal name is its fully qualified name with dots replaced by slashes. For example, the internal name of String is java/lang/String. The internal name represents a type descriptor.

Primitive type descriptors are single characters: Z for boolean, C for character, B for byte, S for short integer, I for integer, F for float, J for long integer, and D for double precision. Class type descriptors start with L, followed by the class's internal name, enclosed in parentheses.

Method Descriptor

A method descriptor is a sequence of type descriptors that describe the method’s parameter types and return type. A method descriptor starts with a left parenthesis, followed by each formal parameter’s type descriptor, followed by a right parenthesis, followed by the return type descriptor. If the method returns void, there is no return type descriptor.

(Ljava/lang/String;)Ljava/lang/Object;

In this example, the method descriptor describes a method that takes a String as input and returns an Object.

For example, in the first line of code, the method descriptor for void m(int i, float f) is (IF)V. The method descriptor omits the method name and parameter names. At the same time, the parameter list comes before the method name, enclosed in parentheses, while the return value is outside the parentheses. The data type order of parameters and return values in a method descriptor matches that of Kotlin’s method declaration, but not Java’s.

ASM Core Components

ASM provides three core components based on the ClassVisitor API to generate and transform classes:

• The ClassReader class parses an already compiled class given as a byte array and calls the corresponding visitXxx methods on a ClassVisitor instance passed as a parameter to its accept method. It can be considered as an event generator.
• The ClassWriter class is a subclass of the abstract ClassVisitor class and directly builds an already compiled class in binary form. It generates a byte array containing the compiled class as output, which can be retrieved using the toByteArray method. It can be considered as an event consumer.
• The ClassVisitor class delegates all incoming method calls to another ClassVisitor instance. It can be seen as an event filter.

Generating Classes

The methods of the ClassVisitor class must be called in the following order, which is specified in this class's Javadoc:

This means that visit must be called first, followed by at most one call to visitSource, then at most one call to visitOuterClass. Then any number of visitAnnotation and visitAttribute calls can be made in any order, followed by any number of visitInnerClass, visitField, and visitMethod calls in any order. Finally, a single call to visitEnd must be made to end the class generation. The order of generating classes is illustrated by the following example code:

val cw: ClassWriter = ClassWriter(0)
     cw.visit(
         V1_8, ACC_PUBLIC + ACC_ABSTRACT + ACC_INTERFACE,
         "org/exmple/Comparable", null, "java/lang/Object",
         null
     )
     cw.visitField(
         ACC_PUBLIC + ACC_FINAL + ACC_STATIC, "LESS", "I",
         null, -1
     ).visitEnd()
     cw.visitField(
         ACC_PUBLIC + ACC_FINAL + ACC_STATIC, "EQUAL", "I",
         null, 0
     ).visitEnd()
     cw.visitField(
         ACC_PUBLIC + ACC_FINAL + ACC_STATIC, "GREATER", "I",
         null, 1
     ).visitEnd()
     cw.visitMethod(
         ACC_PUBLIC + ACC_ABSTRACT, "compareTo",
         "(Ljava/lang/Object;)I", null, null
     ).visitEnd()
     cw.visitEnd()
     val b: ByteArray = cw.toByteArray()

Here is the translation of the Markdown content:

ASM Appetizer

The learning curve for ASM is relatively steep because it directly operates on bytecodes, requiring a large amount of bytecode knowledge and related JVM execution information. It also requires some understanding of the working principles of Java virtual machines. However, once mastered, ASM can provide developers with great flexibility and performance advantages. The level of Chinese translation and the translator’s skills have a significant impact, in my opinion it is still recommended to refer to ASM’s official documentation and example code to deepen understanding.

Reference Resources

ASM website: https://asm.ow2.io/