Blog
The JIT Compiler in Java
Java is both a compiled and interpreted language. While the javac tool takes the source code .java files you writes and compiles them to bytecode .class files, the JVM interprets this bytecode to produce machine code. This machine code is then fed to the CPU to ultimately process or run your program.
To maximize performance, Java utilizes a just-in-time (JIT) compiler to dynamically compile bytecode to machine code. This JIT compiler is baked into the JVM and works automatically without any developer intervention.
While developers rarely need to concern themselves with the inner workings of this JIT, it's beneficial to understand what's happening under the hood. This starts with understanding the difference between compiled vs interpreted languages..
Preface: Compiled vs Interpreted Languages
Compiled languages
Languages like C and C++ are compiled languages. These languages compile source code into an executable binary file that runs directly on the processor (CPU).
Compiled languages are good because they don't require any overhead during runtime. Unlike an interpreted language, compiled code doesn't need to be evaluated at runtime. This makes compiled languages generally faster than interpreted languages.
Compiled languages can be bad because they lack portability. The compiled binary output is often specific to a certain architecture or platform. The compiled output of your program may work on this machine but not that machine etc.
Compiled languages are also slower on startup and during testing. This is because they need to be compiled first!
Interpreted languages
Languages like php, Python, and JavaScript are interpreted languages. These languages rely on a separate program (the interpreter) to "interpret" the code during runtime.
Instead of compiling the source code upfront, interpreted languages read source code line by line, generating the binary instructions through the interpreter itself. This makes it possible for interpreted languages to be "platform agnostic" meaning they can run on different platforms the same way.
Interpreted languages are good because they are portable. You can download php for Mac, Windows, etc and your program will execute the same way.
Interpreted languages are bad because they take longer to execute code. Unlike a compiled language, interpreters can't feed compiled code directly to a processor. They have to "interpret" the source code line by line.
Interpreted languages make for faster startup times. You don't have to wait for the compile step when your program starts.
JIT: The Best Of Both Worlds...
The JIT compiler allows Java to be both a compiled and interpreted language. It combines the performance advantage of a compiled language with the flexibility of an interpreted language.
When your Java program executes, the JVM interprets bytecode as part of code analysis and optimization. It then uses this analysis to dynamically compile and store frequently reused methods to avoid interpreting the same bytecode over and over.
What does this JIT compiler look like? Where does it live? How does it work?
What is the JIT Compiler in Java?
The JIT compiler is part of the JVM that continuously runs to optimize performance. While the JVM interprets bytecode, the JIT analyses execution and dynamically compiles frequently executed bytecode to machine code. This prevents the JVM from having to interpret the same bytecode over and over.
The JIT compiler is not some separate application you download and run alongside your program. It is simply a description of functionality that exists within the JVM.
How the JIT Compiler Works in Java
The source code you write (.java) is compiled by javac to bytecode. When your program executes, the JVM starts interpreting this bytecode.
The interpreter translates the bytecode into machine code and feeds it to the CPU for processing. While this happens, the JIT profiles and analyses the code execution. It tracks the most frequently called methods and dynamically compiles these to machine code at runtime.
This dynamic process prevents the interpreter from having to "interpret" the same methods over and over. Instead of having to translate bytecode to machine code, the interpreter can instead leverage cached machine code instructions from memory.
This is what makes Java both a compiled and interpreted language. While the JVM initially interprets code, the JIT compiler works to selectively compile frequently executed code "just in time".
This gives Java the advantage of interpreted languages (faster startup times, more portability) and compiled languages (faster execution, optimized performance).
Client Side, Server Side, and Tiered Compilers
Before Java 8, you could specify either client or server JIT compilers. While client compilers were better suited for apps having fewer resources available, server compilers worked better for long running apps.
A client side application may be more sensitive to startup time and have fewer resources available for code optimizations and JIT overhead. A server side application may be long running and more appropriate for heavier optimizations and longer compile times.
The key difference between the two is the level of optimization and profiling. While profiling allows the JIT to better optimize the machine code it sends to the processor, it comes with a performance cost. Client side compilers minimize profiling to reduce overhead and startup time whereas server side compilers rely more heavily on profiling and optimizations.
Beginning with Java 8, the JVM takes a "tiered approach" where both client side and server side techniques are used. During execution, the JVM initially interprets the bytecode. It tracks frequently used methods and optimizes code based on the client side (C1) approach. The JIT continuously analyzes execution and migrates to more of a server side (C2) approach for additional optimizations.
Graal JIT
While modern Java installs utilize the tiered approach of combining C1 and C2 compilers, the C2 compiler is dated and hard to maintain. For these reasons, the Graal JIT compiler was developed as part of a larger GraalVm project.
The Graal JIT compiler is written in Java, giving it the advantage of IDE integration, exception handling, and integration with profilers and debuggers.
The Graal JIT compiler also allows ahead of time (AOT) compiling where execution code is compiled upfront. This reduces startup time experienced with the conventional JIT compiler.
Java JIT Compiler Download?
The JIT compiler is part of the JVM.
There is a popular misconception that you can "download" a JIT compiler for Java. While older versions allowed you to install a JIT as a plugin, all standard Java installations include the JIT compiler as part of the JVM.
Java JIT Compiler Optimizations
As the JIT dynamically compiles code to machine language, it also performs optimizations on your source code to maximize performance. Following are some examples of these optimizations:
Let's start with a basic method:
public void myMethod() {
a = b.get();
z = b.get();
sum = a + z;
}
The JIT compiler will optimize the original myMethod() through several steps...
1. Inline Methods
public void myMethod() {
a = b.value;
z = b.value;
sum = a + z;
}
Notice how the get() method is replaced with directly accessing the value. This eliminates method invocations and improves efficiency.
2. Redundant Loads
public void myMethod() {
a = b.value;
z = a;
sum = a + z;
}
By setting z = a, the compiler reduces the latencies of accessing b.value to improve performance.
3. Copy Propagations
public void myMethod() {
a = b.value;
a = a;
sum = a + a;
}
Notice how this optimization removes unnecessary variables. Since z ultimately just equals a the JIT compiler removes it altogether.
4. Eliminate Dead Code
public void myMethod() {
a = b.value;
sum = a + a;
}
Since a = a is redundant, the JIT compiler simply removes this "dead code" during optimization.
Apart from these steps, JIT compiler also includes other optimizations like:
Removing Null Checks
Let's say you have a basic null check like:
if (myVar == null)
If JIT notices myVar is never called with null then it will remove this code altogether.
Branch Predictions
Let's say you have an if..else branch like this:
if (x >= y) {
do this...
} else {
do that...
}
If the JIT compiler notices that x < y more often than not, it will reverse the branch...
if (x < y) {
do that...
} else {
do this...
}
Unrolling Nested Loops
If you have a nested loop, the JIT compiler will unroll it to minimize assembly jumps..
for(int i = 0; i < list.length; i++){
for(int j = 0; j < list2.length; j++){
result[i] += list[i][j];
}
}
becomes...
for(int i = 0; i < list.length; i++){
result[i] += list[i][0];
result[i] += list[i][1];
result[i] += list[i][2];
}
Thread Local Storage
The JIT compiler utilizes thread local storage (TLS) to more efficiently store variables. Because the thread object is stored directly on the CPU register, it can be more efficient as storage space...
Integer i = new Integer();
becomes...
ThreadLocal i = new ThreadLocal();
Conclusion
The JIT compiler makes Java both an interpreted and compiled language. While the JVM acts like an interpreter to translate bytecode at execution time, the JIT compiler analyses and profiles execution to dynamically compile frequently accessed machine code.
This makes Java faster and more efficient than most interpreted languages yet also gives Java the flexibility of being portable across different architectures.
While the JIT compiler continuously runs, it performs optimizations on your code. This essentially "refactors" source code to optimize it for lower level processing.
