Performance Tuning in Production

James Gough
Sadiq Jaffer
Kirk Pepperdine
Richard Warburton

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Session Overview
• Optimizing Java: a brief tour of the JVM
• Moving to G1GC
• Production Profiling: What, Why and How

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

Optimizing Java: A JVM Tour

James Gough

@Jim__Gough
http://jamesgough.net

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 This Talk

• Who Am I

• Creating Bytecode

• Classloading

• Profiling Code

• Runtime Optimisations

• JITWatch

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 About Me
• Started programming BASIC on the C64

• Worked as a Java and Web Developer

• Helped to design and test JSR 310

• Spent 4 years training Java and C++

• Written a book called Optimizing Java

• Work at Morgan Stanley

• Building Client Facing Technology

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Creating Bytecode

JVM
+

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Creating Bytecode

JVM
+

=
Java
source
code

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Creating Bytecode

JVM

javac

Java
source
code

Class file creation

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe
 Creating Bytecode

JVM
.class
file

javac

Java
source
code

Class file creation

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe
 The anatomy of a classfile

Magic Number 0xCAFEBABE

Version of Class The minor and major versions of the class file
File Format
Constant Pool Pool of constants for the class

Access Flags For example whether the class is abstract, static, etc.

This Class The name of the current class

Super Class The name of the super class

Interfaces Any interfaces in the class

Fields Any fields in the class

Methods Any methods in the class

Attributes Any attributes of the class (e.g. name of the sourcefile, etc.)

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 The anatomy of a classfile

My Very Cute Animal Turns Savage In Full Moon Areas

M V C A T S I F M A
Magic Version Constant Access This Super Interfaces Fields Methods Attributes

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Type Descriptors

• Describe signatures Descriptor Type

B byte
• Common in javap output
C char
• E.g. D double

• ()Ljava/lang/String; F float

I int
• (I)V
J long
• (Ljava/lang/String;I)J
L<type>; Reference type

S short

Z boolean

[ Array-of

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 How Bytecode is executed

JVM
.class
file classloader

javac

Java
source
code

Class file creation Continuous (Just In Time) compilation

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe
 Classloaders

• Classes are loaded just before they are needed

• proven by the painful ClassNotFoundException

• Loads classfile into the Class object

• mechanism for representing classes in the VM

• Example used in watching-classloader

• https://github.com/jpgough/watching-classloader

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Interpreting Bytecode

JVM
.class classloader method cache
file

javac JVM
Interpreter

Java
source
code

Class file creation Continuous (Just In Time) compilation

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe
 Interpreting Bytecode

• Bytecode initially fully interpreted

• Conversion of each instruction to machine instruction

• Time not spent compiling code that is only used once

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 How does Interpreting Help?

• Provides the opportunity to observe code execution paths

• may not be the same for each execution of the app

• The profiler observes the execution and looks for the best
optimisations

• Code is compiled after hitting a threshold

• Configurable

• JVM can revert optimisation decisions

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Profiling Code

JVM
.Class classloader method cache
file

JIT compiler

javac Profile Guided

Interpreter
Optimisation

Java
source
code

Class file creation Continuous (Just In Time) compilation

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe
 Profiling Code

• Looking for loops or frequent execution of code blocks

• Barometer used to count the number of executions

• Threshold is reached and mode changes to tracing

• Tracing follows the execution path involving that method

• proactively looking for optimisation opportunities

• often stored as an intermediate representation

• traces are used in the code generation phase

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 The Hotspot JVM

JVM
.Class classloader method cache
file

JIT compiler

javac
Interpreter Profiler Emitter

Java
source
code

Class file creation Continuous (Just In Time) compilation

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe
 Viewing Code Compilation

java -XX:-TieredCompilation -XX:+PrintCompilation HelloWorld 2> /dev/null

Time
Task Method Name (size of compiled code)
Offset
321 40 sun.nio.cs.StreamEncoder::isOpen (5 bytes)
322 41 sun.nio.cs.StreamEncoder::implFlushBuffer (15 bytes)
327 42 sun.nio.cs.StreamEncoder::writeBytes (132 bytes)
331 43 ! java.io.PrintStream::write (69 bytes)
335 44 s java.io.BufferedOutputStream::write (67 bytes)
337 46 java.nio.Buffer::clear (20 bytes)
337 47 java.lang.String::indexOf (7 bytes)
338 48 ! java.io.PrintStream::println (24 bytes)
338 49 java.io.PrintStream::print (13 bytes)
343 50 ! java.io.PrintStream::write (83 bytes)
346 51 ! java.io.PrintStream::newLine (73 bytes)
347 52 java.io.BufferedWriter::newLine (9 bytes)
347 53 % HelloWorld::main @ 2 (23 bytes)
! method has exception handler(s)
s method declared synchronized
n native method (no compilation, generate wrapper)
% on-stack replacement used

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Inlining

• Calling a method has an overhead

• creation of a new stack frame

• copying values required to the stack frame

• returning from the stack frame post execution

• Consider a method call in a for loop

public class HelloWorld {
public static void main(String[] args) {
for(int i=0; i < 100_000; i++) {
System.err.println("Hello World");
}
}
}

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Inlining

java -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining HelloWorld 2> /dev/null

@ 40 java.io.BufferedOutputStream::flush (12 bytes) inline (hot)

\-> TypeProfile (19272/19272 counts) = java/io/BufferedOutputStream
@ 1 java.io.BufferedOutputStream::flushBuffer (29 bytes) inline (hot)
@ 20 java.io.FileOutputStream::write (12 bytes) inline (hot)
\-> TypeProfile (4696/4696 counts) = java/io/FileOutputStream
@ 8 java.io.FileOutputStream::writeBytes (0 bytes) native method
@ 8 java.io.OutputStream::flush (1 bytes) inline (hot)
\-> TypeProfile (7047/7047 counts) = java/io/FileOutputStream
!m @ 13 java.io.PrintStream::println (24 bytes)
@ 6 java.io.PrintStream::print (13 bytes)
!m @ 9 java.io.PrintStream::write (83 bytes) callee is too large
!m @ 10 java.io.PrintStream::newLine (73 bytes) callee is too large
!m @ 13 java.io.PrintStream::println (24 bytes)
@ 6 java.io.PrintStream::print (13 bytes)
!m @ 9 java.io.PrintStream::write (83 bytes) callee is too large
!m @ 10 java.io.PrintStream::newLine (73 bytes) callee is too large
!m @ 13 java.io.PrintStream::println (24 bytes) already compiled into a big method

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Constant Subexpression Elimination

• Compiler hunts through code for common expressions

• if results analyses replacement with a single variable

• Relies on data flow analysis of the program

• which is done during the profiling and tracing part

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Dead Code Elimination

• Removes code that is never executed

• shrinks the size of the program

• avoid executing irrelevant operations

• Dynamic dead code elimination

• eliminated base on possible set of values

• determined at runtime

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Register Allocation

• Identification of variables suitable for registers

• to avoid cache misses

• improve execution speed of the program

• Uses data from the trace to make informed decision

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Loop-Invariant Code Motion

• Involves removal of code from loops

• for code that doesn’t impact the outcome of the loop

• moved above the loop to avoid unnecessary execution

• Hoisted code can now be cached in a register

• improving performance of the loop execution

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Escape Analysis
• Introduced in later versions of Java 6

• Analyses code to assert if an object reference

• returns or leaves the scope of the method

• stored in global variables

• Allocates unescaped objects on the stack

• avoids the cost of garbage collection

• prevents workload pressures on Eden

• beneficial effects to counter high infant mortality GC impact

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe
 Loop Unrolling

private static final String[] RESPONSES = 


{ "Yes", "No", "Maybe" };

public void processResponses () {

for ( String response: RESPONSES ) {
process(response);
}
}

private static final String[] RESPONSES = 


{ "Yes", "No", "Maybe" };

public void processResponses () {

process(RESPONSES[0]);
process(RESPONSES[1]);
process(RESPONSES[2]);
}

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Loop Unrolling
@Benchmark
public long intStride1()
{
long sum = 0;
for (int i = 0; i < MAX; i++)
{
sum += data[i];
}
return sum;
}

@Benchmark
public long longStride1()
{
long sum = 0; Benchmark Mode Cnt Score Error Units
for (long l = 0; l < MAX; l++) LoopUnrollingCounter.intStride1 thrpt 200 2423.818 ± 2.547 ops/s
{ LoopUnrollingCounter.longStride1 thrpt 200 1469.833 ± 0.721 ops/s
sum += data[(int) l];
}
return sum;
}

Excerpt From: Benjamin J. Evans, James Gough, and Chris Newland. “Optimizing Java.” iBooks.

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Loop Unrolling
• Can unroll int, char and short loops

• Can remove safe point checks

• Removes back branches and branch prediction cost

• Reduces the work needed by each “iteration”

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Monomorphic Dispatch

• When HotSpot encounters a virtual call site, often only one type
will ever be seen there
• e.g. There's only one implementing class for an interface

• Hotspot can optimize vtable lookup

• Subclasses have the same vtable structure as their parent

• Hotspot can collapse the child into the parent

• Classloading tricks can invalidate monomorphic dispatch

• The class word in the header is checked

• If changed then this optimisation is backed out

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Code Cache

JVM
.Class classloader method cache
file

JIT compiler

javac
Interpreter Profiler Emitter

Java
source code cache
code

Class file creation Continuous (Just In Time) compilation

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe
 Code Cache

• The code cache contains the JIT native compiled code

• Code is JIT'd on a per method basis

• 1. This occurs when an entry counter is exceeded

• 2. Internal Representation (IR) is built

• 3. Optimisations are applied

• 4. JIT turns IR into native code

• Pointers are swizzled to use the native code

• native code is executed on the next call

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Introduction to JITWatch

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Summary

• Java has carried a brand name of being slow

• Java can emit instructions comparable to C++

• javac doesn’t do much optimisation

• We can make better decisions from profiling at runtime

• JITWatch makes life easier

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe

 Performance Landscape
Hardware
JVM
.Class classloader Garbage
method cache
file Collection
JIT compiler

javac Executing
Interpreter Profiler Emitte Code Quality

Java
source code cache
code

Databases/Networks/IO bound operations

#PerfTuningInProd @Jim__Gough, @opsian, @RichardWarburto, @kcpeppe