Using GraalVM & AWS Lambda in Java for cold start problems
We look into the GraalVM, AWS Lambda, and Java’s cold start problem and suggest a solution to it.
Intro
EPAM isn't just a platform with remote IT jobs. It's a vibrant and supportive community of software engineers, quality assurance specialists, UX designers, and other professionals. We’ve never found an issue they didn't know a solution to — our tutorials library speaks for itself!
For this tutorial, we asked Aleksandr Filichkin, Lead Software Engineer and Technical Lead at EPAM Anywhere, to share his practical experience with Java and AWS. Aleksandr is a big AWS fan with five years of production experience and AWS certification.
He'll explain how to use GraalVM and AWS Lambda in Java to solve the cold start problem.
Prerequisites
We'll test a REST service that saves queries to a DynamoDB database, using AWS Lambda. It's a viable solution since AWS Lambda offers terrific scalability and Java support. Besides, we only have to pay for usage, which is budget-friendly.
Now let's see how we can use AWS Lambda with GraalVM to resolve Java’s cold start problem and significantly improve performance.
What is AWS Lambda?
AWS Lambda is a compute service that allows you to run code for almost any type of application or backend service and handles all of the administration processes, which includes the maintenance of server and operating system. It's a budget-friendly solution because AWS Lambda only charges you for the time when the service is running. In addition, AWS Lambda offers great speed and performance capabilities, and seamlessly integrates with other AWS services.
Here's a scheme of the AWS Lambda operating cycle:
REST services implementation with AWS Lambda
Version 1: plain Java w/o improvements
- Java 11
- AWS SDK-V2 for DynamoDB (extended DynamoDb client)
- No DI (Spring, Dagger, etc)
- No special frameworks
Check the code implementation on GitHub.
The handler is LambdaV1.java
Result:
- Duration: 10845.21 ms
- Billed Duration: 10846 ms
- Memory: 256 MB
- Max Memory Used: 168 MB
- Init Duration: 2650.86 ms
Version 2: plain Java with improvements
- Java 11
- AWS SDK-V2 for Dynamodb
- No DI (Spring, etc)
- No special frameworks
- Utilize CPU burst on startup (move everything to static, warm-up DynamoDB client)
- Reduce dependencies (exclude Netty)
- Specify AWS Regions
- Specify Credential Provider
Also, check out the handler LabmdaV2.java
Result:
- Duration: 4037.08 ms
- Billed Duration: 4038 ms
- Memory: 256 MB
- Max Memory Used: 170 MB
- Init Duration: 3604.04 ms
As you can see, the billable time has been reduced by 2.5 times. Let's review our final solution with AWS Lambda custom runtime and GraalVM.
What is AWS Lambda Custom runtime?
AWS Lambda Custom runtime was introduced in 2018. The runtime comes in a function.zip file with a bootstrap shell script or binary executable file (compiled for Amazon Linux). You can implement runtime in any programming language and include it in your function's deployment package in the form of an executable file.
What is GraalVM?
GraalVM is a Java and JDK virtual machine based on HotSpot/OpenJDK. Out of the box, the tool offers fast Java execution, execution of programs written in platform-dependent languages, support for multiple programming languages, JVM application augmentation, and more.
GraalVM comes in two editions:
Community edition. This version is available for free, and you can use it even in commercial projects. It's built from the GraalVM sources available on GitHub. The community edition provides distributions based on OpenJDK 11 for Linux, macOS, and Windows platforms on x86 64-bit systems, and for Linux on ARM 64-bit systems.
Enterprise edition. This edition provides additional performance, security, and scalability. It's an optimal choice for applications in production.
AOT vs JIT: Startup Time
JIT:
- Load JVM executables
- Load classes from the file system
- Verify bytecodes
- Start interpreting
- Run static initializers
- First-tier compilation
- Gather profiling feedback
- Second tier compilation (C2 or GraalVM)
- Finally, run with the best machine code
AOT:
- Load executable with a prepared heap
- Immediately start with the best machine code
Version 3: AWS Lambda Custom Runtime + GraalVM
Check the code implementation here.
To build a native binary, let's use Docker: GraalVM-Dockerfile
Result:
- Duration: 372.73 ms
- Billed Duration: 704 ms
- Memory: 256 MB
- Max Memory Used: 90 MB
- Init Duration: 330.61 ms
GraalVM Native drawbacks:
Even though GraalVM can be a lifesaver when it comes to Java's cold start problem, it comes with some limitations, including:
- Manual/explicit mapping for reflections
- Not all libraries can be compiled (closed-world assumption)
- Slow (CPU intensive) build time
- Large size of binary file
- Only Serial GC is supported for the GraalVM CE version
Useful GraalVM tips:
- Use JVM agentlib to track all usages of dynamic features of an execution on a regular Java VM:$JAVA_HOME/bin/java -agentlib:native-image-agent=config-output-dir=path
- Use Dashboard to analyze the binary file
- For logger, use slf4j-simple
- Use UPX if the binary file is big (AWS Lambda limit is 250 MB)
- Use Quarkus, Micronaut, etc.
Wrapping up
Now let's summarize our tutorial. As you can see, GraalVM solves the Java cold start issue with its amazing performance capabilities. However, we can achieve great results only with additional explicit configuration. GraalVM also comes with several limitations. For example, it compiles with a limited list of libraries and isn't suitable for large enterprise projects.
As for the warm-up state, Aleksandr says he has sent approximately 10,000 requests to the single instance of Lambda V2 (Java optimized) and Lambda V3 (GraalVM). GraalVM has constant great performance with approximately 7ms. Java demonstrates inferior performance at the beginning and then becomes ~15 ms, which looks like the Second-tier JIT optimization wasn't applied.
Useful links
AWS Lambda battle: x86 vs ARM(Graviton2) by Aleksandr Filichkin
Repository for the tutorial on GitHub
GraalVM native images explained by Oleg Šelajev