Benchmarking Java vs. C# in AWS Lambda
Java and C# are both available as runtime languages for AWS Lambda. Both are object oriented, implement garbage collection and are a top choice for serious software developers.
This article will be comparing a simple compute workload written in both languages and to see how they compare with respect to compute performance.
Additionally, I will also present results for the new Graviton2 based architecture as compared to the x86_64 (previous AWS Architecture). For more on Graviton2 lambdas, please refer to https://aws.amazon.com/blogs/aws/aws-lambda-functions-powered-by-aws-graviton2-processor-run-your-functions-on-arm-and-get-up-to-34-better-price-performance/
Currently, these are the following choices of Java and DotNet C# runtimes that are available for AWS:
For this comparison, we will compare the latest runtimes of each. This means comparing Java 11 Lambda runtime vs. the DotNet 3.1. I hope it won’t be long for AWS to implement DotNet 5 for Lambdas but it is not currently available at the time of the writing of this article.
Workload description
The workload presented is identical for both Java and C#. The workload takes a little over 30 seconds of compute time. It is essentially 2 loops, calculating a square root and appending it to a string.
Java workload:
public static void executeCPUIntensiveTask()
{
//some cpu intensive task
long limit = 5000;
String lastSqRoot = "";
for (int i=0;i<limit;i++){
for (int j=0;j<limit;j++){
double sqroot = Math.sqrt(j);
lastSqRoot = ""+sqroot;
}
}
System.out.println("lastSqRoot="+lastSqRoot);
}C# workload:
private static void executeCPUIntensiveTask(){
//some cpu intensive task
long limit = 5000;
String lastSqRoot = "";
for (int i = 0; i < limit; i++)
{
for (int j = 0; j < limit; j++)
{
double sqroot = Math.Sqrt(j);
lastSqRoot = "" + sqroot;
}
}
Console.WriteLine("lastSqRoot=" + lastSqRoot);
}For more on how the code was used to generate the results please refer to:
Results
We can see in our AWS Console, all 4 Lambdas (DotNet 3.1 x86_64, DotNet 3.1 arm64(Gravition2), Java 11 x86_64, Java 11 arm64 (Graviton2)
It is interesting to note that the Code size for the DotNet 3.1 Lambdas is about a third of the size of the Java 11. This probably means that deploying DotNet Lambdas is quicker.
I have run these Lambda configurations in the previous implementation articles but I will run a new series of tests to hopefully confirm and compare the duration.
The following results were obtained:
The results fall in line with what I have previously measured.
Findings
DotNet 3.1 Lambdas have a much smaller code size than Java 11 Lambdas which are about a third the size of the Java code sizes. This means that DotNet Lambda functions will be quicker to deploy than Java 11 Lambda functions.
For Java 11, upgrading to the Graviton2 from x86_64 architecture results in Lambdas which run 2.09 (109.28%) times faster.
The DotNet 3.1(C#) x86_64 Lambdas and DotNet(C#) 3.1 Graviton 2 Lambdas perform similarly with the difference being less than 1%. It was odd that the Graviton2 averaged out to be slightly slower. This was discussed with AWS and they confirmed this and was pointed out that customers can still benefit from the cost savings of the Graviton2 based Lambdas.
Java 11 x86_64 Lambdas are 7.30% faster than DotNet 3.1 x86_64 Lambdas
Java 11 Graviton2 Lambdas are 2.27 (127.21%) times faster than DotNet 3.1 Graviton2 Lambdas.
Java 11 is faster than C#(DotNet 3.1) as a Lambda runtime. (Compute based workload)
Conclusion
It is interesting to do this comparison to the 2 languages in the AWS Lambda environment. It would be interesting to do a comparison when support for DotNet 5 for AWS Lambda comes out eventually and see how that compares.
AWS states that the move to Graviton2 based lambdas will result in up to 34% improvement in price performance. If a Lambda takes less time to run, the billed duration will also be lower.
All Lambdas running in the Graviton2 architecture will benefit from a price improvement of about 20%. Further information can be seen at https://aws.amazon.com/lambda/pricing/
For DotNet(C#), if you make the switch from DotNet 3.1 x86_64 to DotNet 3.1 Arm, the performance is similar with less than 1% performance difference but you will save about 20% on your Lambda compute time.
For Java 11, if you make the switch from Java 11 x86_64 to Graviton, the billed duration will end up being 2.09 times lower and in addition you will get about 20% off the Lambda compute time. This results in faster and cheaper Lambdas. This results in a theoretical cost reduction of about 61.72% when you consider a lower billed duration and the cheaper arm(Graviton2) price.
Both are great software development languages. It must be stated that this test was for compute only without taking into consideration other items such as networking, database access, interacting with other services.
Hopefully this article, illustrated a useful comparison with respect to comparisons between Java 11, C#(DotNet 3.1) and the different architectures(Graviton2(arm64) and x86_64) when implementing AWS Lambdas.
Do not hesitate to reach out to us at Xerris.com for more information about your organization’s cloud and digital transformation needs and how we can help you.
