Using Deep Java Library to do Machine Learning on SpringBoot

Many AWS customers—startups and large enterprises—are on a path to adopt machine learning and deep learning in their existing applications. The reasons for machine learning adoption are dictated by the pace of innovation in the industry, with business use cases ranging from customer service (including object detection from images and video streams, sentiment analysis) to fraud detection and collaboration.

However, until recently, the adoption learning curve was steep and required development of internal technical expertise in new programming languages (e.g., Python) and frameworks, with cascading effect on the whole software development lifecycle, from coding to building, testing, and deployment. The approach outlined in this blog post enables enterprises to leverage existing talent and resources (frameworks, pipelines, and deployments) to integrate machine learning capabilities.

Introduction

Spring Boot, one of the most popular and widespread open source frameworks for microservices development, has simplified the implementation of distributed systems.

Despite the broad appeal of this framework, there are few options to easily integrate it with Machine Learning (ML). Existing solutions such as stock APIs often do not meet customized application requirements, and developing customized solutions is time-consuming and not cost-effective.

Developers approached the integration of machine learning capabilities into existing applications in a number of ways. Taking inference as an example, current options vary from using stock API to having a Python or C++ based application wrapped with an API for remote calls. Stock API, though based on robust models, may not quite fit your domain or industry, causing problems that will be discovered in production and few options to address them. In other cases, when running inference at scale (for example, in streaming applications or latency-sensitive microservices), making a remote call may not be a viable option for performance reasons.

Recognizing this challenge, we at AWS have created a few open source projects to facilitate the adoption of ML for Java and microservices, and ultimately to help our customers, partners and the open source community as a whole. These initiatives align closely with the AWS goal to take technology that was traditionally cost-prohibitive and difficult for many organizations to adopt, and make it accessible to a much broader audience.

In this blog post we will demonstrate how Java users can integrate ML into their Spring applications with Spring Boot Starter for Deep Java Library (DJL). We will review how to apply these frameworks in action and integrate ML capabilities into a microservice, demonstrating common deep learning use cases around object detection and classification.

DJL overview

Deep Java Library (DJL) is an open source, high-level, framework-agnostic Java API for deep learning. It is designed to be easy to get started with and simple to use for Java developers. DJL provides a native Java development experience and functions like any other regular Java library.

DJL provides a convenient abstraction layer for using the most popular AI/ML frameworks such as Apache MXNet, PyTorch, and TensorFlow. However, it is not just a convenience on top of the existing libraries (some of which provide Java API / bindings). With DJL API, you are getting a uniform and consistent layer that can interact with all of these frameworks, allowing you to swap out the framework of your choice without any impact to the client code.

This unique feature, in combination with a fairly rich model zoo repository (a repository with pre-trained models), can enable ML engineers to find optimal models for the task at hand regardless of the underlying model implementation.

For more information on DJL please refer to the DJL GitHub Repository and FAQ.

DJL Spring Boot Starter

Spring Boot Starter is a one-stop shop for all Spring and related technologies that you need in your project, without having to hunt through sample code and copy-paste loads of dependency descriptors. Please see the official Spring Boot documentation for more information on starters.

Following this definition, DJL Spring Boot Starter provides all dependencies required to start using DJL in Spring as a single artifact. In addition to dependency management, the starter includes an auto-configuration that makes it possible to automatically wire dependencies based on the configuration file supplied by the user, and make them available as beans in the Spring Application context.

Dependency management

The DJL library is platform-specific, but it provides ways to automatically look up the correct dependency based on the target operating system. DJL can also be configured with different underlying engines (such as MXNet, PyTorch, or TensorFlow); the user is expected to make this choice before the starter is used. However, even after the choice is made, the underlying engine as well as the target operating system architecture can be changed by modifying your Maven (or Gradle) dependency, with no impact to your code.

Starter dependency management is organized so as to provide the most flexibility to the user.

For the MXNet starter, the following operating system classifiers are supported: osx-x86_64 for Mac OS X, linux-x86_64 for generic Linux, win-x86_64 for Windows distributions, and auto for automatic detection of the target operating system. The last option requires connectivity to the external artifact repository (e.g., Maven Central) at runtime, which may be an issue for systems with tight security constraints and restricted egress.

Here’s an example of MXNet Dependency for Linux architecture, optimized for container workloads:

AWS

Auto dependency that will download the correct artifact at runtime:

AWS

Using PyTorch as an underlying engine, the starter dependency is:

AWS

Gradle dependencies will look similar. It is important to set the JNA version as "jna.version=5.3.0" inside your gradle.properties, since the Spring Boot parent POM uses an older version of JNA, which will not work with the DJL starter. Here is an example of your Gradle build file build.gradle.kts(Kotlin DSL is used in this example), assuming the Spring Boot plugin is registered:

AWS

Spring auto-configuration

Once dependencies are configured correctly in your Spring Boot application, the next step is to configure your beans and wire them properly for injection. It is fairly easy to configure DJL-related beans and make them available in the Spring application context, but it requires internal knowledge of the library as well as the peculiarities of individual classes for proper scoping—some beans are thread-safe, others should be scoped per request/thread. To assist with this configuration, the DJL Spring Boot starter provides an auto-configuration.

This component is separate from the dependency component and requires an explicit dependency. We did it this way for a couple of reasons:

1. Some developers prefer to have full control over configuration options and may not want the Spring “auto magic”. In such cases, the starter will support just the basic set of dependencies and allow developers to wire components explicitly.
2. The auto-configuration component is generic for all kinds of the DJL configurations: Regardless of the underlying target operating system or the actual engine, the auto-configuration component remains the same. So, using the same auto-configuration, developers can swap underlying dependencies as a single step operation without any impact to the code.

Declaring dependency on auto-configuration in Maven:

AWS

Or in Gradle build.gradle.kts:

AWS

Once the dependency is declared, the Spring Boot framework will automatically locate the configuration and wire the required components. At present, for inference it will look up the model from the model zoo repository and create a predictor that will be readily available to run inference.

Users are expected to supply a standard Spring configuration (application.yml or application.properties) model to use one of the supported application types:

AWS

For example, in order to run object detection in images, the user can set the application type to OBJECT_DETECTION. DJL-related configuration should be namespaced under djl root, for example djl.application-type=OBJECT_DETECTION if application.properties is used.

Here is an example of a yaml configuration for DJL auto-configuration:

AWS

IDE support

Configuration creation does not have to be fully manual—the DJL Spring Boot Starter provides a configuration content assistant for most IDEs with the help of Spring IDE plugins (tested on IntelliJ IDEA, but expected to work in Eclipse with STS and NetBeans IDE).

For IntelliJ, you can use Ctrl+Space for auto-completion and Ctrl+J for quick doc on any property.

Simple application walkthrough

Below is a code example based on a Simple Spring Boot application that demonstrates single-shot object detection with DJL and MXNet.

Dependencies are set up per the Maven dependency section. Configuration setup is identical to the application.yml in the example. The application is a regular console Spring Boot application with a single class (note that no other code is required).

Injecting a predictor for object detection

AWS

Using predictor in a try-with-resources block to make sure it is closed after each use is recommended, hence the supplier of predictor is injected for convenient instantiation.

Object detection in Java

AWS

The above code will run object detection on the supplied image (expected to be in the class path) and output the results to the logger in the following form:

AWS

Building and running

Assuming you have checked out the repository and are now in the root of the repository:

AWS

Alternatively, you can run it directly with java -jar command:

AWS

More complex application example

Below is a code example based on a REST API example leveraging DJL Spring Boot Starter that demonstrates a RESTful API implementation that can take images from an Amazon Simple Storage Service (Amazon S3) bucket and stores the object detection results back in S3. This API represents a microservice that can take any image reference (uploaded to a predefined Amazon S3 bucket) and run object detection on it. It could be leveraged directly by REST clients like Postman, or within a mobile or web application.

The API portion is using Gradle as its build system and leverages Spring MVC for REST API implementation. For simplicity, the controller is implemented as a blocking call. For high-volume production usage, it is recommended to use a reactive style of API implementation such as WebFlux. There is an example of an explicit programmatic configuration of DJL components in your Spring Boot application.

The web application portion is implemented as a Kotlin Spring MVC application with a reactive REST API client to invoke the backend API.

The user is presented with a list of files and an option to select any file for upload and object detection, the API output is displayed directly below:

AWS

The result of the object detection is applied back to the image, highlighting the areas where objects were detected and uploaded to an S3 bucket:

AWS

In subsequent posts, we will review how to containerize these applications and organize a full CI/CD pipeline deploying to Amazon Elastic Kubernetes Service (Amazon EKS).

Conclusion

Adopting machine learning with DJL and Spring Boot is a simple and powerful approach that enables customers to combine an existing battle-tested microservice technology stack and the most proven deep learning frameworks like MXNet, PyTorch, and Tensorflow through the convenient abstraction layers provided by DJL and Spring Boot.

References

1. DJL Spring Boot Starter—Spring Boot starter is open source software released under the Apache 2.0 license that allows Spring Boot developers to start using DJL for inference.
2. DJL Spring Boot Demo—Demo repository containing Java API that leverages the starter and Gradle as well as a small Kotlin-based web application.
3. DJL—DJL (Deep Java Library) main repository on GitHub, released under Apache 2.0 license.