Modern Java with Spring AI 2.0

Jul 23

Source code for this article: https://github.com/Pointer-Tech/spring-ai-example

AI is undeniably transforming the software industry and Python is often thought of as the language of AI. Python is widely used in the data science and physical science communities and it's notoriously easy to use, making it a great candidate for projects that need prototypes to be built quickly. At the same time, Java has dominated as the scalable enterprise language of choice for decades. With Python's popularity among developers working with AI, there are questions about where Java fits into the new software landscape as AI applications become ubiquitous. In this article we will explore how Spring AI 2.0 will enable Java to be the prefered language for the future of enterprise AI applications.

An analysis of SDKs available for MCP implementations showed that Java outperforms Python by 30x on average latency. Java's latest LTS releases offer quality of life improvements, such as virtual threading and records. Java is evolving as a language in ways that ensure that it should remain the de facto standard for fast and maintainable code at scale.

Java's popular framework, Spring, is already a top choice for developers who need to bootstrap production-grade web APIs quickly and now Spring AI offers features to connect your enterprise data and APIs with AI models. Their recent version 2.0 release expanded the language's AI features, making it easier than ever for developers to integrate agentic features into their Spring applications.

To get an idea of how easy it is to create an AI-enabled Spring REST API, let's take a look at some examples of how Spring AI works. To start, let’s create a simple endpoint that can tell a joke.

Config

  
    
spring:
 application:
   name: "Hello Spring AI"
 ai:
   openai:
     api-key: ${API_KEY}
     chat:
         enabled: true

  
  

Controller

  
@RestController
public class HelloController {

   private final ChatClient chatClient;

   public HelloController(ChatClient.Builder chatClient) {
       this.chatClient = chatClient.build();
   }

   @GetMapping("/joke/{topic}")
   public String hello(@PathVariable("topic") String topic) {
       return chatClient.prompt(String.format("Tell me a joke about %s", topic)).call().content();
   }
}

And the endpoint response with a topic of robots ...

  
Why was the robot so upset? People kept pushing its buttons.

Want another one? What do you call a pirate robot? Arrr2-D2.

* The example uses Spring Boot 4.0.5, Spring AI 2.0, OpenAI, and Java 25. Quickly bootstrap an application with the Spring Boot initializer

Less than 15 lines of code and some configuration and we are ready to laugh. Entertainment varies with model selection! The same principles of Spring apply to Spring AI. Spring handles the boiler-plate, the developer handles the business logic.

Spring AI is feature-rich, but starting with the basics is important. Large language models are non-deterministic by design, the same prompt can elicit two completely different responses. This can prove problematic when trying to marshal responses into java objects. Lets extend our joke application with Spring AI’s structured output support to marshall an object.

  
@GetMapping("/jokeObject/{topic}")
public Joke jokeObject(@PathVariable("topic") String topic) {
   return chatClient.prompt(String.format("""
           Tell me a joke about %s. Give it a rating from 0 to 10.
           Come up with some commentary about why it was rated that,
           and make it comical.
           """, topic)).call().entity(Joke.class);
}

public record Joke (String topic, String joke, int rating, String commentary){}

And the response ...

  
{
  "topic": "robots",
  "joke": "Why did the robot go on a diet? It had too many bytes.",
  "rating": 7,
  "commentary": "I give this one a 7 out of 10: it's punny, efficient, and perfectly byte-sized. It loses points for being an old classic that made at least two motherboards roll their LEDs, but gains points for causing minimal system lag and maximum eye-roll efficiency. Verdict: solid maintenance-level humor -- reboot your smile and try not to snort oil."
}

The object is created via a built-in system prompt that spring provides to “encourage” the model to respond in JSON. This works well most of the time, but there are no guarantees the model responds correctly. In enterprise application development, most of the time is not good enough. To make this application have an acceptable level of risk, a Spring AI advisor is needed.

An advisor is similar to a web request interceptor for the ChatClient. An advisor will be executed after the client response and can manipulate the call chain. Custom advisors can be created by extending the CallAdvisor or StreamAdvisor interface, but the Spring AI provided StructuredOutputValidationAdvisor is all that’s needed to make object marshalling production ready. When this is configured, it will have the client call retry a number of times to marshal the output. A successful marshal can’t be guaranteed, but the likelihood of failure is drastically reduced to the point where it should be an acceptable risk. This is an example of a recursive advisor that can reinvoke the model. Some other built in advisors include a logging advisor, chat memory advisor, and a tool calling advisor.

The tool calling advisor is an incredibly powerful feature. A Spring AI tool is an annotated method that is exposed to a model that it can decide to leverage based on its description. Since models are stateless they can't respond with what day of the week it is or what the weather is going to be in the next few days. Take a look at this response from the model when asked about what to wear today.

  
Do you want a general, season-based suggestion or a recommendation tied to 
the current live forecast? I can give a useful, safe outfit plan 
immediately; if you want an exact match for today's weather tell me the 
current temperature/conditions where you are or say you want me to tailor 
by time of day and activity.

The model has no idea what the current weather is. What will happen when it has a tool that can provide it that state?

The tool description will be passed into the chat client context which enables the model to leverage its functionality. When the client is asked what to wear in Barcelona tomorrow, it will use the tool to determine the temperature, a stateful ask. When the temperature is returned from the tool, the chat client can make its recommendation.

Controller

  
@GetMapping("/whatToWear/{city}")
public String whatToWear(@PathVariable("city") String city) {
   return chatClient.prompt(String.format("""
           Make a recommendation on what to wear in %s today
           """, city)).tools(new WeatherTools()).call().content();
}

Tool Class

  
public class WeatherTools {
   @Tool(description = "Returns the weather prediction for the current day in Barcelona")
   public String weather() {
       System.out.println("Calling the weather tool");
       return "The weather will be 70 degrees F with a slight chance of showers in the afternoon";
   }

Application logs after calling the weather endpoint

And the response for the city of Barcelona ...

  
    
Today in Barcelona it's about 70°F (≈21°C) with a slight chance of 
afternoon showers. Recommendation:
- Layers: a light T‑shirt or breathable top (cotton/linen or a
moisture‑wicking tee) with a light sweater or thin cardigan you can remove
if it warms up. 
- Outerwear: a packable waterproof jacket or a light trench. A small 
umbrella or a compact rain shell is wise because of the afternoon shower 
risk.
- Bottoms: chinos, jeans or casual trousers; or a skirt/shorts if you 
prefer (bring the cardigan/jacket).
...

  
  

Tool calling is how the model can leverage a customer's business model to answer useful questions in real time. But this can be a dangerous proposition, there are no guard rails in place to prevent the AI from exposing the data in its tool set. For instance, a clever adversary could abuse the chat client to expose potentially sensitive information if it is not gated and access controlled. This is leading to a few missing features within the Spring AI project.

Spring Security hasn’t been fully integrated into the AI project. The pace at which AI has evolved made it difficult to keep Spring AI on the cutting edge. There is only so much developer capacity to go around. To help bridge the gap before an official integration, the Spring AI team has embraced several community solutions, such as the MCP Security project.

Another hurdle is limited client customization, as there isn’t a unified way to configure the underlying web clients used by the various vendor SDKs. This can prove problematic, for example if the model the app will interact with is sitting behind a proxy that requires mutual tls authentication. The solution requires some vendor specific configuration, but the higher level ChatClient is still agnostic. For OpenAI, registering a customizer bean allows deeper configuration options such as setting an SSL context.

  
@FunctionalInterface
public interface OpenAiHttpClientBuilderCustomizer {
    void customize(SpringAiOpenAiHttpClient.Builder builder);
}

As the Spring AI project matures, these pain points will ease. The security support and client customization are on the Spring AI team’s radar,

What does all this mean? A software developer wants to provide the best tool to their users while making it as simple as possible for them to use it. Can Claude or other agentic cli tools perform the same tasks Spring AI can? The answer is yes, after hours of configuration, setting up MCP servers, skills, prompts, the list goes on. Spring AI drastically reduces that setup time for the developer, and completely removes it for the customer. The chat client could grow into a fully agentic service with limitless possibilities. It can expose a database safely and let the model extrapolate on the data. The Spring AI abstractions also save the customer from vendor lock-in, providing maximum flexibility.

As AI continues to revolutionize the way that software is written and used, Spring AI makes a strong case that Java and Spring should continue to be the tools of choice for implementing enterprise software solutions in the future.

JavaSpringAISpring AI

Thomas Evans