{"id":9727,"date":"2024-09-04T11:27:39","date_gmt":"2024-09-04T11:27:39","guid":{"rendered":"https:\/\/ciit-training.com\/?p=9727"},"modified":"2024-09-04T11:58:41","modified_gmt":"2024-09-04T11:58:41","slug":"spring-ai-getting-started-with-large-language-models-in-java-spring","status":"publish","type":"post","link":"https:\/\/ciit-training.com\/en\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/","title":{"rendered":"Spring AI \u2013 getting started with large language models in Java Spring"},"content":{"rendered":"\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Introduction to Spring AI<\/span><\/h3>

On July 24<\/span>th<\/span> 2023 Dr Mark Pollack made the first commit of the new Spring AI framework. Inspired by Python frameworks such as LangChain and LlamaIndex, Dr Pollack\u2019s intention was to bring large language models to the Spring development community. Whilst not a direct port of LangChain, Spring AI nevertheless allows developers to integrate large language models into their applications with the familiar modularity and portability of the Spring framework. The core idea behind Spring AI is connecting your Data and APIs with AI models.<\/span><\/p>

\u00a0<\/p>

Spring AI provides the necessary foundation to develop AI applications with multiple implementations to allow for easy component swapping. It now provides support for all major Model providers including OpenAI, Microsoft, Amazon, Google, and Hugging Face.<\/span><\/p>

\u00a0<\/p>

Key features of Spring AI include:\u00a0<\/span><\/p>

  • Support for Chat, Text to Image, Audio Transcription and Text to Speech models with more to come.<\/span><\/li>
  • A portable API for all AI providers with synchronous and streamed API options.<\/span><\/li>
  • Access to model specific features is also supported.<\/span><\/li>
  • Mapping of AI Model output to POJOs.<\/span><\/li>
  • Support for all major vector database providers such as Apache Cassandra, Azure Vector Search, Chroma, Milvus, MongoDB Atlas, Neo4j, Oracle, PineCone, Redis, Weaviate and many more.<\/span><\/li>
  • Portable API across Vector Store providers with SQL-style metadata filtering.<\/span><\/li>
  • Function Calling.<\/span><\/li>
  • Spring Boot Auto Configuration.<\/span><\/li>
  • ETL (Extract Transform Load) pipeline to bring your data to the AI Model.<\/span><\/li><\/ul>
    \u00a0<\/div>

    The feature set allows you to implement the typical Q&A with data concept whilst also enabling much deeper integration into your app by registering functions with the AI model. The AI Model may then request Spring AI call your method. To be clear, the AI Model cannot call functions in your app, Spring AI provides a neat wrapper in which to describe your function and the arguments it takes so that your model may request it be called by the Spring AI function registry. This opens up so much possibility, not only within your own codebase but also the possibility to call third party APIs.<\/span><\/p>

    \u00a0<\/p>

    \u00a0<\/span><\/h3>

    Connecting your Spring Application to an AI Model<\/span><\/h3>

    Let\u2019s assume you\u2019ve deployed an AI Model. We\u2019ll use the Azure OpenAI models as an example (in case you have not yet deployed a Model, a walk-through of model deployment in the Azure cloud can be found at the end of the article).\u00a0<\/span><\/p>

    If you are working in an existing project, add the following dependency to your build.gradle file:\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

    \n\t\t\t\t
    \n\t\t\t\t\t\t\t
    \n\t\t\t
    \n\t\t\t\t\n\t\t\t\t\timplementation 'org.springframework.ai:spring-ai-azure-openai-spring-boot-starter'<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e449cc1 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"e449cc1\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">If you\u2019re starting from scratch, it\u2019s really no different from starting any other project, just make sure to add the dependency relevant to the model you\u2019re using. One thing to keep in mind is that you\u2019ll need to be on at least Spring version 3.2.0 to use Spring AI.<\/span><\/p><p><span style=\"font-weight: 400;\">You will then need to add your model deployment endpoint and API key to your application.properties files as well as the names of your chat model and your embedding model.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-716cecc exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"716cecc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>spring.ai.azure.openai.api-key=yourApiKey\nspring.ai.azure.openai.endpoint=https:\/\/your-end-point.azure.com\/\nspring.ai.azure.openai.chat.options.deployment-name=nameOfYourChatModel\nspring.ai.azure.openai.embedding.options.deployment-name=nameOfYourEmbeddingModel\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b9ed79a exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"b9ed79a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">You can now start the application. If you have entered everything correctly your application will run and is connected to your AI Model.<\/span><\/p><p><span style=\"font-weight: 400;\">\u00a0<\/span><\/p><h3><span style=\"font-weight: 400;\">Rest Controller First Implementation<\/span><\/h3><p><span style=\"font-weight: 400;\">We will now create a simple rest controller endpoint to test the connection to our AI Model.<\/span><\/p><p><span style=\"font-weight: 400;\">Create a new Java Class in this package and call it something like \u201cTextGenerationController\u201d. we typically use Lombok in our applications, if you haven\u2019t already, add the Lombok dependency to your build.gradle file:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e947465 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"e947465\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>compileOnly 'org.projectlombok:lombok'\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-efe3ffc exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"efe3ffc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">We are now ready to implement out first simple rest controller. Add the following code to the TextGenerationController:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a3c4e88 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"a3c4e88\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>@RestController\n@RequiredArgsConstructor\n@CrossOrigin(\"*\")\npublic class TextGenerationController {\n\n   private final AzureOpenAiChatModel chatModel;\n\n   @GetMapping(\"\/ai\/generate\")\n   public Map generate(@RequestParam(value = \"message\", defaultValue = \"Tell me a joke\") String message) {\n       return Map.of(\"generation\", chatModel.call(message));\n   }\n}\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-83f0647 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"83f0647\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">To test your rest controller, you will need to either create a test http file (if you\u2019re using an IDE like IntelliJ) or call the endpoint from a browser or tool such as Postman. Simply call the endpoint you\u2019ve created to test the AI Model output. Java Spring Applications run on port 8080 by default so unless you\u2019ve set a different port you\u2019re API call will look as follows:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c4f06c5 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"c4f06c5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>http:\/\/localhost:8080\/ai\/generate\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5b81e07 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"5b81e07\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">If everything has worked correctly you should see a hilarious response like this:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c979fb4 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"c979fb4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>{\"generation\": \"Sure, here's one for you:\\n\\nWhy don\u2019t skeletons fight each other?\\n\\nThey don\u2019t have the guts!\"}<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-419b76e e-flex e-con-boxed e-con e-parent\" data-id=\"419b76e\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-3d49009 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"3d49009\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<h3><span style=\"font-weight: 400;\">Prompts<\/span><\/h3><p><span style=\"font-weight: 400;\">A Prompt in simple terms is an instruction to the AI Model. In the previous simple rest controller implementation the prompt took the form of a <\/span><span style=\"font-weight: 400;\">String <\/span><span style=\"font-weight: 400;\">passed with our request to the model. We used a default <\/span><span style=\"font-weight: 400;\">String <\/span><span style=\"font-weight: 400;\">to make it easy but in a real-world scenario we would use the message passed by the user.\u00a0<\/span><\/p><p><span style=\"font-weight: 400;\">But there is more to it than simple user queries, there are also System Prompts. A System Prompt is hard coded into the application and primes the behaviour of the model.<\/span><\/p><p><span style=\"font-weight: 400;\">System Prompts:\u00a0<\/span><\/p><ul><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Hard coded<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Initial input or instruction provided to AI model<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Sets the context and provides guidance to the model<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Frames the conversation or the task at hand<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Can include specific instructions, keywords, or contextual information to guide the model&#8217;s understanding and generation process<\/span><\/li><\/ul><div>\u00a0<\/div><p><span style=\"font-weight: 400;\">Lets give our AI model some personality. Create a new directory called \u201cprompts\u201d in the \u201dresources\u201d folder. Create a file called \u201csystem-prompt.st\u201d. In the file we will write the following:<\/span><\/p><p><span style=\"font-weight: 400;\">&#8222;You are a witty poet who loves to speak in rhymes, the sillier the better&#8220;<\/span><\/p><p><span style=\"font-weight: 400;\">Add the system-prompt file to the rest controller using a <\/span><span style=\"font-weight: 400;\">@Value <\/span><span style=\"font-weight: 400;\">annotation <\/span><span style=\"font-weight: 400;\">(org.springframework.beans.factory.annotation <\/span><span style=\"font-weight: 400;\">NOT <\/span><span style=\"font-weight: 400;\">Lombok)<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-13cbd7d exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"13cbd7d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>@Value(\"classpath:\/prompts\/system-prompt.st\")\nprivate Resource systemPrompt;<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-54b064d exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"54b064d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Create a new endpoint that returns the ChatResponse data type. This time we will pass a \u201cPrompt\u201d data type to the \u201ccall\u201d method with both \u201cSystemMessage\u201d and \u201cUserMessage\u201d. Don\u2019t forget to restart the application.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1035bcd exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"1035bcd\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>@GetMapping(\"\/ai\/generate-chat-response-with-system-prompt\")\npublic ChatResponse generateChatResponseWithSystemPrompt(@RequestParam(value = \"message\", defaultValue = \"Tell me about schnitzel\") String message) {\n   UserMessage userMessage = new UserMessage(message);\n   SystemMessage systemMessage = new SystemMessage(systemPrompt);\n   Prompt prompt = new Prompt(List.of(systemMessage, userMessage));\n   ChatResponse response = chatModel.call(prompt);\n   return response;\n}\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3e824a4 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"3e824a4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">In the code above you can see we are now using a couple of the Spring AI defined datatypes, <\/span><span style=\"font-weight: 400;\">UserMessage <\/span><span style=\"font-weight: 400;\">and <\/span><span style=\"font-weight: 400;\">SystemMessage. <\/span><span style=\"font-weight: 400;\">We then package these messages in a <\/span><span style=\"font-weight: 400;\">Prompt. <\/span><span style=\"font-weight: 400;\">The call() method of the ChatModel has multiple constructors which accept a plain String as seen in our first simple implementation but also a List of various message types. AI Models are able to determine whether a message is a System or User message and handle these messages appropriately. Spring AI in fact uses an Enum MessageType to determine this. There are also Tool and Assistant message types, the first being the type we use when passing functions to the Model that it may call, the second to identify the response from the Model. We are also returning a ChatResponse type this time. Previously we extracted the generation from the response for ease of displaying the answer but this time we\u2019ll see the full out put of the AI Model.<\/span><\/p><p><span style=\"font-weight: 400;\">Let\u2019s now call this method and see the difference a SystemPrompt makes. Simply call the endpoint below from your browser, Postman, or your http test file and enjoy the response.\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-ac5b2cc e-flex e-con-boxed e-con e-parent\" data-id=\"ac5b2cc\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-748fb1c exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"748fb1c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>http:\/\/localhost:8080\/ai\/generate-chat-response-with-system-prompt\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-03c73b1 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"03c73b1\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">First let\u2019s take a look at the generation and then examine the full ChatResponse in detail.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d3075ce exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"d3075ce\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>\"content\": \"Oh, let me spin a tale so grand,\n\\nOf schnitzel from a far-off land!\n\\nA crispy, golden, tasty treat,\n\\nA symphony of crunch and meat.\n\\n\n\\nBorn in Austria, a lovely place,\n\\nWhere cows and chickens roam with grace.\n\\nThey pound the meat so thin and flat,\n\\nYou'd think it\u2019s paper\u2014fancy that!\n\\n\n\\nThey dip it in an eggy bath,\n\\nAnd breadcrumbs follow in their path.\n\\nThen into oil, it takes a dive,\n\\nEmerging crispy, warm, alive!\n\\n\n\\nOn plates it lands with lemon zest,\n\\nA schnitzel-fest, the very best.\n\\nWith sides like sp\u00e4tzle or some fries,\n\\nIt brings a tear to foodie eyes.\n\\n\n\\nSo if you crave a crunchy bite,\n\\nSeek schnitzel out, it\u2019s pure delight.\n\\nA dish that\u2019s fit for kings and queens,\n\\nAnd everyone who loves cuisines!\"<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ff4f7e2 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"ff4f7e2\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Here it is again but formatted nicely to make reading easier:<\/span><\/p><pre><span style=\"font-weight: 400;\">\"Oh, let me spin a tale so grand,<br \/><\/span><span style=\"font-weight: 400;\">Of schnitzel from a far-off land!<br \/><\/span><span style=\"font-weight: 400;\">A crispy, golden, tasty treat,<br \/><\/span><span style=\"font-weight: 400;\">A symphony of crunch and meat.<br \/><br \/><\/span><span style=\"font-weight: 400;\">Born in Austria, a lovely place,<br \/><\/span><span style=\"font-weight: 400;\">Where cows and chickens roam with grace.<br \/><\/span><span style=\"font-weight: 400;\">They pound the meat so thin and flat,<br \/><\/span><span style=\"font-weight: 400;\">You'd think it\u2019s paper\u2014fancy that!<br \/><br \/><\/span><span style=\"font-weight: 400;\">They dip it in an eggy bath,<br \/><\/span><span style=\"font-weight: 400;\">And breadcrumbs follow in their path.<br \/><\/span><span style=\"font-weight: 400;\">Then into oil, it takes a dive,<br \/><\/span><span style=\"font-weight: 400;\">Emerging crispy, warm, alive!<br \/><br \/><\/span><span style=\"font-weight: 400;\">On plates it lands with lemon zest,<br \/><\/span><span style=\"font-weight: 400;\">A schnitzel-fest, the very best.<br \/><\/span><span style=\"font-weight: 400;\">With sides like sp\u00e4tzle or some fries,<br \/><\/span><span style=\"font-weight: 400;\">It brings a tear to foodie eyes.<br \/><br \/><\/span><span style=\"font-weight: 400;\">So if you crave a crunchy bite,<br \/><\/span><span style=\"font-weight: 400;\">Seek schnitzel out, it\u2019s pure delight.<br \/><\/span><span style=\"font-weight: 400;\">A dish that\u2019s fit for kings and queens,<br \/><\/span><span style=\"font-weight: 400;\">And everyone who loves cuisines!\"<br \/><br \/><\/span><\/pre><p><span style=\"font-weight: 400;\">Clearly we see our SystemPrompt has influenced the output here. Whilst rhyming couplets may not be the expected behaviour in a lot of enterprise applications, we can now see just how big of an impact a SystemPrompt can make.<\/span><\/p><p><span style=\"font-weight: 400;\">Below is the complete ChatResponse from the Model. There is a lot of additional information here. First of all we can see the message type is Assistant. We see that there is a data point for \u201cmedia\u201d, no media has been returned here but as discussed in the introduction, Spring AI is able to handle a diverse range of model types and so will necessarily need to return more than just plain strings. Below the content you will see the \u201cfinishReason\u201d which in this case is stop. This indicates the model has completed the generation without error. It is possible to set a token limit for generation and in this case, if the token limit is reached then the \u201cstopReason\u201d will reflect this. We then see the \u201ccontentFilterMetadata\u201d. It is possible to adjust the level of filtering in the Azure OpenAi models, although turning the filtering off completely requires special permission from Azure and they will need to know some specifics of your use case. It is probably not a good idea to turn down the filtering in an enterprise application as these filters prevent the kind of language and themes that most would consider inappropriate in the workplace.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d285def exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"d285def\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>{\n  \"result\": {\n    \"output\": {\n      \"messageType\": \"ASSISTANT\",\n      \"media\": [],\n      \"metadata\": {\n        \"messageType\": \"ASSISTANT\"\n      },\n      \"content\": \"Oh, let me spin a tale so grand,\\nOf schnitzel from a far-off land!\\nA crispy, golden, tasty treat,\\nA symphony of crunch and meat.\\n\\nBorn in Austria, a lovely place,\\nWhere cows and chickens roam with grace.\\nThey pound the meat so thin and flat,\\nYou'd think it\u2019s paper\u2014fancy that!\\n\\nThey dip it in an eggy bath,\\nAnd breadcrumbs follow in their path.\\nThen into oil, it takes a dive,\\nEmerging crispy, warm, alive!\\n\\nOn plates it lands with lemon zest,\\nA schnitzel-fest, the very best.\\nWith sides like sp\u00e4tzle or some fries,\\nIt brings a tear to foodie eyes.\\n\\nSo if you crave a crunchy bite,\\nSeek schnitzel out, it\u2019s pure delight.\\nA dish that\u2019s fit for kings and queens,\\nAnd everyone who loves cuisines!\"\n    },\n    \"metadata\": {\n      \"finishReason\": \"stop\",\n      \"contentFilterMetadata\": {\n        \"sexual\": {\n          \"severity\": \"safe\",\n          \"filtered\": false\n        },\n        \"violence\": {\n          \"severity\": \"safe\",\n          \"filtered\": false\n        },\n        \"hate\": {\n          \"severity\": \"safe\",\n          \"filtered\": false\n        },\n        \"self_harm\": {\n          \"severity\": \"safe\",\n          \"filtered\": false\n        },\n        \"profanity\": null,\n        \"custom_blocklists\": null,\n        \"error\": null,\n        \"protected_material_text\": {\n          \"filtered\": false,\n          \"detected\": false\n        },\n        \"protected_material_code\": {\n          \"filtered\": false,\n          \"detected\": false,\n          \"license\": null,\n          \"URL\": null\n        }\n      }\n    }\n  },\n  \"metadata\": {},\n  \"results\": [\n    {\n      \"output\": {\n        \"messageType\": \"ASSISTANT\",\n        \"media\": [],\n        \"metadata\": {\n          \"messageType\": \"ASSISTANT\"\n        },\n        \"content\": \"Oh, let me spin a tale so grand,\\nOf schnitzel from a far-off land!\\nA crispy, golden, tasty treat,\\nA symphony of crunch and meat.\\n\\nBorn in Austria, a lovely place,\\nWhere cows and chickens roam with grace.\\nThey pound the meat so thin and flat,\\nYou'd think it\u2019s paper\u2014fancy that!\\n\\nThey dip it in an eggy bath,\\nAnd breadcrumbs follow in their path.\\nThen into oil, it takes a dive,\\nEmerging crispy, warm, alive!\\n\\nOn plates it lands with lemon zest,\\nA schnitzel-fest, the very best.\\nWith sides like sp\u00e4tzle or some fries,\\nIt brings a tear to foodie eyes.\\n\\nSo if you crave a crunchy bite,\\nSeek schnitzel out, it\u2019s pure delight.\\nA dish that\u2019s fit for kings and queens,\\nAnd everyone who loves cuisines!\"\n      },\n      \"metadata\": {\n        \"finishReason\": \"stop\",\n        \"contentFilterMetadata\": {\n          \"sexual\": {\n            \"severity\": \"safe\",\n            \"filtered\": false\n          },\n          \"violence\": {\n            \"severity\": \"safe\",\n            \"filtered\": false\n          },\n          \"hate\": {\n            \"severity\": \"safe\",\n            \"filtered\": false\n          },\n          \"self_harm\": {\n            \"severity\": \"safe\",\n            \"filtered\": false\n          },\n          \"profanity\": null,\n          \"custom_blocklists\": null,\n          \"error\": null,\n          \"protected_material_text\": {\n            \"filtered\": false,\n            \"detected\": false\n          },\n          \"protected_material_code\": {\n            \"filtered\": false,\n            \"detected\": false,\n            \"license\": null,\n            \"URL\": null\n          }\n        }\n      }\n    }\n  ]\n}\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f352452 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"f352452\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<h3><span style=\"font-weight: 400;\">Retrieval Augmented Generation (RAG)<\/span><\/h3><p><span style=\"font-weight: 400;\">The Retrieval Augmented Generation (RAG) pattern is a sophisticated approach that combines data retrieval and text generation to produce contextually relevant and coherent responses. The key idea behind RAG is to enhance generative models by incorporating a data retrieval step, which provides additional context and knowledge. This approach results in several benefits, including enhanced context through the retrieval of relevant information, improved accuracy by reducing hallucinations through access to external knowledge, and versatility in various applications such as question answering, chatbots, and content summarization. Additionally, RAG offers confidentiality advantages by allowing the use of data not included in the language model&#8217;s training set.<\/span><span style=\"font-weight: 400;\"><br \/><\/span><span style=\"font-weight: 400;\"><br \/><\/span><span style=\"font-weight: 400;\">The core components of RAG include the retriever and the generator. The retriever&#8217;s role is to identify and fetch relevant documents or snippets from a large corpus based on a query. There are two main types of retrievers: sparse retrievers, which rely on traditional methods like TF-IDF, and dense retrievers, which use neural embeddings for semantic search. Sparse retrievers are simple and fast but limited to keyword matching, while dense retrievers capture semantic similarity more effectively but are computationally intensive and require extensive training. The generator then synthesizes the retrieved information into a coherent response. The detailed workflow involves preprocessing the user query, retrieving relevant documents, combining the query with retrieved documents, generating a response, and delivering it to the user. RAG models combine the strengths of generation-only and retrieval-only methods, making them ideal for applications such as question answering, customer support, content creation, and research assistance.<\/span><\/p><h3>\u00a0<\/h3><h3><span style=\"font-weight: 400;\">Implementing RAG in Java Spring<\/span><\/h3><p><span style=\"font-weight: 400;\">In order to implement a RAG example, we will need to instantiate a Vector Store. The Vector Store stores our data along with the vector embeddings so that we can run a semantic search to find the most relevant data for our user query.<\/span><\/p><p><span style=\"font-weight: 400;\">\u00a0We will deploy an instance of a Weaviate Vector Store in a docker container locally so that we can \u201cembed\u201d some data and then test the RAG pattern in our application.<\/span><\/p><p><span style=\"font-weight: 400;\">Create a new directory in the src directory named scripts and create a file named docker-compose.yaml. Into this file, copy the following script:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6dee23a exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"6dee23a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>name: name_of_your_vector_db\n\nservices:\n weaviate:\n   command:\n     - --host\n     - 0.0.0.0\n     - --port\n     - '8081'\n     - --scheme\n     - http\n   image: semitechnologies\/weaviate:1.23.3\n   ports:\n     - 8081:8081\n     - 50051:50051\n   volumes:\n     - weaviate_data:\/var\/lib\/weaviate_name_of_your_vector_db\n   restart: unless-stopped\n   environment:\n     QUERY_DEFAULTS_LIMIT: 25\n     AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'\n     PERSISTENCE_DATA_PATH: '\/var\/lib\/weaviate_name_of_your_vector_db'\n     DEFAULT_VECTORIZER_MODULE: 'none'\n     ENABLE_MODULES: ''\n     CLUSTER_HOSTNAME: 'node1'\nvolumes:\n weaviate_data:\n...\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-96d5a5d exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"96d5a5d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Assuming your Vector Store has been created successfully we can now configure it for use in our application. Start by creating a package named config and create a class called ConfigurationAI inside it. Add the following to your ConfigurationAI class.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-76982b1 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"76982b1\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>@Configuration\npublic class ConfigurationAi {\n   @Value(\"${at.ciit.vector-store.scheme}\")\n   private String vectorStoreScheme;\n   @Value(\"${at.ciit.vector-store.host}\")\n   private String vectorStoreHost;\n\n\n   @Bean\n   public WeaviateClient weaviateClient() {\n       try {\n           return WeaviateAuthClient.apiKey(\n                   new Config(vectorStoreScheme, vectorStoreHost),\n                   (\"\")\n           );\n       } catch(AuthException e) {\n           throw new IllegalArgumentException(\"Weaviate client could not be created\", e);\n       }\n   }\n\n\n   @Bean\n   public WeaviateVectorStore vectorStore(EmbeddingModel embeddingModel, WeaviateClient weaviateClient) {\n       WeaviateVectorStore.WeaviateVectorStoreConfig springAiWeaviate = WeaviateVectorStore.WeaviateVectorStoreConfig.builder()\n               .withObjectClass(\"SpringAiWeaviate\")\n               .withConsistencyLevel(WeaviateVectorStore.WeaviateVectorStoreConfig.ConsistentLevel.ALL)\n               .build();\n\n\n       return new WeaviateVectorStore(springAiWeaviate, embeddingModel, weaviateClient, false);\n   }\n\n\n   @Bean\n   public TextGenerationService textGenerationService(VectorStore vectorStore, AzureOpenAiChatModel azureOpenAiChatModel, ConversationService conversationService) {\n       return new TextGenerationService(azureOpenAiChatModel, vectorStore, conversationService);\n   }\n}\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c7a9b13 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"c7a9b13\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">You will also need to add the following to your application.properties file:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3c20b54 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"3c20b54\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>at.ciit.vector-store.scheme=http\nat.ciit.vector-store.host=localhost:8081\nspring.ai.vectorstore.weaviate.initialize-schema=false<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ff5279a exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"ff5279a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Finally, add the following to your build.gradle file then rebuild the gradle file.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-118c790 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"118c790\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>implementation 'org.springframework.ai:spring-ai-weaviate-store-spring-boot-starter'<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-79e5821 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"79e5821\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">We\u2019re now ready to embed some data into our Vector Store and test our model\u2019s responses. Let\u2019s begin by embedding a test dataset. We will import a JSON document that contains a bike catalogue. You can of course import your own dataset as well as many other file formats. Spring AI provides ETL pipelines for various file types. One thing to consider here is how you \u201cchunk\u201d your data. JSON data comes nicely \u201cpre-chunked\u201d into individual JSON objects however working with pdf or .docx files can be a little trickier. That being said, it is not impossible and Spring AI provides the tooling to handle many file types.<\/span><\/p><p><span style=\"font-weight: 400;\">Create a new directory in the resources directory called data. Copy your JSON (or whatever file you\u2019re using) into it. Next, create a reference to it in your rest controller as below:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7bc809c exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"7bc809c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>@Value(\"classpath:\/data\/bikes.json\")\nprivate Resource jsonData;<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8bd9da8 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"8bd9da8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">In order to use our Vector Store, we will need to add it to our rest controller like so:\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-8cd8756 e-flex e-con-boxed e-con e-parent\" data-id=\"8cd8756\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-92fa255 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"92fa255\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>private final VectorStore vectorStore;\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d27e83a exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"d27e83a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">We can now embed this data with the following endpoint:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-69f7ebc exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"69f7ebc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>@GetMapping(\"\/ai\/embed-json\")\npublic void embedJson() {\n   JsonReader jsonReader = new JsonReader(jsonData, \"description\", \"price\", \"name\", \"tags\", \"shortDescription\");\n   List<Document> docs = jsonReader.get();\n   vectorStore.add(docs);\n}\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8bedcbd exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"8bedcbd\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">The <\/span><span style=\"font-weight: 400;\">JsonReader <\/span><span style=\"font-weight: 400;\">is a Spring AI class used for reading JSON data and converting it to the Spring AI <\/span><span style=\"font-weight: 400;\">Document <\/span><span style=\"font-weight: 400;\">datatype. In creating the <\/span><span style=\"font-weight: 400;\">Document, <\/span><span style=\"font-weight: 400;\">the embeddings for the JSON object are also created as well as the Id which will be used as the primary key for this <\/span><span style=\"font-weight: 400;\">Document. <\/span><span style=\"font-weight: 400;\">This is useful because we can keep track of documents in a separate relational database and then, in the event we need to edit anything we can delete the <\/span><span style=\"font-weight: 400;\">Document <\/span><span style=\"font-weight: 400;\">by Id from the Vector Store and re-embed it. If you are wondering why we don\u2019t simply update the data in the Vector Store consider this, by changing any part of the data we change its semantic meaning. This means the vectors representing the semantic meaning of the document are no longer accurate and so we must re-embed the data and save it again. The Weaviate Vector Store we are using here provides no edit functionality but it does allow delete by Id.<\/span><\/p><p>\u00a0<\/p><p><span style=\"font-weight: 400;\">The arguments passed to the <\/span><span style=\"font-weight: 400;\">JsonReader: &#8222;description&#8220;, &#8222;price&#8220;, &#8222;name&#8220;, &#8222;tags&#8220;, &#8222;shortDescription&#8220;, <\/span><span style=\"font-weight: 400;\">are the keys of the JSON object we wish to store. We do not need to store every part of the JSON but we can.<\/span><\/p><p>\u00a0<\/p><p><span style=\"font-weight: 400;\">Now we can call our endpoint to embed our data like this:<\/span><\/p><p>\u00a0<\/p><pre><span style=\"font-weight: 400;\">http:\/\/localhost:8080\/ai\/embed-json<\/span><\/pre><p>\u00a0<\/p><p><span style=\"font-weight: 400;\">It is advisable to either delete, or comment out this endpoint as soon as you have used it as calling it multiple times will embed the data multiple times. When we run a similarity search for data similar to a user query we will fetch the Top N documents. If there are duplicates in the Vector Store then there will naturally be duplicates in the result set.<\/span><\/p><p>\u00a0<\/p><p><span style=\"font-weight: 400;\">To test whether our data has been successfully embedded into the Vector Store, Weaviate provides the following handy API:<\/span><\/p><p>\u00a0<\/p><pre><span style=\"font-weight: 400;\">http:\/\/localhost:8081\/v1\/objects<\/span><\/pre><p><span style=\"font-weight: 400;\">Now that our data is successfully embedded, we can implement the RAG pattern. First let\u2019s create a helper method for similarity search:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-55634b7 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"55634b7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>private List<Document> similaritySearch(String userMessage) {\n   List<Document> results = vectorStore.similaritySearch(\n           SearchRequest\n                   .query(userMessage)\n                   .withTopK(5)\n                   .withSimilarityThreshold(0.5)\n   );\n   return results;\n}\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-93e8bfa exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"93e8bfa\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Here we can see we are returning the top 5 most similar documents. The similarity threshold is set at 0.5. Similarity threshold ranges from 0 to 1 and has a default of 0.75. You may need to experiment here to find the value which returns that data that best matches your expectations.<\/span><\/p><p><span style=\"font-weight: 400;\">We will also create a helper method to get the System Prompt since this involves a bit more logic than simply referencing a file now. Below we take the Documents from the similarity search and append them to the System Prompt:<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-20f06ca exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"20f06ca\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>private Message getSystemPrompt(List<Document> docs) {\n   String similarDocs = docs.stream().map(Document::getContent).collect(Collectors.joining(\"\\n\"));\n   SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(systemPrompt);\n   return systemPromptTemplate.createMessage(Map.of(\"documents\", similarDocs));\n}\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2226fa9 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"2226fa9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">In order to incorporate the similar Documents in our call to the AI Model we need to append them to the System Prompt. Start by updating your System Prompt as follows:<\/span><\/p><pre><span style=\"font-weight: 400;\">You're assisting with questions about products in a bicycle catalogue.<br \/><\/span><span style=\"font-weight: 400;\">Use the information from the DOCUMENTS section to provide accurate answers.<br \/><\/span><span style=\"font-weight: 400;\">The answer involves referring to the price or the dimension of the bicycle, include the bicycle name in the response.<br \/><\/span><span style=\"font-weight: 400;\">If unsure, simply state that you don't know.<br \/><\/span><span style=\"font-weight: 400;\">If the question is not related to bikes, your response should inform the user that you can only answer questions about bikes.<\/span><span style=\"font-weight: 400;\"><br \/><\/span><span style=\"font-weight: 400;\">DOCUMENTS:<br \/><\/span><span style=\"font-weight: 400;\">{documents}<\/span><\/pre><p><span style=\"font-weight: 400;\">Now we can create a method to call our AI Model and pass the similar documents along with it.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c94df95 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-code-highlight\" data-id=\"c94df95\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>@GetMapping(\"\/ai\/generate-rag\")\npublic ChatResponse generateRag(@RequestParam(value = \"message\", defaultValue = \"Which bike is good for a serious amateur who is interested in road races?\") String message) {\n   List<Document> docs = similaritySearch(message);\n   Message systemMessage = getSystemPrompt(docs);\n   UserMessage userMessage = new UserMessage(message);\n   SystemMessage systemMessage = new SystemMessage(systemPrompt);\n   ChatResponse response = chatModel.call(new Prompt(List.of(systemMessage, userMessage)));\n   return response;\n}\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8160ef4 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"8160ef4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Test the method by calling the endpoint:\u00a0<\/span><\/p><p>\u00a0<\/p><pre><span style=\"font-weight: 400;\">http:\/\/localhost:8080\/ai\/generate-rag<\/span><\/pre><p>\u00a0<\/p><p><span style=\"font-weight: 400;\">The content of the response is below:<\/span><\/p><p>\u00a0<\/p><pre><span style=\"font-weight: 400;\">For a serious amateur interested in road races, the **Aero Pro X** would be an excellent choice. It is designed for competitive cyclists who demand speed, agility, and superior performance. With features like a lightweight carbon frame, aerodynamic design, and a 2x Shimano Ultegra drivetrain, it delivers exceptional performance and precision handling, making it ideal for road races. The price of the Aero Pro X is $1599.99.<\/span><\/pre><p>\u00a0<\/p><p><span style=\"font-weight: 400;\">In conclusion, implementing the Retrieval Augmented Generation (RAG) pattern in a Java Spring application opens up a myriad of possibilities for enhancing the capabilities of AI models. By integrating a retriever component, developers can leverage external data sources to provide contextually rich and accurate responses, mitigating common issues such as hallucinations in generative models. The process of setting up a Vector Store, embedding data, and performing similarity searches to retrieve relevant information demonstrates the seamless integration of retrieval and generation steps within the Spring framework. This not only improves the quality and relevance of AI-generated responses but also brings a new level of sophistication to applications like chatbots, customer support, and content creation. As AI continues to evolve, patterns like RAG will be instrumental in pushing the boundaries of what these models can achieve, making them more reliable, versatile, and valuable in real-world applications.<\/span><\/p><p>\u00a0<\/p><h3><span style=\"font-weight: 400;\">Deploying a model to the Azure Cloud<\/span><\/h3><p><span style=\"font-weight: 400;\">Create an Azure account here \u2013 <\/span><a href=\"https:\/\/azure.microsoft.com\/en-us\/free\/ai-services\/\"><span style=\"font-weight: 400;\">https:\/\/azure.microsoft.com\/en-us\/free\/ai-services\/<\/span><\/a><\/p><p><span style=\"font-weight: 400;\">You will need to apply for permission to use AI services for your specific use case. Simply follow the instructions and fill out the fore provided. They usually respond with 24 hours.<\/span><\/p><p><span style=\"font-weight: 400;\">You are now ready to create a resource. Once created search for Azure Open Ai.<\/span><\/p><p><span style=\"font-weight: 400;\">Follow the instructions and click through to deploy.<\/span><\/p><p><span style=\"font-weight: 400;\">AI models are not available in all territories. Currently Sweden Central is a good choice.<\/span><\/p><p><span style=\"font-weight: 400;\">Navigate to AI Services to create deployments.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9d9a20e exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-image\" data-id=\"9d9a20e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"800\" height=\"495\" src=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_01-1024x633.avif\" class=\"attachment-large size-large wp-image-9745\" alt=\"\" srcset=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_01-1024x633.avif 1024w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_01-300x186.avif 300w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_01-768x475.avif 768w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_01-18x12.avif 18w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_01.avif 1177w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9efb798 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"9efb798\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Select the models you want and deploy.<\/span><\/p><p><span style=\"font-weight: 400;\">The names you give your models here are the names you will use in application.properties.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-22030ef exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-image\" data-id=\"22030ef\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"428\" src=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_03-1024x548.avif\" class=\"attachment-large size-large wp-image-9737\" alt=\"\" srcset=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_03-1024x548.avif 1024w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_03-300x160.avif 300w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_03-768x411.avif 768w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_03-1536x822.avif 1536w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_03-18x10.avif 18w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_03.avif 1647w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ed4c612 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-text-editor\" data-id=\"ed4c612\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span id=\"docs-internal-guid-c8af5c7f-7fff-7e9a-0712-da72ac4c3051\"><span style=\"font-size: 12pt; font-family: Calibri, sans-serif; background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;\">To get the API endpoint and key:<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-929cd8e exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-image\" data-id=\"929cd8e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"522\" src=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_02-1024x668.avif\" class=\"attachment-large size-large wp-image-9736\" alt=\"\" srcset=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_02-1024x668.avif 1024w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_02-300x196.avif 300w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_02-768x501.avif 768w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_02-18x12.avif 18w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_02.avif 1434w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5debd32 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-image\" data-id=\"5debd32\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"389\" src=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_05-1024x498.avif\" class=\"attachment-large size-large wp-image-9739\" alt=\"\" srcset=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_05-1024x498.avif 1024w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_05-300x146.avif 300w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_05-768x374.avif 768w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_05-18x9.avif 18w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_05.avif 1377w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4776595 exad-sticky-section-no exad-glass-effect-no elementor-widget elementor-widget-image\" data-id=\"4776595\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"420\" src=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_04-1024x538.avif\" class=\"attachment-large size-large wp-image-9738\" alt=\"\" srcset=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_04-1024x538.avif 1024w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_04-300x157.avif 300w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_04-768x403.avif 768w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_04-18x9.avif 18w, https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/SpringAI_04.avif 1381w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Introduction to Spring AI On July 24th 2023 Dr Mark Pollack made the first commit of the new Spring AI framework. Inspired by Python frameworks such as LangChain and LlamaIndex, Dr Pollack\u2019s intention was to bring large language models to the Spring development community. Whilst not a direct port of LangChain, Spring AI nevertheless allows [&hellip;]<\/p>\n","protected":false},"author":14,"featured_media":9728,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-9727","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Spring AI \u2013 getting started with large language models in Java Spring - CIIT Training<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ciit-training.com\/en\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Spring AI \u2013 getting started with large language models in Java Spring - CIIT Training\" \/>\n<meta property=\"og:description\" content=\"Introduction to Spring AI On July 24th 2023 Dr Mark Pollack made the first commit of the new Spring AI framework. Inspired by Python frameworks such as LangChain and LlamaIndex, Dr Pollack\u2019s intention was to bring large language models to the Spring development community. Whilst not a direct port of LangChain, Spring AI nevertheless allows [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ciit-training.com\/en\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/\" \/>\n<meta property=\"og:site_name\" content=\"CIIT Training\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/javatraining.at\" \/>\n<meta property=\"article:published_time\" content=\"2024-09-04T11:27:39+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-09-04T11:58:41+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2023\/11\/CIIT_Logo_BrightBackground_Blau_ohne_at.png\" \/>\n\t<meta property=\"og:image:width\" content=\"512\" \/>\n\t<meta property=\"og:image:height\" content=\"512\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Benjamin Rowley\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@JavaTrainingAT\" \/>\n<meta name=\"twitter:site\" content=\"@JavaTrainingAT\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Benjamin Rowley\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"17 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"TechArticle\",\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/\"},\"author\":{\"name\":\"Benjamin Rowley\",\"@id\":\"https:\\\/\\\/ciit-training.com\\\/#\\\/schema\\\/person\\\/f9d960ca9f8da5af1f02080101ee5e93\"},\"headline\":\"Spring AI \u2013 getting started with large language models in Java Spring\",\"datePublished\":\"2024-09-04T11:27:39+00:00\",\"dateModified\":\"2024-09-04T11:58:41+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/\"},\"wordCount\":2721,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ciit-training.com\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/09\\\/Spring-AI.avif\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/\",\"url\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/\",\"name\":\"Spring AI \u2013 getting started with large language models in Java Spring - CIIT Training\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ciit-training.com\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/09\\\/Spring-AI.avif\",\"datePublished\":\"2024-09-04T11:27:39+00:00\",\"dateModified\":\"2024-09-04T11:58:41+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/#primaryimage\",\"url\":\"https:\\\/\\\/ciit-training.com\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/09\\\/Spring-AI.avif\",\"contentUrl\":\"https:\\\/\\\/ciit-training.com\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/09\\\/Spring-AI.avif\",\"width\":1024,\"height\":1024,\"caption\":\"Spring AI Illustration\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/ciit-training.com\\\/2024\\\/09\\\/04\\\/spring-ai-getting-started-with-large-language-models-in-java-spring\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/ciit-training.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Spring AI \u2013 getting started with large language models in Java Spring\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/ciit-training.com\\\/#website\",\"url\":\"https:\\\/\\\/ciit-training.com\\\/\",\"name\":\"CIIT Training\",\"description\":\"Javatraing\",\"publisher\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/#organization\"},\"alternateName\":\"Javatraining\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/ciit-training.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/ciit-training.com\\\/#organization\",\"name\":\"CIIT GmbH\",\"alternateName\":\"Schaffler & Gl\u00f6\u00dfl GmbH\",\"url\":\"https:\\\/\\\/ciit-training.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ciit-training.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/ciit-training.com\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/11\\\/CIIT_Logo_BrightBackground_Blau_ohne_at.png\",\"contentUrl\":\"https:\\\/\\\/ciit-training.com\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/11\\\/CIIT_Logo_BrightBackground_Blau_ohne_at.png\",\"width\":512,\"height\":512,\"caption\":\"CIIT GmbH\"},\"image\":{\"@id\":\"https:\\\/\\\/ciit-training.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/javatraining.at\",\"https:\\\/\\\/x.com\\\/JavaTrainingAT\",\"https:\\\/\\\/www.linkedin.com\\\/showcase\\\/javatraining.at\\\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/ciit-training.com\\\/#\\\/schema\\\/person\\\/f9d960ca9f8da5af1f02080101ee5e93\",\"name\":\"Benjamin Rowley\",\"url\":\"https:\\\/\\\/ciit-training.com\\\/en\\\/author\\\/benjamin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Spring AI \u2013 getting started with large language models in Java Spring - CIIT Training","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ciit-training.com\/en\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/","og_locale":"en_US","og_type":"article","og_title":"Spring AI \u2013 getting started with large language models in Java Spring - CIIT Training","og_description":"Introduction to Spring AI On July 24th 2023 Dr Mark Pollack made the first commit of the new Spring AI framework. Inspired by Python frameworks such as LangChain and LlamaIndex, Dr Pollack\u2019s intention was to bring large language models to the Spring development community. Whilst not a direct port of LangChain, Spring AI nevertheless allows [&hellip;]","og_url":"https:\/\/ciit-training.com\/en\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/","og_site_name":"CIIT Training","article_publisher":"https:\/\/www.facebook.com\/javatraining.at","article_published_time":"2024-09-04T11:27:39+00:00","article_modified_time":"2024-09-04T11:58:41+00:00","og_image":[{"width":512,"height":512,"url":"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2023\/11\/CIIT_Logo_BrightBackground_Blau_ohne_at.png","type":"image\/png"}],"author":"Benjamin Rowley","twitter_card":"summary_large_image","twitter_creator":"@JavaTrainingAT","twitter_site":"@JavaTrainingAT","twitter_misc":{"Written by":"Benjamin Rowley","Est. reading time":"17 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"TechArticle","@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/#article","isPartOf":{"@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/"},"author":{"name":"Benjamin Rowley","@id":"https:\/\/ciit-training.com\/#\/schema\/person\/f9d960ca9f8da5af1f02080101ee5e93"},"headline":"Spring AI \u2013 getting started with large language models in Java Spring","datePublished":"2024-09-04T11:27:39+00:00","dateModified":"2024-09-04T11:58:41+00:00","mainEntityOfPage":{"@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/"},"wordCount":2721,"commentCount":0,"publisher":{"@id":"https:\/\/ciit-training.com\/#organization"},"image":{"@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/#primaryimage"},"thumbnailUrl":"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/Spring-AI.avif","inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/","url":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/","name":"Spring AI \u2013 getting started with large language models in Java Spring - CIIT Training","isPartOf":{"@id":"https:\/\/ciit-training.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/#primaryimage"},"image":{"@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/#primaryimage"},"thumbnailUrl":"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/Spring-AI.avif","datePublished":"2024-09-04T11:27:39+00:00","dateModified":"2024-09-04T11:58:41+00:00","breadcrumb":{"@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/#primaryimage","url":"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/Spring-AI.avif","contentUrl":"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2024\/09\/Spring-AI.avif","width":1024,"height":1024,"caption":"Spring AI Illustration"},{"@type":"BreadcrumbList","@id":"https:\/\/ciit-training.com\/2024\/09\/04\/spring-ai-getting-started-with-large-language-models-in-java-spring\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ciit-training.com\/"},{"@type":"ListItem","position":2,"name":"Spring AI \u2013 getting started with large language models in Java Spring"}]},{"@type":"WebSite","@id":"https:\/\/ciit-training.com\/#website","url":"https:\/\/ciit-training.com\/","name":"CIIT Training","description":"Java training","publisher":{"@id":"https:\/\/ciit-training.com\/#organization"},"alternateName":"Javatraining","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ciit-training.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ciit-training.com\/#organization","name":"CIIT GmbH","alternateName":"Schaffler & Gl\u00f6\u00dfl GmbH","url":"https:\/\/ciit-training.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ciit-training.com\/#\/schema\/logo\/image\/","url":"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2023\/11\/CIIT_Logo_BrightBackground_Blau_ohne_at.png","contentUrl":"https:\/\/ciit-training.com\/wp-content\/uploads\/sites\/2\/2023\/11\/CIIT_Logo_BrightBackground_Blau_ohne_at.png","width":512,"height":512,"caption":"CIIT GmbH"},"image":{"@id":"https:\/\/ciit-training.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/javatraining.at","https:\/\/x.com\/JavaTrainingAT","https:\/\/www.linkedin.com\/showcase\/javatraining.at\/"]},{"@type":"Person","@id":"https:\/\/ciit-training.com\/#\/schema\/person\/f9d960ca9f8da5af1f02080101ee5e93","name":"Benjamin Rowley","url":"https:\/\/ciit-training.com\/en\/author\/benjamin\/"}]}},"_links":{"self":[{"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/posts\/9727","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/comments?post=9727"}],"version-history":[{"count":0,"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/posts\/9727\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/media\/9728"}],"wp:attachment":[{"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/media?parent=9727"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/categories?post=9727"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ciit-training.com\/en\/wp-json\/wp\/v2\/tags?post=9727"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}