Arnau Aregall

Exploring Spring Cloud Function, AWS Lambda and AI: building a native serverless application with Kotlin powered by OpenAI.

Exploring Spring Cloud Function, AWS Lambda and AI: building a native serverless application with Kotlin powered by OpenAI.

Arnau Aregall's photo
Arnau Aregall
·

29 min read

Table of contents

Introduction

Last May 2023, I had the pleasure to attend the Spring I/O Conference (10th edition!) in Barcelona, Spain. On the last day, Oleg Zhurakousky, one of the core maintainers of Spring Cloud Function, showcased the project in a talk that I enjoyed quite a lot. I got quite impressed by the capabilities of the project.

Since the first half of 2023 the tech landscape has been hyped by OpenAI products and IA in general, and Spring ecosystem is making big steps forward to bring JVM based applications to a native runtime, I thought it would be quite interesting to explore both the integration of Spring Cloud Function applications with OpenAI being deployed natively to a cloud provider such as AWS.

This article is the outcome of such exploration. I hope you enjoy it!

Overview

This article walks through the process of building and deploying a native serverless application as an AWS Lambda that interacts with OpenAI API, implemented using Spring Cloud Function and Kotlin.

The goal is to demonstrate:

  • How to implement and test functions using Spring Cloud Function and Kotlin.

  • How to integrate our functions with the OpenAI GPT model using their API.

  • How to build a serverless native executable of the application using GraalVM.

  • How to deploy the application as an AWS Lambda with a native runtime using SAM (Serverless Application Model).

Our application will be an AI-powered weather API: given some GPS coordinates, it will return us a quite accurate weather forecast for the given location for the next 7 days in a human-friendly way, as we asked a real meteorologist.

Example:

The weather forecast for Barcelona (Spain) from July 1st to July 7th shows temperatures ranging from 20.7 ºC to 29.5 ºC, with no precipitation expected throughout the period.

TLDR; show me the code!

The full source code of the article is published on GitHub 😉.

https://github.com/ArnauAregall/aws-lambda-spring-cloud-function

To follow the walk-through steps I assume you meet the following requirements:

🍃☁️ What is Spring Cloud Function?

With Spring Cloud Function, we can write functions as standalone units of code in our Spring Boot applications, and expose them as RESTful APIs, message-driven services, or event handlers.

From the official reference documentation:

Spring Cloud Function is a project with the following high-level goals:

  • Promote the implementation of business logic via functions.

  • Decouple the development lifecycle of business logic from any specific runtime target so that the same code can run as a web endpoint, a stream processor, or a task.

  • Support a uniform programming model across serverless providers, as well as the ability to run standalone (locally or in a PaaS).

  • Enable Spring Boot features (auto-configuration, dependency injection, metrics) on serverless providers.

It abstracts away all of the transport details and infrastructure, allowing the developer to keep all the familiar tools and processes, and focus firmly on business logic.

One of its key advantages is its ability to abstract away the underlying serverless platform, meaning we can develop our functions using Spring's abstractions and then deploy them on various serverless platforms like AWS Lambda, Azure Functions, or Google Cloud Functions without making significant changes to our code.

One of the features I liked the most is the ability to invoke functions in a composite way, meaning that each function has its input and produces its output, and those functions can be chained.

Let's understand better how Spring Cloud Function works with a simple example.

Understanding by example: the "Hello World" with Spring Cloud Function

A "Hello World" application using Spring Cloud Function, which is showcased a lot in the reference documentation, would be an application that declares two individual and isolated functions:

  • Given a String input, return it uppercased.

  • Given a String input, return it reversed.

@Configuration
class Functions {
    @Bean
    fun uppercase(): (String) -> String = String::uppercase

    @Bean
    fun reverse(): (String) -> String = String::reversed
}

@SpringBootApplication
class App

fun main(args: Array<String>) {
    runApplication<App>(*args)
}

As surprising as this might seem, this is a valid and compliant Spring Cloud Function application code.

We declare our functions as regular Spring @Beans, and the framework automatically exposes HTTP endpoints for us to invoke them individually or via composition.

Invoking the uppercase function:

POST localhost:8080/uppercase
Content-Type: text/plain

"Hello Spring Cloud Function!"

HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8

"HELLO SPRING CLOUD FUNCTION!"

Invoking the reverse function:

POST localhost:8080/reverse
Content-Type: text/plain

"Hello Spring Cloud Function!"

HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8

"!noitcnuF duolC gnirpS olleH"

Invoking a composite function: reverse,uppercase

POST localhost:8080/reverse,uppercase
Content-Type: text/plain

"Hello Spring Cloud Function!"

HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8

"!NOITCNUF DUOLC GNIRPS OLLEH"

⚙️ Setting up our project

Scaffolding our project structure with Spring Intializr

We will delegate the bootiful Spring Initalizr our project bootstrapping.

Head to https://start.spring.io and build a new project with the following configuration:

  • Project: Gradle - Kotlin

  • Language: Kotlin

  • Spring Boot 3.1.1

  • Java 17

  • Dependencies:

    • Function

    • Spring Reactive Web (WebFlux)

    • GraalVM Native Support

    • Testcontainers

    • Spring Boot DevTools (optional)

Follow 🍃🔗 this link to automatically generate the project with the mentioned configuration.

Reviewing the Gradle setup

Once downloaded, we will define our dependencies versions in a gradle.properties file placed at the root of our project:

kotlinVersion=1.8.22
springBootVersion=3.1.1
springCloudVersion=2022.0.3
springBootDependencyManagementVersion=1.1.0
mockServerVersion=5.15.0
testcontainersVersion=1.18.3
graalVMBuildToolsVersion=0.9.23
nettyResolverDnsMacosVersion=4.1.94.Final

Note: This article was written on June 2023, probably by the time you are reading it new versions may have been released - stay up to date!

Then we will customize the plugin management under settings.gradle.kts file to use the aimed versions of each plugin:

rootProject.name = "aws-lambda-spring-cloud-function"

pluginManagement {
    val kotlinVersion: String by settings
    val springBootVersion: String by settings
    val springBootDependencyManagementVersion: String by settings
    val graalVMBuildToolsVersion: String by settings

    plugins {
        id("org.springframework.boot") version springBootVersion
        id("io.spring.dependency-management") version springBootDependencyManagementVersion
        id("org.graalvm.buildtools.native") version graalVMBuildToolsVersion
        kotlin("jvm") version kotlinVersion
        kotlin("plugin.spring") version kotlinVersion
    }
}

And finally, our build.gradle.kts file would be the following:

// group, version, repositories...

plugins {
    id("org.springframework.boot")
    id("io.spring.dependency-management")
    id("org.graalvm.buildtools.native")
    kotlin("jvm")
    kotlin("plugin.spring")
}

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-webflux")
    implementation("org.springframework.cloud:spring-cloud-function-web")
    implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
    implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor")

    developmentOnly("org.springframework.boot:spring-boot-devtools")

    testImplementation("org.springframework.boot:spring-boot-starter-test")
    testImplementation("org.springframework.boot:spring-boot-testcontainers")
    testImplementation("io.projectreactor:reactor-test")
    testImplementation("org.testcontainers:mockserver:${property("testcontainersVersion")}")
    testImplementation("org.mock-server:mockserver-client-java:${property("mockServerVersion")}")
    testImplementation("io.netty:netty-resolver-dns-native-macos:${property("nettyResolverDnsMacosVersion")}:osx-aarch_64")
}

dependencyManagement {
    imports {
        mavenBom("org.springframework.cloud:spring-cloud-dependencies:${property("springCloudVersion")}")
    }
}

tasks.withType<KotlinCompile> {
    kotlinOptions {
        freeCompilerArgs = listOf("-Xjsr305=strict")
        jvmTarget = "17"
    }
}

tasks.withType<Test> {
    useJUnitPlatform()
}

As we will be using spring-boot-starter-webflux, our runtime will be a Netty server, thus our application will be naturally asynchronous and nonblocking.

Please note we added Testcontainers, MockServer and Netty DNS resolver for macOS to later implement reactive integration tests:

testImplementation("org.testcontainers:mockserver:${property("testcontainersVersion")}")
testImplementation("org.mock-server:mockserver-client-java:${property("mockServerVersion")}")
testImplementation("io.netty:netty-resolver-dns-native-macos:${property("nettyResolverDnsMacosVersion")}:osx-aarch_64")

The Netty DNS resolver dependency can be omitted if your local machine does not run macOS, although if you do you might need to adjust the OS architecture classifier (the one provided in the example above is for Mac M1).

Implementing our Functions

Quoting the article introduction, the main purpose of our application would be:

Given some GPS coordinates, return a quite accurate weather forecast for that location for the next 7 days in a human-friendly way, as it was a real meteorologist.

Our application will expose 3 main functions:

  • Forecast

    • Function name: forecast

    • Input: 📍Geo Location coordinates.

    • Output: 🌤️ The weather forecast for those coordinates fetched from OpenMeteo API.

  • Question

    • Function name: question

    • Input: ❓Questions for the AI model.

    • Output: 🤖 The Answer from the AI model to those questions.

  • Forecast to Question (aka "Glue Function")

    • Function name: forecastToQuestion

    • Input: 🌤️ A Forecast from OpenMeteo API.

    • Output:❓ A Question asking for a weather forecast that an AI model can understand.

📍🌤️ The Forecast function: calling OpenMeteo API

We will implement our Forecast function logic inside a "weather" package, following a package-by-feature strategy to prove that our function is self-contained and independent from the rest:

├── src
│   ├── main
│   │   ├── kotlin
│   │   │   └── tech
│   │   │       └── aaregall
│   │   │           └── lab
│   │   │               └── functions
│   │   │                   ├── App.kt
│   │   │                   └── weather
│   │   │                       ├── WeatherFunctions.kt
│   │   │                       ├── domain
│   │   │                       │   ├── Forecast.kt
│   │   │                       │   └── GeoLocation.kt
│   │   │                       └── service
│   │   │                           ├── WeatherService.kt
│   │   │                           └── openmeteo
│   │   │                               ├── OpenMeteoClient.kt
│   │   │                               └── OpenMeteoDTOs.kt

This would be our "weather" domain model:

// input
data class GeoLocation (val latitude: Float, val longitude: Float)

// output
data class Forecast(
    val geoLocation: GeoLocation,
    val hourlyForecasts: List<HourlyForecast>
)
data class HourlyForecast (
    val time: LocalDateTime,
    val temperature: Float,
    val precipitation: Float
)

We will then expose our forecast function with a @Bean definition:

@Configuration
class WeatherFunctions {

    @Bean
    fun forecast(weatherService: WeatherService): (Flux<GeoLocation>) -> Flux<Forecast> = weatherService::getForecast

}

Our function will be just as an entry point for our application, which has the WeatherService component injected, who will be responsible for handling the business logic.

@Service
class WeatherService(val openMeteoClient: OpenMeteoClient) {

    fun getForecast(geoLocations: Flux<GeoLocation>): Flux<Forecast> =
        geoLocations.map { openMeteoClient.getForecast(it.latitude, it.longitude) }
            .flatMap { it.map(responseToForecast()) }

    private fun responseToForecast(): (OpenMeteoForecastResponse) -> (Forecast) = {
        Forecast(GeoLocation(it.latitude, it.longitude),
            it.hourly.time.mapIndexed {
                  timeIndex, localDateTime -> HourlyForecast(localDateTime, it.hourly.temperature[timeIndex], it.hourly.precipitation[timeIndex])
            }.toList()
        )
    }
}

Finally the OpenMeteoClient injected in WeatherService will be the component that performs the HTTP calls to OpenMeteo REST API, using Spring Framework 6 HTTP Interfaces to build the rest client.

The HTTP client has a straightforward signature, where all the parameters are sent via query params.

@ConfigurationProperties("app.open-meteo")
data class OpenMeteoProperties (
    val baseUrl: String,
    val timeout: Duration,
    val hourlyParams: List<String>
)

@Component
class OpenMeteoClient(private val openMeteoProperties: OpenMeteoProperties, webClientBuilder: WebClient.Builder) {

    private val logger: Logger = LoggerFactory.getLogger(this::class.java)

    private val openMeteoHttpClient: OpenMeteoHttpClient = HttpServiceProxyFactory
        .builder(
            WebClientAdapter.forClient(
                webClientBuilder
                    .baseUrl(openMeteoProperties.baseUrl)
                    .clientConnector(ReactorClientHttpConnector(HttpClient.create().responseTimeout(openMeteoProperties.timeout)))
                    .defaultStatusHandler(not(HttpStatusCode::is2xxSuccessful)) {
                        Mono.error(OpenMeteoException(it.statusCode()))
                    }
                    .build()

            )
        )
        .build()
        .createClient()

    fun getForecast(latitude: Float, longitude: Float): Flux<OpenMeteoForecastResponse> =
        openMeteoHttpClient.getForecast(latitude, longitude, openMeteoProperties.hourlyParams.joinToString(","))
            .doOnError { logger.error("An error occurred while calling OpenMeteo for forecast", it) }

}

private class OpenMeteoException(
    status: HttpStatusCode,
    reason: String? = "Error: OpenMeteo API responded with $status") : ResponseStatusException(status, reason)

private fun interface OpenMeteoHttpClient {

    @GetExchange("/forecast")
    fun getForecast(
        @RequestParam("latitude") latitude: Float,
        @RequestParam("longitude") longitude: Float,
        @RequestParam("hourly") hourly: String
    ): Flux<OpenMeteoForecastResponse>

}

Note: OpenMeteoForecastResponse is the corresponding DTO for the API response, omitted in the article - you can check the full source code on GitHub.

Finally, we will define the Open Meteo configuration properties under application.yml:

app:
  open-meteo:
    base-url: "https://api.open-meteo.com/v1/"
    timeout: "PT3S"
    hourly-params:
      - "temperature_2m"
      - "precipitation"

At this point, by running our application and invoking POST localhost:8080/forecast with a valid body with some GPS coordinates, it should return us the weather forecast for that location fetched from OpenMeteo API in our on format.

Example: Get the weather forecast for Barcelona and Freiburg from OpenMeteo API.

POST localhost:8080/forecast
Accept: application/json
Content-Type: application/stream+json

# Barcelona => 41.3874° N, 2.1686° E
# Freiburg  => 47.9990° N, 7.8421° E

[
  {
    "latitude": 41.3874,
    "longitude": 2.1686
  },
  {
    "latitude": 47.9990,
    "longitude": 7.8421
  }
]

The response (shortened hourlyForecasts array) would be:

[
  {
    "geoLocation": {
      "latitude": 41.39,
      "longitude": 2.17
    },
    "hourlyForecasts": [
      {
        "time": "2023-07-01T00:00:00",
        "temperature": 24.4,
        "precipitation": 0.0
      },
      {
        "time": "2023-07-07T23:00:00",
        "temperature": 22.7,
        "precipitation": 0.0
      }
    ]
  },
  {
    "geoLocation": {
      "latitude": 48.0,
      "longitude": 7.8399997
    },
    "hourlyForecasts": [
      {
        "time": "2023-07-01T00:00:00",
        "temperature": 17.4,
        "precipitation": 0.0
      },
      {
        "time": "2023-07-07T23:00:00",
        "temperature": 14.7,
        "precipitation": 0.0
      }
    ]
  }
]

❓🤖 The Question function: calling OpenAI API

Following the same package-by-feature strategy, we will now implement all the Question and OpenAI-related code in a "question" package.

├── src
│   ├── main
│   │   ├── kotlin
│   │   │   └── tech
│   │   │       └── aaregall
│   │   │           └── lab
│   │   │               └── functions
│   │   │                   ├── App.kt
│   │   │                   ├── question
│   │   │                   │   ├── QuestionFunctions.kt
│   │   │                   │   ├── domain
│   │   │                   │   │   ├── Answer.kt
│   │   │                   │   │   └── Question.kt
│   │   │                   │   └── service
│   │   │                   │       ├── QuestionService.kt
│   │   │                   │       └── openai
│   │   │                   │           ├── OpenAiClient.kt
│   │   │                   │           └── OpenAiDTOs.kt

Our "question" domain application model would be the following:

// input
data class Question(val messages: List<String>)

// output
data class Answer(val answer: String)

The question function will be defined as @Bean definition, following the same approach as before and providing just an entry point to the application:

@Configuration
class QuestionFunctions {

    @Bean
    fun question(questionService: QuestionService): (Flux<Question>) -> Flux<Answer> = questionService::answerQuestion

}

The business logic would be handled by QuestionService:

@Service
class QuestionService(val openAiClient: OpenAiClient) {

    fun answerQuestion(questions: Flux<Question>): Flux<Answer> =
        questions.map { openAiClient.chatCompletion(it.messages) }
            .flatMap { it.map(responseToAnswer()) }

    private fun responseToAnswer(): (OpenAiChatCompletionResponse) -> (Answer) = {
        Answer(it.choices.map(OpenAiChatAnswerChoice::message).map(OpenAiMessage::content).joinToString("\n"))
    }

}

And finally the OpenAiClient injected in QuestionService similarly will perform the HTTP calls to OpenAI API, being responsible for injecting our Open AI API Key as Authorization header.

The HTTP client method signature requires a payload of type OpenAiChatCompletionRequest, which will be automatically serialized to JSON.

@ConfigurationProperties("app.openai")
data class OpenAiProperties (
    val baseUrl: String,
    val timeout: Duration,
    val apiKey: String,
    val model: String
)

@Component
class OpenAiClient(private val openAiClientProperties: OpenAiProperties, webClientBuilder: WebClient.Builder) {

    private val logger: Logger = LoggerFactory.getLogger(this::class.java)

    private val openAiHttpClient: OpenAiHttpClient = HttpServiceProxyFactory
        .builder(
            WebClientAdapter.forClient(
                webClientBuilder
                    .baseUrl(openAiClientProperties.baseUrl)
                    .clientConnector(ReactorClientHttpConnector(HttpClient.create().responseTimeout(openAiClientProperties.timeout)))
                    .defaultHeader("Authorization", "Bearer ${openAiClientProperties.apiKey}")
                    .defaultStatusHandler(not(HttpStatusCode::is2xxSuccessful)) {
                        Mono.error(OpenAiException(it.statusCode()))
                    }
                    .build()
            )
        )
        .build()
        .createClient()

    fun chatCompletion(messages: List<String>): Flux<OpenAiChatCompletionResponse> =
        openAiHttpClient.chatCompletion(OpenAiChatCompletionRequest(openAiClientProperties.model, messages.map { OpenAiMessage("user", it) }))
            .doOnError { logger.error("An error occurred while calling OpenAI for chat completion", it) }

}

private class OpenAiException(
    status: HttpStatusCode,
    message: String? = "Error: OpenAI API responded with $status") : ResponseStatusException(status, message)

private fun interface OpenAiHttpClient {

    @PostExchange("chat/completions")
    fun chatCompletion(@RequestBody openAiChatCompletionRequest: OpenAiChatCompletionRequest): Flux<OpenAiChatCompletionResponse>

}

Note: Both OpenAiChatCompletionRequest and OpenAiChatCompletionResponse classes are the corresponding DTOs for the API call, omitted in the article - you can check the full source code on GitHub.

Finally, we will define the Open AI API configuration properties under application.yml:

app:
  open-meteo:
  # ...
  openai:
    base-url: "https://api.openai.com/v1/"
    timeout: "PT10S"
    api-key: ${OPENAI_API_KEY}
    model: "gpt-3.5-turbo-16k"

To avoid hard-coding the Open AI API Key in our source code we can resolve it from an environment variable such as OPENAI_API_KEY:

export OPENAI_API_KEY=...

At this point, by running again our application and invoking POST localhost:8080/question with a valid body, it should return us the answer from the OpenAI Chat GPT model.

POST localhost:8080/question
Accept: application/json
Content-Type: application/stream+json

[
  {
    "messages": [
      "Can you list the top 5 Java Frameworks?",
      "Please only include the name."
    ]
  },
  {
    "messages": [
      "Which is the capital of Catalonia?"
    ]
  }
]

And the response:

[
  {
    "answer": "1. Spring\n2. Hibernate\n3. Apache Struts\n4. Play Framework\n5. Grails"
  },
  {
    "answer": "The capital of Catalonia is Barcelona."
  }
]

Note: when sending multiple simultaneous requests to Open AI, we are at risk to surpass the requests per minute (RPM) limits of each model quickly. Please read more about it in Open AI's official documentation.

🔌 The "Glue Function": transforming a Forecast to a Question

To somehow transform a Forecast function output to something the Question function can send to the AI model, we will need a "glue function" named forecastToQuestion.

We will implement all our application glue functions under a new GlueFunctions @Configuration class under a new "glue" package. With this approach, we can easily identify functions that mix different features/domains, keeping the features themselves isolated from the others.

├── src
│   ├── main
│   │   ├── kotlin
│   │   │   └── tech
│   │   │       └── aaregall
│   │   │           └── lab
│   │   │               └── functions
│   │   │                   ├── App.kt
│   │   │                   ├── glue
│   │   │                   │   └── GlueFunctions.kt

The forecastToFunction definition would be as follows:

@Configuration
class GlueFunctions {

    @Bean
    fun forecastToQuestion(): (Flux<Forecast>) -> Flux<Question> = {
        it.map { forecast -> Question(listOf(
            "Given the following weather forecast on coordinates ${forecast.geoLocation.latitude}, ${forecast.geoLocation.longitude}:",
            forecast.hourlyForecasts.joinToString("\n"),
            "Write a 1 to 3 sentence summary of the weather forecast in common vocabulary.",
            "Include the start and end dates of the period in the following format: Monday 1st of January",
            "Use the character º to indicate the temperature is in Celsius.",
            "Include highest and lowest temperature indications of the whole period.",
            "Include the chance of rain.",
            "Include the location name in the following format: City (Country).",
        )) }
    }

}

As you see we are constructing what in AI is known as the prompt.

This glue function is key to our weather forecast AI-powered application: it exposes some weather forecast data inside a prompt and asks the AI to generate a summary about it.

To accomplish a readable format of the HourlyForecast class, we can override the toString() method as follows:

data class HourlyForecast (
    val time: LocalDateTime,
    val temperature: Float,
    val precipitation: Float
) {

    override fun toString(): String {
        return "At $time, temperature will be $temperature º, precipitation of $precipitation mm"
    }
}

At this point, we can already put into practice an example of chaining functions.

We will invoke POST localhost:8080/forecast,forecastToQuestion with a valid body containing a Geo Location coordinates object, and it should return us the forecast in a Question format.

POST localhost:8080/forecast,forecastToQuestion
Accept: application/json
Content-Type: application/stream+json

# Barcelona => 41.3874° N, 2.1686° E

{
  "latitude": 41.3874,
  "longitude": 2.1686
}

The response (forecast shortened):

[
  {
    "messages": [
      "Given the following weather forecast on coordinates 41.39, 2.17:",
      "At 2023-07-01T00:00, temperature will be 24.4 º, precipitation of 0.0 mm\nAt 2023-07-01T01:00, temperature will be 21.0 º, precipitation of 0.0 mm...",
      "Write a 1 to 3 sentence summary of the weather forecast in common vocabulary.",
      "Include the start and end dates of the period in the following format: Monday 1st of January",
      "Use the character º to indicate the temperature is in Celsius.",
      "Include highest and lowest temperature indications of the whole period.",
      "Include the chance of rain.",
      "Include the location name in the following format: City (Country)."
    ]
  }
]

🪄 Getting a weather forecast from AI

Now that we have all our three functions implemented, we can invoke our application making use of function composition to get a weather forecast for a given location provided by Open AI.

Probably you guessed it right!

We will perform a POST /localhost:8080/forecast,forecastToQuestion,question request sending the Geo Location coordinates.

POST localhost:8080/forecast,forecastToQuestion,question
Accept: application/json
Content-Type: application/stream+json

{
  "latitude": 41.3874,
  "longitude": 2.1686
}

The request will invoke the three functions chained in order, bypassing the output of each one as input to the next one.

Under the hood:

  • forecast: request body will be marshaled as a Flux<GeoLocation> and transformed to a Flux<Forecast> (by calling OpenMeteo).

  • forecastToQuestion: the Flux<Forecast> output from the previous function will be transformed to Flux<Question>.

  • question: the Flux<Question> output from the previous function will be transformed to Flux<Answer> (by calling Open AI).

The response would be somehow similar to:

{
    "answer": "The weather forecast for Barcelona (Spain) from July 1st to July 7th shows temperatures ranging from 20.7 ºC to 29.5 ºC, with no precipitation expected throughout the period."
}

Note that Open AI answers may differ from each invocation, as the model does not always return the same responses in terms of text and grammatical syntax.

🧪 Testing our functions

We can write automated integration tests of our functions by taking advantage of Spring Boot's first class support for Testcontainers. When our tests run, we will start a MockServer container and fake the Open Meteo and Open AI responses, writing some expectations.

We could implement a test configuration class that defines:

  • A WebTestClient bean bound to the application context, which we will be able to inject into all our test classes.

  • A MockServerContainer running on the version of the MockServer library dependency we are using.

  • A MockServerClient bound to the MockServerContainer host and port, re-usable across all our integration tests.

@Configuration
class TestConfig {

    @Bean
    fun webTestClient(applicationContext: ApplicationContext): WebTestClient =
        WebTestClient.bindToApplicationContext(applicationContext).build()

    @Configuration
    class MockServerConfig {

        @Bean
        fun mockServerContainer(): MockServerContainer = MockServerContainer(
            DockerImageName.parse("mockserver/mockserver")
                .withTag(MockServerClient::class.java.`package`.implementationVersion))

        @Bean
        fun mockServerClient(mockServerContainer: MockServerContainer): MockServerClient =
            MockServerClient(mockServerContainer.host, mockServerContainer.serverPort)
    }
}

A happy path integration test for our question function would be as follows:

@SpringBootTest
class QuestionFunctionsTest(
    @Autowired val webTestClient: WebTestClient,
    @Autowired val mockServerClient: MockServerClient,
    @Autowired val openAiProperties: OpenAiProperties) {

    @Test
    fun `When OpenAI API returns results, then returns OK and response contains Answer`() {
        mockServerClient.reset()
            .`when`(
                request()
                    .withMethod(POST.name())
                    .withPath("/chat/completions")
                    .withHeader("Authorization", "Bearer ${openAiProperties.apiKey}")
            ).respond(
                response()
                    .withStatusCode(HttpStatus.OK.value())
                    .withHeader(CONTENT_TYPE, APPLICATION_JSON_VALUE)
                    .withBody("""
                            {
                              "id": "chatcmpl-123",
                              "object": "chat.completion",
                              "created": 1677652288,
                              "choices": [
                                {
                                    "index": 0,
                                    "message": {
                                        "role": "assistant",
                                        "content": "Hello there, glad you are so polite!"
                                    }
                                },
                                {
                                    "index": 1,
                                    "message": {
                                        "role": "assistant",
                                        "content": "How may I assist you today?"
                                    },
                                    "finish_reason": "stop"
                                }
                              ],
                              "usage": {
                                "prompt_tokens": 9,
                                "completion_tokens": 12,
                                "total_tokens": 21
                              }
                            }
                        """.trimIndent()))

        webTestClient
            .post()
            .uri("/question")
            .header(CONTENT_TYPE, APPLICATION_JSON_VALUE)
            .bodyValue("""
                    {"messages": ["Could you please say hello?", "Thank you!"]}
                """.trimIndent())
            .exchange()
            .expectHeader().contentType(APPLICATION_JSON_VALUE)
            .expectStatus().is2xxSuccessful
            .expectBody()
            .jsonPath("$").isArray
            .jsonPath("$").isNotEmpty
            .jsonPath("$[0].answer").isEqualTo("Hello there, glad you are so polite!\nHow may I assist you today?")
    }

}

Please note we need to override the OpenAI API URL for our integration tests - check out the repository source code on GitHub to see the complete integration test setup.

🚀 Going native with GraalVM

Now that our application is implemented and tested, we might want to build a native executable to take profit from Spring Boot AOT compilation benefits, not only by not needing a JVM to run our application but also by drastically improving the start-up time.

Registering reflection bindings

When building a native executable, we will need to register reflection hints for all our application domain classes that need to be serialized (typically input/output classes).

This means that in our application source code, we need to declare reflection hints for all the classes that are, for example, serialized/deserialized from/to JSON.

To do so, we will create a new @Configuration class named NativeConfig:

├── src
│   ├── main
│   │   ├── kotlin
│   │   │   └── tech
│   │   │       └── aaregall
│   │   │           └── lab
│   │   │               └── functions
│   │   │                   ├── App.kt
│   │   │                   ├── NativeConfig.kt

We will use Spring 6's @RegisterReflectionForBinding annotation to instruct Spring AOT compiler to generate native hints for all our domain classes.

From the annotation JavaDoc:

Indicates that the classes specified in the annotation attributes require some reflection hints for binding or reflection-based serialization purposes. For each class specified, hints on constructors, fields, properties, record components, including types transitively used on properties and record components are registered.

It's proved that it also supports Kotlin data classes.

@Configuration
@RegisterReflectionForBinding(classes = [
    GeoLocation::class,
    Forecast::class,
    Question::class,
    Answer::class,
    OpenAiChatCompletionRequest::class,
    OpenAiChatCompletionResponse::class,
    OpenMeteoForecastResponse::class
])
class NativeConfig

Declaring the Proxy Interfaces of our HTTP clients

As explained, we have implemented our Open Meteo and Open AI HTTP clients using Spring HTTP Interfaces. The instances of those interfaces are generated automatically by Spring using Java dynamic proxies and invoked using reflection on runtime.

As GraalVM native images do not provide any way to generate and interpret bytecode at runtime, all dynamic proxy classes need to be generated at native image build time, by making use of code static analysis.

This can be achieved in different ways, on of them is creating a proxy-config.json file defining the list of interfaces required to build the proxy classes.

For our application, we will create the proxy-config.json file placed under the directory src/main/resources/META-INF/native-image:

├── src
│   ├── main
│   │   └── resources
│   │       ├── META-INF
│   │       │   └── native-image
│   │       │       └── proxy-config.json
│   │       └── application.yml

The proxy-config.json file should contain an array of objects, each of them with an interfaces property as an array of Strings containing the interfaces needed to implement in significant order.

[
  {
    "interfaces": [
      "tech.aaregall.lab.functions.weather.service.openmeteo.OpenMeteoHttpClient",
      "org.springframework.aop.SpringProxy",
      "org.springframework.aop.framework.Advised",
      "org.springframework.core.DecoratingProxy"
    ]
  },
  {
    "interfaces": [
      "tech.aaregall.lab.functions.question.service.openai.OpenAiHttpClient",
      "org.springframework.aop.SpringProxy",
      "org.springframework.aop.framework.Advised",
      "org.springframework.core.DecoratingProxy"
    ]
  }
]

Check out GraalVM official docs on Dyanmic Proxy generation for more details.

Running our application natively

At this point, our application should be fully compatible with GraalVM and native executables.

We could run ./gradlew clean nativeRun on our terminal to execute our application natively.

Alternatively, we could build the native executable and execute it in two different steps:

$ ./gradlew clean nativeCompile
$ ./build/native/nativeCompile/aws-lambda-spring-cloud-function

2023-07-01T13:36:36.135+02:00  INFO 74417 --- [           main] tech.aaregall.lab.functions.AppKt        : Starting AOT-processed AppKt using Java 17.0.6 with PID ...
2023-07-01T13:36:36.165+02:00  INFO 74417 --- [           main] o.s.b.web.embedded.netty.NettyWebServer  : Netty started on port 8080
2023-07-01T13:36:36.165+02:00  INFO 74417 --- [           main] tech.aaregall.lab.functions.AppKt        : Started AppKt in 0.036 seconds (process running for 0.043)

Remember native image compilation requires a JDK GraalVM distribution.

java --version
openjdk 17.0.6 2023-01-17
OpenJDK Runtime Environment GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13, mixed mode, sharing)

☁️ Deploying the application as an AWS Lambda with a native runtime

Now that our Spring Boot application can be run as a native executable and its start time takes less than half a second, it's a great candidate to deploy it on a serverless platform such as AWS Lambda.

Adding Spring Cloud Function AWS adapter dependency

Spring Cloud Function project already provides a library that helps developers deploy their functions on AWS Lambdas. It contains the necessary classes to implement a custom runtime event loop, resolve the Lambda destinations, or automatically convert AWS Event Messages to objects that our Functions can understand, so we do not need to modify our application code to consume APIGatewayV2HTTPEvent, SQSEvents, for example, and just work with plain application POJOs.

All we need to do is import the spring-cloud-function-adapter-aws library in our dependencies:

implementation("org.springframework.cloud:spring-cloud-function-web")
implementation("org.springframework.cloud:spring-cloud-function-adapter-aws")

Defining our AWS infrastructure with a SAM

An AWS CloudFormation template is a formatted text file (JSON or YAML) that describes your AWS infrastructure.

AWS SAM (Serverless Application Model) has its particular template anatomy, which is very similar to the CloudFormation ones but with a few differences.

The serverless infrastructure of our application would be the following:

  • An HTTP API Gateway, with a Stage that we will use for Metrics and Monitoring.

  • A Permission that grants the HTTP API Gateway to invoke our Lambda application.

  • Our Lambda application itself (the native Spring Cloud Function we have just developed) with an Execution role.

    • Will run in a provided.al2 runtime on arm64 architecture.

  • A LogGroup bounded to the Lambda application to define the log retention period (30 days).

We can then create a template.yml file at the root of our project with the following content to define this infrastructure:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: tech.aaregall.lab:aws-lambda-spring-cloud-function

Parameters:
  OpenAiApiKey:
    Type: String
    Description: OpenAI API key

Globals:
  Api:
    EndpointConfiguration: REGIONAL # API Gateway regional endpoints
  Function:
    Timeout: 20
    MemorySize: 1512
    Runtime: provided.al2
    Architectures:
      - arm64
    Tracing: Active # https://docs.aws.amazon.com/lambda/latest/dg/lambda-x-ray.html

Resources:
  # Lambda Function
  SpringCloudFunctionLambda:
    Type: AWS::Serverless::Function
    Properties:
      Handler: none # Native runtime does not need a handler
      CodeUri: .
      Policies: AWSLambdaBasicExecutionRole
      Environment:
        Variables:
          OPENAI_API_KEY: !Ref OpenAiApiKey
      Events:
        HttpApiEvent:
          Type: HttpApi
          Properties:
            TimeoutInMillis: 20000
            PayloadFormatVersion: '1.0'
    Metadata:
      BuildMethod: makefile # Instruct SAM how to build application.

  # Lambda LogGroup
  SpringCloudFunctionLambdaLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Join [ '/', [ '/aws/lambda', !Ref SpringCloudFunctionLambda ] ]
      RetentionInDays: 30

Outputs:
  SpringCloudFunctionLambda:
    Description: AWS Lambda Spring Cloud Function
    Value: !Sub 'https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com'
    Export:
      Name: SpringCloudFunctionLambda

Notice we defined an OpenAiApiKey parameter that we will need to provide when deploying our SAM.

The value of that parameter will be defined as the value for OPENAI_API_KEY environment variable on our Lambda. Thus, the application.yml placeholder that we defined before, will resolve it from the SAM parameter.

Also, notice we define the BuildMethod as a makefile, which means we will instruct SAM on how to build the executable of our Lambda using a Makefile.

Defining the AWS Labmda bootstrap with a Makefile

Following the AWS custom runtimes guide, we need to create a file named bootstrap on our lambda deployment package.

We will generate that by adding a Makefile to the root of our project with the following content:

build-SpringCloudFunctionLambda:
    ./gradlew --no-daemon clean nativeCompile
    echo '#!/bin/sh' > ./build/bootstrap
    echo 'set -euo pipefail' >> ./build/bootstrap # Exit immediately if any command fails, propagate errors in pipelines.
    echo './aws-lambda-spring-cloud-function' >> ./build/bootstrap
    chmod +x ./build/bootstrap
    cp ./build/bootstrap $(ARTIFACTS_DIR)
    cp ./build/native/nativeCompile/aws-lambda-spring-cloud-function $(ARTIFACTS_DIR)

Notice we are using the ./gradlew --no-daemon clean nativeCompile command to generate the native executable.

The placeholder $(ARTIFACTS_DIR) refers to AWS SAM artifact dir (by default .aws-sam).

The content of our bootstrap file will then effectively be:

#!/bin/sh
set -euo pipefail
./aws-lambda-spring-cloud-function

Building a Docker image to run our SAM build

To make sure the generated native executable of our application is compatible with the AWS Lambda provided.al2 runtime (Linux), we will need to run our SAM build in a Docker container.

This is not necessary if you are using a concrete distribution of Linux in your local machine, but if you are using a Mac or Windows, the runtime of your local machine is different from the AWS Linux one so when trying to execute a SAM deployment package in AWS that is not built in a machine compatible with AWS Linux, it will just do nothing.

Therefore, to deploy a fully compatible package to AWS, is a good practice to run the build of your applications inside a Docker container compatible with provided.al2 (honestly this took me quite some time to figure it out empirically).

To accomplish this, we will create a new directory named aws-image at the root of our project:

├── aws-image
│   ├── Dockerfile
│   └── build-aws-image.sh

The Dockerfile will use the public AWS base image, and install SDKMan! and a JDK 17 with GraalVM 22.3.x on the container, as well as Python and some required pip dependencies from AWS needed to build the application.

# Image used to build the native application AWS artifact in a container with an OS compatible with Amazon Linux.
FROM public.ecr.aws/amazonlinux/amazonlinux:2

RUN yum -y update \
    && yum install -y zip unzip tar gzip bzip2-devel ed gcc gcc-c++ gcc-gfortran \
    less libcurl-devel openssl openssl-devel readline-devel xz-devel \
    zlib-devel glibc-static libcxx libcxx-devel llvm-toolset-7 zlib-static \
    && rm -rf /var/cache/yum

# Install SDKMan and Java GraalVM
RUN curl -s "https://get.sdkman.io" | bash
RUN source "$HOME/.sdkman/bin/sdkman-init.sh" && sdk install java 22.3.1.r17-grl
ENV JAVA_HOME "/root/.sdkman/candidates/java/current"

# Install AWS Lambda Builders
RUN amazon-linux-extras enable python3.8
RUN yum clean metadata && yum -y install python3.8
RUN curl -L get-pip.io | python3.8
RUN pip3 install aws-lambda-builders

The build-aws-image.sh file will be a simple script to build the Docker image with a tag name from any directory of our local filesystem.

#!/bin/sh

SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
set -e
docker build -t tech.aaregall.lab/amazonlinux-graalvm:latest "$SCRIPT_DIR"

Building and deploying our AWS application

The next steps are building the SAM deployment package and deploying it to our favorite AWS region, using sam build and sam deploy CLI commands.

Building the SAM deployment package

We will first build and tag the Docker image based on Amazon Linux with GraalVM installed, and verify the image is correctly built.

$ ./aws-image/build-aws-image.sh

$ docker image ls tech.aaregall.lab/amazonlinux-graalvm
REPOSITORY                              TAG       IMAGE ID       CREATED         SIZE
tech.aaregall.lab/amazonlinux-graalvm   latest    27e8a73380b0   7 seconds ago   1.54GB

Note this step does not need to be repeated every time we want to deploy our SAM package, as long as the image is present.

Then we can instruct SAM to build our deployment package in a Docker container using our custom image with the following command:

sam build --use-container --build-image tech.aaregall.lab/amazonlinux-graalvm:latest

This command will start a Docker container and will build our application project inside of it with the instructions provided in the Makefile.

By default, it will use the Gradle Wrapper of our project, which will be downloaded in the container itself. Alternatively, you could customize the image and install Gradle with a RUN command on the Docker file, but using the Gradle wrapper simplifies and centralizes the whole process for this article's purpose.

The whole build process takes around 3 minutes to complete on a Mac M1 with 32GB of RAM.

Once finished, the .aws-sam directory should have been generated with the following structure:

.aws-sam
├── build
│   ├── SpringCloudFunctionLambda
│   │   ├── aws-lambda-spring-cloud-function
│   │   └── bootstrap
│   └── template.yaml
└── build.toml

Deploying the SAM package to AWS

At this point, we should be ready to deploy the SAM deployment package to our AWS region.

First, we will export a couple of environment variables:

export AWS_REGION=...
export OPENAI_API_KEY=...

And secondly, run the sam deploy specifying the AWS region and the OpenAiApiKey parameter:

sam deploy --region $AWS_REGION --parameter-overrides ParameterKey=OpenAiApiKey,ParameterValue=$OPENAI_API_KEY

I recommend running it with --guided argument the first time, as the CLI will prompt us to configure some settings for the deployment, for example, if the deployment has the capability to create IAM roles or to auto-resolve the S3 bucket where our deployment package will be stored.

We can define a samconfig.toml file with the following default deploy parameters to avoid being prompted for the same options every time we run sam deploy:

version = 0.1
[default.deploy.parameters]
stack_name = "aws-lambda-spring-cloud-function"
resolve_s3 = true
s3_prefix = "aws-lambda-spring-cloud-function"
confirm_changeset = true
capabilities = "CAPABILITY_IAM"
disable_rollback = false

With these defaults, the CloudFormation resources will be added to the stack changeset, and after that, we will need to confirm to deploy the changes.

Once confirmed, CloudFormation events will start to refresh, and finally produce the expected Output: the URL of the API Gateway to our AWS Lambda! 🎉

If we navigate to our AWS Console > CloudFormation section, we should see the application stack created with all the resources described above:

Also, you can inspect how the resources were generated, for example, our Lambda named "SpringCloudFunctionLambda":

Finally, we can test our application by calling the API Gateway URL like a regular endpoint, although we will need to provide the function definition using a special Spring Cloud Function HTTP header (spring.cloud.function.definition):

POST https://abcdefghij.execute-api.xxx.amazonaws.com
Accept: application/json
Content-Type: application/json
spring.cloud.function.definition: forecast,forecastToQuestion,question

{
  "latitude": 41.3874,
  "longitude": 2.1686
}

HTTP/1.1 200 OK
Date: Sat, 01 Jul 2023 16:43:46 GMT
Content-Type: application/json
Connection: keep-alive
Apigw-Requestid: Ha82tghuliAEPvQ=

{
    "answer": "The weather forecast for Barcelona (Spain) from July 1st to July 7th shows temperatures ranging from 20.7 ºC to 29.5 ºC, with no precipitation expected throughout the period."
}

If we prefer to only expose one concrete function, or a concrete function composition, and don't let the API Gateway consumer decide the function definition itself, we can always define the spring.cloud.function.definition configuration property on our application.yml: in that case, the request will not require the function definition header.

Limitations and troubleshooting

Worth to be mentioned, as of the time of this writing, the Spring Cloud Function AWS adapter had some limitations.

First, you need to know that if your approach is letting the client/event producer define the function to be executed (using spring.cloud.function.definition header as we have seen), the AWS adapter library will set that value as a Spring Environment property, meaning that while the lambda is running, that will be the function definition for every request/event that interacts with the lambda until the process shuts down and a new invocation comes.

This means two consecutive invocations to the same Lambda using a different function definition header, will effectively invoke the function defined by the very first invocation while the Lambda process is running. Please check the AWS Lambda runtime environment official docs and this GitHub issue if you are interested in knowing more about this bug. (Edit September 26th, 2023: the reported issue was fixed on Spring Cloud Function 4.0.5 ✅ - the issue was only reproducible when using Kotlin).

Another limitation I found during my explorations is related to reactive types body mapping, meaning the AWS adapter is not fully able to map an array of GeoLocations to a Flux<GeoLocation> object.

Anyhow, with individual objects on the body as showcased, it works perfectly, which is fine for this article. Please check this GitHub issue if you are interested in this detail. (Edit July 13th, 2023: the reported issue was fixed on Spring Cloud Function 4.0.5 ✅)

If we want to troubleshoot or inspect the logs of our application running on AWS, we can easily do that from our terminal using the following command:

sam logs --stack-name aws-lambda-spring-cloud-function

I recommend enabling the DEBUG level on Spring Cloud Functions AWS Adapter package to fully understand the lifecycle of invocations to our applications. It provides helpful insights about how everything works under the hood, and especially what actions the library performs for us, like function lookup, message conversion, etc.

logging:
  level:
    org.springframework.cloud.function.adapter.aws: DEBUG

Conclusions

This article demonstrated:

  • How to set up a Spring Cloud Function application project from scratch.

  • How to keep our focus on implementing functions with business logic taking benefit from Spring Framework abstractions and Spring Boot features, like dependency injection, automatic POJO JSON serialization, or auto-configuration.

  • How to integrate our Spring applications with different APIs using HTTP Interfaces.

  • How to write integration tests for Spring Cloud Function applications.

  • How to generate a native executable using GraalVM that we can deploy to a serverless platform.

  • How to set up an AWS SAM build process and how to deploy that package to AWS as an AWS Lambda using a custom native runtime - a process that is not straightforward and requires some attention from us as developers.

  • Know which are the limitations of Spring Cloud Function application when deployed as AWS Lambdas.

The source code of the application developed is available in the linked GitHub repository.

If you enjoyed the article, let me know by dropping a star ⭐️ on the GitHub repository!

And of course, you can also give me feedback by writing a comment or reaching me directly through my linked social networks. ☺️

AWSopenaiKotlinSpringbootSpring Cloud Function