Exploring Spring Cloud Function, AWS Lambda and AI: building a native serverless application with Kotlin powered by OpenAI.
Table of contents
- Introduction
- βοΈ Setting up our project
- Implementing our Functions
- π Going native with GraalVM
- βοΈ Deploying the application as an AWS Lambda with a native runtime
- Conclusions
Introduction
Last May 2023, I had the pleasure to attend the Spring I/O Conference (10th edition!) in Barcelona, Spain. On the last day, Oleg Zhurakousky, one of the core maintainers of Spring Cloud Function, showcased the project in a talk that I enjoyed quite a lot. I got quite impressed by the capabilities of the project.
Since the first half of 2023 the tech landscape has been hyped by OpenAI products and IA in general, and Spring ecosystem is making big steps forward to bring JVM based applications to a native runtime, I thought it would be quite interesting to explore both the integration of Spring Cloud Function applications with OpenAI being deployed natively to a cloud provider such as AWS.
This article is the outcome of such exploration. I hope you enjoy it!
Overview
This article walks through the process of building and deploying a native serverless application as an AWS Lambda that interacts with OpenAI API, implemented using Spring Cloud Function and Kotlin.
The goal is to demonstrate:
How to implement and test functions using Spring Cloud Function and Kotlin.
How to integrate our functions with the OpenAI GPT model using their API.
How to build a serverless native executable of the application using GraalVM.
How to deploy the application as an AWS Lambda with a native runtime using SAM (Serverless Application Model).
Our application will be an AI-powered weather API: given some GPS coordinates, it will return us a quite accurate weather forecast for the given location for the next 7 days in a human-friendly way, as we asked a real meteorologist.
Example:
The weather forecast for Barcelona (Spain) from July 1st to July 7th shows temperatures ranging from 20.7 ΒΊC to 29.5 ΒΊC, with no precipitation expected throughout the period.
TLDR; show me the code!
The full source code of the article is published on GitHub π.
https://github.com/ArnauAregall/aws-lambda-spring-cloud-function
Recommended pre-steps
To follow the walk-through steps I assume you meet the following requirements:
Some general knowledge about:
Spring Framework, Spring Boot and Spring WebFlux.
Testcontainers.
AWS, especially Lambda and CloudFormation.
The following software is installed:
A Java 17+ GraalVM runtime (I recommend using π SDKMan!).
Docker.
AWS CLI is correctly configured with your AWS account - in other words,
aws iam list-users
command should work for you (π official AWS CLI guide).AWS SAM CLI installed (π official AWS SAM CLI guide).
An OpenAI API key (π official OpenAI API developer guide).
πβοΈ What is Spring Cloud Function?
With Spring Cloud Function, we can write functions as standalone units of code in our Spring Boot applications, and expose them as RESTful APIs, message-driven services, or event handlers.
From the official reference documentation:
Spring Cloud Function is a project with the following high-level goals:
Promote the implementation of business logic via functions.
Decouple the development lifecycle of business logic from any specific runtime target so that the same code can run as a web endpoint, a stream processor, or a task.
Support a uniform programming model across serverless providers, as well as the ability to run standalone (locally or in a PaaS).
Enable Spring Boot features (auto-configuration, dependency injection, metrics) on serverless providers.
It abstracts away all of the transport details and infrastructure, allowing the developer to keep all the familiar tools and processes, and focus firmly on business logic.
One of its key advantages is its ability to abstract away the underlying serverless platform, meaning we can develop our functions using Spring's abstractions and then deploy them on various serverless platforms like AWS Lambda, Azure Functions, or Google Cloud Functions without making significant changes to our code.
One of the features I liked the most is the ability to invoke functions in a composite way, meaning that each function has its input and produces its output, and those functions can be chained.
Let's understand better how Spring Cloud Function works with a simple example.
Understanding by example: the "Hello World" with Spring Cloud Function
A "Hello World" application using Spring Cloud Function, which is showcased a lot in the reference documentation, would be an application that declares two individual and isolated functions:
Given a String input, return it uppercased.
Given a String input, return it reversed.
@Configuration
class Functions {
@Bean
fun uppercase(): (String) -> String = String::uppercase
@Bean
fun reverse(): (String) -> String = String::reversed
}
@SpringBootApplication
class App
fun main(args: Array<String>) {
runApplication<App>(*args)
}
As surprising as this might seem, this is a valid and compliant Spring Cloud Function application code.
We declare our functions as regular Spring @Bean
s, and the framework automatically exposes HTTP endpoints for us to invoke them individually or via composition.
Invoking the uppercase
function:
POST localhost:8080/uppercase
Content-Type: text/plain
"Hello Spring Cloud Function!"
HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8
"HELLO SPRING CLOUD FUNCTION!"
Invoking the reverse
function:
POST localhost:8080/reverse
Content-Type: text/plain
"Hello Spring Cloud Function!"
HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8
"!noitcnuF duolC gnirpS olleH"
Invoking a composite function: reverse,uppercase
POST localhost:8080/reverse,uppercase
Content-Type: text/plain
"Hello Spring Cloud Function!"
HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8
"!NOITCNUF DUOLC GNIRPS OLLEH"
βοΈ Setting up our project
Scaffolding our project structure with Spring Intializr
We will delegate the bootiful Spring Initalizr our project bootstrapping.
Head to https://start.spring.io and build a new project with the following configuration:
Project: Gradle - Kotlin
Language: Kotlin
Spring Boot 3.1.1
Java 17
Dependencies:
Function
Spring Reactive Web (WebFlux)
GraalVM Native Support
Testcontainers
Spring Boot DevTools (optional)
Follow ππ this link to automatically generate the project with the mentioned configuration.
Reviewing the Gradle setup
Once downloaded, we will define our dependencies versions in a gradle.properties
file placed at the root of our project:
kotlinVersion=1.8.22
springBootVersion=3.1.1
springCloudVersion=2022.0.3
springBootDependencyManagementVersion=1.1.0
mockServerVersion=5.15.0
testcontainersVersion=1.18.3
graalVMBuildToolsVersion=0.9.23
nettyResolverDnsMacosVersion=4.1.94.Final
Note: This article was written on June 2023, probably by the time you are reading it new versions may have been released - stay up to date!
Then we will customize the plugin management under settings.gradle.kts
file to use the aimed versions of each plugin:
rootProject.name = "aws-lambda-spring-cloud-function"
pluginManagement {
val kotlinVersion: String by settings
val springBootVersion: String by settings
val springBootDependencyManagementVersion: String by settings
val graalVMBuildToolsVersion: String by settings
plugins {
id("org.springframework.boot") version springBootVersion
id("io.spring.dependency-management") version springBootDependencyManagementVersion
id("org.graalvm.buildtools.native") version graalVMBuildToolsVersion
kotlin("jvm") version kotlinVersion
kotlin("plugin.spring") version kotlinVersion
}
}
And finally, our build.gradle.kts
file would be the following:
// group, version, repositories...
plugins {
id("org.springframework.boot")
id("io.spring.dependency-management")
id("org.graalvm.buildtools.native")
kotlin("jvm")
kotlin("plugin.spring")
}
dependencies {
implementation("org.springframework.boot:spring-boot-starter-webflux")
implementation("org.springframework.cloud:spring-cloud-function-web")
implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor")
developmentOnly("org.springframework.boot:spring-boot-devtools")
testImplementation("org.springframework.boot:spring-boot-starter-test")
testImplementation("org.springframework.boot:spring-boot-testcontainers")
testImplementation("io.projectreactor:reactor-test")
testImplementation("org.testcontainers:mockserver:${property("testcontainersVersion")}")
testImplementation("org.mock-server:mockserver-client-java:${property("mockServerVersion")}")
testImplementation("io.netty:netty-resolver-dns-native-macos:${property("nettyResolverDnsMacosVersion")}:osx-aarch_64")
}
dependencyManagement {
imports {
mavenBom("org.springframework.cloud:spring-cloud-dependencies:${property("springCloudVersion")}")
}
}
tasks.withType<KotlinCompile> {
kotlinOptions {
freeCompilerArgs = listOf("-Xjsr305=strict")
jvmTarget = "17"
}
}
tasks.withType<Test> {
useJUnitPlatform()
}
As we will be using spring-boot-starter-webflux
, our runtime will be a Netty server, thus our application will be naturally asynchronous and nonblocking.
Please note we added Testcontainers, MockServer and Netty DNS resolver for macOS to later implement reactive integration tests:
testImplementation("org.testcontainers:mockserver:${property("testcontainersVersion")}")
testImplementation("org.mock-server:mockserver-client-java:${property("mockServerVersion")}")
testImplementation("io.netty:netty-resolver-dns-native-macos:${property("nettyResolverDnsMacosVersion")}:osx-aarch_64")
The Netty DNS resolver dependency can be omitted if your local machine does not run macOS, although if you do you might need to adjust the OS architecture classifier (the one provided in the example above is for Mac M1).
Implementing our Functions
Quoting the article introduction, the main purpose of our application would be:
Given some GPS coordinates, return a quite accurate weather forecast for that location for the next 7 days in a human-friendly way, as it was a real meteorologist.
Our application will expose 3 main functions:
Forecast
Function name:
forecast
Input: πGeo Location coordinates.
Output: π€οΈ The weather forecast for those coordinates fetched from OpenMeteo API.
Question
Function name:
question
Input: βQuestions for the AI model.
Output: π€ The Answer from the AI model to those questions.
Forecast to Question (aka "Glue Function")
Function name:
forecastToQuestion
Input: π€οΈ A Forecast from OpenMeteo API.
Output:β A Question asking for a weather forecast that an AI model can understand.
ππ€οΈ The Forecast function: calling OpenMeteo API
We will implement our Forecast function logic inside a "weather" package, following a package-by-feature strategy to prove that our function is self-contained and independent from the rest:
βββ src
β βββ main
β β βββ kotlin
β β β βββ tech
β β β βββ aaregall
β β β βββ lab
β β β βββ functions
β β β βββ App.kt
β β β βββ weather
β β β βββ WeatherFunctions.kt
β β β βββ domain
β β β β βββ Forecast.kt
β β β β βββ GeoLocation.kt
β β β βββ service
β β β βββ WeatherService.kt
β β β βββ openmeteo
β β β βββ OpenMeteoClient.kt
β β β βββ OpenMeteoDTOs.kt
This would be our "weather" domain model:
// input
data class GeoLocation (val latitude: Float, val longitude: Float)
// output
data class Forecast(
val geoLocation: GeoLocation,
val hourlyForecasts: List<HourlyForecast>
)
data class HourlyForecast (
val time: LocalDateTime,
val temperature: Float,
val precipitation: Float
)
We will then expose our forecast
function with a @Bean
definition:
@Configuration
class WeatherFunctions {
@Bean
fun forecast(weatherService: WeatherService): (Flux<GeoLocation>) -> Flux<Forecast> = weatherService::getForecast
}
Our function will be just as an entry point for our application, which has the WeatherService
component injected, who will be responsible for handling the business logic.
@Service
class WeatherService(val openMeteoClient: OpenMeteoClient) {
fun getForecast(geoLocations: Flux<GeoLocation>): Flux<Forecast> =
geoLocations.map { openMeteoClient.getForecast(it.latitude, it.longitude) }
.flatMap { it.map(responseToForecast()) }
private fun responseToForecast(): (OpenMeteoForecastResponse) -> (Forecast) = {
Forecast(GeoLocation(it.latitude, it.longitude),
it.hourly.time.mapIndexed {
timeIndex, localDateTime -> HourlyForecast(localDateTime, it.hourly.temperature[timeIndex], it.hourly.precipitation[timeIndex])
}.toList()
)
}
}
Finally the OpenMeteoClient
injected in WeatherService
will be the component that performs the HTTP calls to OpenMeteo REST API, using Spring Framework 6 HTTP Interfaces to build the rest client.
The HTTP client has a straightforward signature, where all the parameters are sent via query params.
@ConfigurationProperties("app.open-meteo")
data class OpenMeteoProperties (
val baseUrl: String,
val timeout: Duration,
val hourlyParams: List<String>
)
@Component
class OpenMeteoClient(private val openMeteoProperties: OpenMeteoProperties, webClientBuilder: WebClient.Builder) {
private val logger: Logger = LoggerFactory.getLogger(this::class.java)
private val openMeteoHttpClient: OpenMeteoHttpClient = HttpServiceProxyFactory
.builder(
WebClientAdapter.forClient(
webClientBuilder
.baseUrl(openMeteoProperties.baseUrl)
.clientConnector(ReactorClientHttpConnector(HttpClient.create().responseTimeout(openMeteoProperties.timeout)))
.defaultStatusHandler(not(HttpStatusCode::is2xxSuccessful)) {
Mono.error(OpenMeteoException(it.statusCode()))
}
.build()
)
)
.build()
.createClient()
fun getForecast(latitude: Float, longitude: Float): Flux<OpenMeteoForecastResponse> =
openMeteoHttpClient.getForecast(latitude, longitude, openMeteoProperties.hourlyParams.joinToString(","))
.doOnError { logger.error("An error occurred while calling OpenMeteo for forecast", it) }
}
private class OpenMeteoException(
status: HttpStatusCode,
reason: String? = "Error: OpenMeteo API responded with $status") : ResponseStatusException(status, reason)
private fun interface OpenMeteoHttpClient {
@GetExchange("/forecast")
fun getForecast(
@RequestParam("latitude") latitude: Float,
@RequestParam("longitude") longitude: Float,
@RequestParam("hourly") hourly: String
): Flux<OpenMeteoForecastResponse>
}
Note: OpenMeteoForecastResponse
is the corresponding DTO for the API response, omitted in the article - you can check the full source code on GitHub.
Finally, we will define the Open Meteo configuration properties under application.yml
:
app:
open-meteo:
base-url: "https://api.open-meteo.com/v1/"
timeout: "PT3S"
hourly-params:
- "temperature_2m"
- "precipitation"
At this point, by running our application and invoking POST localhost:8080/forecast
with a valid body with some GPS coordinates, it should return us the weather forecast for that location fetched from OpenMeteo API in our on format.
Example: Get the weather forecast for Barcelona and Freiburg from OpenMeteo API.
POST localhost:8080/forecast
Accept: application/json
Content-Type: application/stream+json
# Barcelona => 41.3874Β° N, 2.1686Β° E
# Freiburg => 47.9990Β° N, 7.8421Β° E
[
{
"latitude": 41.3874,
"longitude": 2.1686
},
{
"latitude": 47.9990,
"longitude": 7.8421
}
]
The response (shortened hourlyForecasts
array) would be:
[
{
"geoLocation": {
"latitude": 41.39,
"longitude": 2.17
},
"hourlyForecasts": [
{
"time": "2023-07-01T00:00:00",
"temperature": 24.4,
"precipitation": 0.0
},
{
"time": "2023-07-07T23:00:00",
"temperature": 22.7,
"precipitation": 0.0
}
]
},
{
"geoLocation": {
"latitude": 48.0,
"longitude": 7.8399997
},
"hourlyForecasts": [
{
"time": "2023-07-01T00:00:00",
"temperature": 17.4,
"precipitation": 0.0
},
{
"time": "2023-07-07T23:00:00",
"temperature": 14.7,
"precipitation": 0.0
}
]
}
]
βπ€ The Question function: calling OpenAI API
Following the same package-by-feature strategy, we will now implement all the Question and OpenAI-related code in a "question" package.
βββ src
β βββ main
β β βββ kotlin
β β β βββ tech
β β β βββ aaregall
β β β βββ lab
β β β βββ functions
β β β βββ App.kt
β β β βββ question
β β β β βββ QuestionFunctions.kt
β β β β βββ domain
β β β β β βββ Answer.kt
β β β β β βββ Question.kt
β β β β βββ service
β β β β βββ QuestionService.kt
β β β β βββ openai
β β β β βββ OpenAiClient.kt
β β β β βββ OpenAiDTOs.kt
Our "question" domain application model would be the following:
// input
data class Question(val messages: List<String>)
// output
data class Answer(val answer: String)
The question
function will be defined as @Bean
definition, following the same approach as before and providing just an entry point to the application:
@Configuration
class QuestionFunctions {
@Bean
fun question(questionService: QuestionService): (Flux<Question>) -> Flux<Answer> = questionService::answerQuestion
}
The business logic would be handled by QuestionService
:
@Service
class QuestionService(val openAiClient: OpenAiClient) {
fun answerQuestion(questions: Flux<Question>): Flux<Answer> =
questions.map { openAiClient.chatCompletion(it.messages) }
.flatMap { it.map(responseToAnswer()) }
private fun responseToAnswer(): (OpenAiChatCompletionResponse) -> (Answer) = {
Answer(it.choices.map(OpenAiChatAnswerChoice::message).map(OpenAiMessage::content).joinToString("\n"))
}
}
And finally the OpenAiClient
injected in QuestionService
similarly will perform the HTTP calls to OpenAI API, being responsible for injecting our Open AI API Key as Authorization
header.
The HTTP client method signature requires a payload of type OpenAiChatCompletionRequest
, which will be automatically serialized to JSON.
@ConfigurationProperties("app.openai")
data class OpenAiProperties (
val baseUrl: String,
val timeout: Duration,
val apiKey: String,
val model: String
)
@Component
class OpenAiClient(private val openAiClientProperties: OpenAiProperties, webClientBuilder: WebClient.Builder) {
private val logger: Logger = LoggerFactory.getLogger(this::class.java)
private val openAiHttpClient: OpenAiHttpClient = HttpServiceProxyFactory
.builder(
WebClientAdapter.forClient(
webClientBuilder
.baseUrl(openAiClientProperties.baseUrl)
.clientConnector(ReactorClientHttpConnector(HttpClient.create().responseTimeout(openAiClientProperties.timeout)))
.defaultHeader("Authorization", "Bearer ${openAiClientProperties.apiKey}")
.defaultStatusHandler(not(HttpStatusCode::is2xxSuccessful)) {
Mono.error(OpenAiException(it.statusCode()))
}
.build()
)
)
.build()
.createClient()
fun chatCompletion(messages: List<String>): Flux<OpenAiChatCompletionResponse> =
openAiHttpClient.chatCompletion(OpenAiChatCompletionRequest(openAiClientProperties.model, messages.map { OpenAiMessage("user", it) }))
.doOnError { logger.error("An error occurred while calling OpenAI for chat completion", it) }
}
private class OpenAiException(
status: HttpStatusCode,
message: String? = "Error: OpenAI API responded with $status") : ResponseStatusException(status, message)
private fun interface OpenAiHttpClient {
@PostExchange("chat/completions")
fun chatCompletion(@RequestBody openAiChatCompletionRequest: OpenAiChatCompletionRequest): Flux<OpenAiChatCompletionResponse>
}
Note: Both OpenAiChatCompletionRequest
and OpenAiChatCompletionResponse
classes are the corresponding DTOs for the API call, omitted in the article - you can check the full source code on GitHub.
Finally, we will define the Open AI API configuration properties under application.yml
:
app:
open-meteo:
# ...
openai:
base-url: "https://api.openai.com/v1/"
timeout: "PT10S"
api-key: ${OPENAI_API_KEY}
model: "gpt-3.5-turbo-16k"
To avoid hard-coding the Open AI API Key in our source code we can resolve it from an environment variable such as OPENAI_API_KEY
:
export OPENAI_API_KEY=...
At this point, by running again our application and invoking POST localhost:8080/question
with a valid body, it should return us the answer from the OpenAI Chat GPT model.
POST localhost:8080/question
Accept: application/json
Content-Type: application/stream+json
[
{
"messages": [
"Can you list the top 5 Java Frameworks?",
"Please only include the name."
]
},
{
"messages": [
"Which is the capital of Catalonia?"
]
}
]
And the response:
[
{
"answer": "1. Spring\n2. Hibernate\n3. Apache Struts\n4. Play Framework\n5. Grails"
},
{
"answer": "The capital of Catalonia is Barcelona."
}
]
Note: when sending multiple simultaneous requests to Open AI, we are at risk to surpass the requests per minute (RPM) limits of each model quickly. Please read more about it in Open AI's official documentation.
π The "Glue Function": transforming a Forecast to a Question
To somehow transform a Forecast function output to something the Question function can send to the AI model, we will need a "glue function" named forecastToQuestion
.
We will implement all our application glue functions under a new GlueFunctions
@Configuration
class under a new "glue" package. With this approach, we can easily identify functions that mix different features/domains, keeping the features themselves isolated from the others.
βββ src
β βββ main
β β βββ kotlin
β β β βββ tech
β β β βββ aaregall
β β β βββ lab
β β β βββ functions
β β β βββ App.kt
β β β βββ glue
β β β β βββ GlueFunctions.kt
The forecastToFunction
definition would be as follows:
@Configuration
class GlueFunctions {
@Bean
fun forecastToQuestion(): (Flux<Forecast>) -> Flux<Question> = {
it.map { forecast -> Question(listOf(
"Given the following weather forecast on coordinates ${forecast.geoLocation.latitude}, ${forecast.geoLocation.longitude}:",
forecast.hourlyForecasts.joinToString("\n"),
"Write a 1 to 3 sentence summary of the weather forecast in common vocabulary.",
"Include the start and end dates of the period in the following format: Monday 1st of January",
"Use the character ΒΊ to indicate the temperature is in Celsius.",
"Include highest and lowest temperature indications of the whole period.",
"Include the chance of rain.",
"Include the location name in the following format: City (Country).",
)) }
}
}
As you see we are constructing what in AI is known as the prompt.
This glue function is key to our weather forecast AI-powered application: it exposes some weather forecast data inside a prompt and asks the AI to generate a summary about it.
To accomplish a readable format of the HourlyForecast
class, we can override the toString()
method as follows:
data class HourlyForecast (
val time: LocalDateTime,
val temperature: Float,
val precipitation: Float
) {
override fun toString(): String {
return "At $time, temperature will be $temperature ΒΊ, precipitation of $precipitation mm"
}
}
At this point, we can already put into practice an example of chaining functions.
We will invoke POST localhost:8080/forecast,forecastToQuestion
with a valid body containing a Geo Location coordinates object, and it should return us the forecast in a Question format.
POST localhost:8080/forecast,forecastToQuestion
Accept: application/json
Content-Type: application/stream+json
# Barcelona => 41.3874Β° N, 2.1686Β° E
{
"latitude": 41.3874,
"longitude": 2.1686
}
The response (forecast shortened):
[
{
"messages": [
"Given the following weather forecast on coordinates 41.39, 2.17:",
"At 2023-07-01T00:00, temperature will be 24.4 ΒΊ, precipitation of 0.0 mm\nAt 2023-07-01T01:00, temperature will be 21.0 ΒΊ, precipitation of 0.0 mm...",
"Write a 1 to 3 sentence summary of the weather forecast in common vocabulary.",
"Include the start and end dates of the period in the following format: Monday 1st of January",
"Use the character ΒΊ to indicate the temperature is in Celsius.",
"Include highest and lowest temperature indications of the whole period.",
"Include the chance of rain.",
"Include the location name in the following format: City (Country)."
]
}
]
πͺ Getting a weather forecast from AI
Now that we have all our three functions implemented, we can invoke our application making use of function composition to get a weather forecast for a given location provided by Open AI.
Probably you guessed it right!
We will perform a POST /localhost:8080/forecast,forecastToQuestion,question
request sending the Geo Location coordinates.
POST localhost:8080/forecast,forecastToQuestion,question
Accept: application/json
Content-Type: application/stream+json
{
"latitude": 41.3874,
"longitude": 2.1686
}
The request will invoke the three functions chained in order, bypassing the output of each one as input to the next one.
Under the hood:
forecast
: request body will be marshaled as aFlux<GeoLocation>
and transformed to aFlux<Forecast>
(by calling OpenMeteo).forecastToQuestion
: theFlux<Forecast>
output from the previous function will be transformed toFlux<Question>
.question
: theFlux<Question>
output from the previous function will be transformed toFlux<Answer>
(by calling Open AI).
The response would be somehow similar to:
{
"answer": "The weather forecast for Barcelona (Spain) from July 1st to July 7th shows temperatures ranging from 20.7 ΒΊC to 29.5 ΒΊC, with no precipitation expected throughout the period."
}
Note that Open AI answers may differ from each invocation, as the model does not always return the same responses in terms of text and grammatical syntax.
π§ͺ Testing our functions
We can write automated integration tests of our functions by taking advantage of Spring Boot's first class support for Testcontainers. When our tests run, we will start a MockServer container and fake the Open Meteo and Open AI responses, writing some expectations.
We could implement a test configuration class that defines:
A
WebTestClient
bean bound to the application context, which we will be able to inject into all our test classes.A
MockServerContainer
running on the version of the MockServer library dependency we are using.A
MockServerClient
bound to the MockServerContainer host and port, re-usable across all our integration tests.
@Configuration
class TestConfig {
@Bean
fun webTestClient(applicationContext: ApplicationContext): WebTestClient =
WebTestClient.bindToApplicationContext(applicationContext).build()
@Configuration
class MockServerConfig {
@Bean
fun mockServerContainer(): MockServerContainer = MockServerContainer(
DockerImageName.parse("mockserver/mockserver")
.withTag(MockServerClient::class.java.`package`.implementationVersion))
@Bean
fun mockServerClient(mockServerContainer: MockServerContainer): MockServerClient =
MockServerClient(mockServerContainer.host, mockServerContainer.serverPort)
}
}
A happy path integration test for our question
function would be as follows:
@SpringBootTest
class QuestionFunctionsTest(
@Autowired val webTestClient: WebTestClient,
@Autowired val mockServerClient: MockServerClient,
@Autowired val openAiProperties: OpenAiProperties) {
@Test
fun `When OpenAI API returns results, then returns OK and response contains Answer`() {
mockServerClient.reset()
.`when`(
request()
.withMethod(POST.name())
.withPath("/chat/completions")
.withHeader("Authorization", "Bearer ${openAiProperties.apiKey}")
).respond(
response()
.withStatusCode(HttpStatus.OK.value())
.withHeader(CONTENT_TYPE, APPLICATION_JSON_VALUE)
.withBody("""
{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1677652288,
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello there, glad you are so polite!"
}
},
{
"index": 1,
"message": {
"role": "assistant",
"content": "How may I assist you today?"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 9,
"completion_tokens": 12,
"total_tokens": 21
}
}
""".trimIndent()))
webTestClient
.post()
.uri("/question")
.header(CONTENT_TYPE, APPLICATION_JSON_VALUE)
.bodyValue("""
{"messages": ["Could you please say hello?", "Thank you!"]}
""".trimIndent())
.exchange()
.expectHeader().contentType(APPLICATION_JSON_VALUE)
.expectStatus().is2xxSuccessful
.expectBody()
.jsonPath("$").isArray
.jsonPath("$").isNotEmpty
.jsonPath("$[0].answer").isEqualTo("Hello there, glad you are so polite!\nHow may I assist you today?")
}
}
Please note we need to override the OpenAI API URL for our integration tests - check out the repository source code on GitHub to see the complete integration test setup.
π Going native with GraalVM
Now that our application is implemented and tested, we might want to build a native executable to take profit from Spring Boot AOT compilation benefits, not only by not needing a JVM to run our application but also by drastically improving the start-up time.
Registering reflection bindings
When building a native executable, we will need to register reflection hints for all our application domain classes that need to be serialized (typically input/output classes).
This means that in our application source code, we need to declare reflection hints for all the classes that are, for example, serialized/deserialized from/to JSON.
To do so, we will create a new @Configuration
class named NativeConfig
:
βββ src
β βββ main
β β βββ kotlin
β β β βββ tech
β β β βββ aaregall
β β β βββ lab
β β β βββ functions
β β β βββ App.kt
β β β βββ NativeConfig.kt
We will use Spring 6's @RegisterReflectionForBinding
annotation to instruct Spring AOT compiler to generate native hints for all our domain classes.
From the annotation JavaDoc:
Indicates that the classes specified in the annotation attributes require some reflection hints for binding or reflection-based serialization purposes. For each class specified, hints on constructors, fields, properties, record components, including types transitively used on properties and record components are registered.
It's proved that it also supports Kotlin data classes.
@Configuration
@RegisterReflectionForBinding(classes = [
GeoLocation::class,
Forecast::class,
Question::class,
Answer::class,
OpenAiChatCompletionRequest::class,
OpenAiChatCompletionResponse::class,
OpenMeteoForecastResponse::class
])
class NativeConfig
Declaring the Proxy Interfaces of our HTTP clients
As explained, we have implemented our Open Meteo and Open AI HTTP clients using Spring HTTP Interfaces. The instances of those interfaces are generated automatically by Spring using Java dynamic proxies and invoked using reflection on runtime.
As GraalVM native images do not provide any way to generate and interpret bytecode at runtime, all dynamic proxy classes need to be generated at native image build time, by making use of code static analysis.
This can be achieved in different ways, on of them is creating a proxy-config.json
file defining the list of interfaces required to build the proxy classes.
For our application, we will create the proxy-config.json
file placed under the directory src/main/resources/META-INF/native-image
:
βββ src
β βββ main
β β βββ resources
β β βββ META-INF
β β β βββ native-image
β β β βββ proxy-config.json
β β βββ application.yml
The proxy-config.json
file should contain an array of objects, each of them with an interfaces
property as an array of Strings containing the interfaces needed to implement in significant order.
[
{
"interfaces": [
"tech.aaregall.lab.functions.weather.service.openmeteo.OpenMeteoHttpClient",
"org.springframework.aop.SpringProxy",
"org.springframework.aop.framework.Advised",
"org.springframework.core.DecoratingProxy"
]
},
{
"interfaces": [
"tech.aaregall.lab.functions.question.service.openai.OpenAiHttpClient",
"org.springframework.aop.SpringProxy",
"org.springframework.aop.framework.Advised",
"org.springframework.core.DecoratingProxy"
]
}
]
Check out GraalVM official docs on Dyanmic Proxy generation for more details.
Running our application natively
At this point, our application should be fully compatible with GraalVM and native executables.
We could run ./gradlew clean nativeRun
on our terminal to execute our application natively.
Alternatively, we could build the native executable and execute it in two different steps:
$ ./gradlew clean nativeCompile
$ ./build/native/nativeCompile/aws-lambda-spring-cloud-function
2023-07-01T13:36:36.135+02:00 INFO 74417 --- [ main] tech.aaregall.lab.functions.AppKt : Starting AOT-processed AppKt using Java 17.0.6 with PID ...
2023-07-01T13:36:36.165+02:00 INFO 74417 --- [ main] o.s.b.web.embedded.netty.NettyWebServer : Netty started on port 8080
2023-07-01T13:36:36.165+02:00 INFO 74417 --- [ main] tech.aaregall.lab.functions.AppKt : Started AppKt in 0.036 seconds (process running for 0.043)
Remember native image compilation requires a JDK GraalVM distribution.
java --version
openjdk 17.0.6 2023-01-17
OpenJDK Runtime Environment GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13, mixed mode, sharing)
βοΈ Deploying the application as an AWS Lambda with a native runtime
Now that our Spring Boot application can be run as a native executable and its start time takes less than half a second, it's a great candidate to deploy it on a serverless platform such as AWS Lambda.
Adding Spring Cloud Function AWS adapter dependency
Spring Cloud Function project already provides a library that helps developers deploy their functions on AWS Lambdas. It contains the necessary classes to implement a custom runtime event loop, resolve the Lambda destinations, or automatically convert AWS Event Messages to objects that our Functions can understand, so we do not need to modify our application code to consume APIGatewayV2HTTPEvent, SQSEvents, for example, and just work with plain application POJOs.
All we need to do is import the spring-cloud-function-adapter-aws
library in our dependencies:
implementation("org.springframework.cloud:spring-cloud-function-web")
implementation("org.springframework.cloud:spring-cloud-function-adapter-aws")
Defining our AWS infrastructure with a SAM
An AWS CloudFormation template is a formatted text file (JSON or YAML) that describes your AWS infrastructure.
AWS SAM (Serverless Application Model) has its particular template anatomy, which is very similar to the CloudFormation ones but with a few differences.
The serverless infrastructure of our application would be the following:
An HTTP API Gateway, with a Stage that we will use for Metrics and Monitoring.
A Permission that grants the HTTP API Gateway to invoke our Lambda application.
Our Lambda application itself (the native Spring Cloud Function we have just developed) with an Execution role.
- Will run in a
provided.al2
runtime onarm64
architecture.
- Will run in a
A LogGroup bounded to the Lambda application to define the log retention period (30 days).
We can then create a template.yml
file at the root of our project with the following content to define this infrastructure:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: tech.aaregall.lab:aws-lambda-spring-cloud-function
Parameters:
OpenAiApiKey:
Type: String
Description: OpenAI API key
Globals:
Api:
EndpointConfiguration: REGIONAL # API Gateway regional endpoints
Function:
Timeout: 20
MemorySize: 1512
Runtime: provided.al2
Architectures:
- arm64
Tracing: Active # https://docs.aws.amazon.com/lambda/latest/dg/lambda-x-ray.html
Resources:
# Lambda Function
SpringCloudFunctionLambda:
Type: AWS::Serverless::Function
Properties:
Handler: none # Native runtime does not need a handler
CodeUri: .
Policies: AWSLambdaBasicExecutionRole
Environment:
Variables:
OPENAI_API_KEY: !Ref OpenAiApiKey
Events:
HttpApiEvent:
Type: HttpApi
Properties:
TimeoutInMillis: 20000
PayloadFormatVersion: '1.0'
Metadata:
BuildMethod: makefile # Instruct SAM how to build application.
# Lambda LogGroup
SpringCloudFunctionLambdaLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Join [ '/', [ '/aws/lambda', !Ref SpringCloudFunctionLambda ] ]
RetentionInDays: 30
Outputs:
SpringCloudFunctionLambda:
Description: AWS Lambda Spring Cloud Function
Value: !Sub 'https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com'
Export:
Name: SpringCloudFunctionLambda
Notice we defined an OpenAiApiKey
parameter that we will need to provide when deploying our SAM.
The value of that parameter will be defined as the value for OPENAI_API_KEY
environment variable on our Lambda. Thus, the application.yml
placeholder that we defined before, will resolve it from the SAM parameter.
Also, notice we define the BuildMethod as a makefile
, which means we will instruct SAM on how to build the executable of our Lambda using a Makefile
.
Defining the AWS Labmda bootstrap
with a Makefile
Following the AWS custom runtimes guide, we need to create a file named bootstrap
on our lambda deployment package.
We will generate that by adding a Makefile
to the root of our project with the following content:
build-SpringCloudFunctionLambda:
./gradlew --no-daemon clean nativeCompile
echo '#!/bin/sh' > ./build/bootstrap
echo 'set -euo pipefail' >> ./build/bootstrap # Exit immediately if any command fails, propagate errors in pipelines.
echo './aws-lambda-spring-cloud-function' >> ./build/bootstrap
chmod +x ./build/bootstrap
cp ./build/bootstrap $(ARTIFACTS_DIR)
cp ./build/native/nativeCompile/aws-lambda-spring-cloud-function $(ARTIFACTS_DIR)
Notice we are using the ./gradlew --no-daemon clean nativeCompile
command to generate the native executable.
The placeholder $(ARTIFACTS_DIR)
refers to AWS SAM artifact dir (by default .aws-sam
).
The content of our bootstrap
file will then effectively be:
#!/bin/sh
set -euo pipefail
./aws-lambda-spring-cloud-function
Building a Docker image to run our SAM build
To make sure the generated native executable of our application is compatible with the AWS Lambda provided.al2
runtime (Linux), we will need to run our SAM build in a Docker container.
This is not necessary if you are using a concrete distribution of Linux in your local machine, but if you are using a Mac or Windows, the runtime of your local machine is different from the AWS Linux one so when trying to execute a SAM deployment package in AWS that is not built in a machine compatible with AWS Linux, it will just do nothing.
Therefore, to deploy a fully compatible package to AWS, is a good practice to run the build of your applications inside a Docker container compatible with provided.al2
(honestly this took me quite some time to figure it out empirically).
To accomplish this, we will create a new directory named aws-image
at the root of our project:
βββ aws-image
β βββ Dockerfile
β βββ build-aws-image.sh
The Dockerfile
will use the public AWS base image, and install SDKMan! and a JDK 17 with GraalVM 22.3.x on the container, as well as Python and some required pip dependencies from AWS needed to build the application.
# Image used to build the native application AWS artifact in a container with an OS compatible with Amazon Linux.
FROM public.ecr.aws/amazonlinux/amazonlinux:2
RUN yum -y update \
&& yum install -y zip unzip tar gzip bzip2-devel ed gcc gcc-c++ gcc-gfortran \
less libcurl-devel openssl openssl-devel readline-devel xz-devel \
zlib-devel glibc-static libcxx libcxx-devel llvm-toolset-7 zlib-static \
&& rm -rf /var/cache/yum
# Install SDKMan and Java GraalVM
RUN curl -s "https://get.sdkman.io" | bash
RUN source "$HOME/.sdkman/bin/sdkman-init.sh" && sdk install java 22.3.1.r17-grl
ENV JAVA_HOME "/root/.sdkman/candidates/java/current"
# Install AWS Lambda Builders
RUN amazon-linux-extras enable python3.8
RUN yum clean metadata && yum -y install python3.8
RUN curl -L get-pip.io | python3.8
RUN pip3 install aws-lambda-builders
The build-aws-image.sh
file will be a simple script to build the Docker image with a tag name from any directory of our local filesystem.
#!/bin/sh
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
set -e
docker build -t tech.aaregall.lab/amazonlinux-graalvm:latest "$SCRIPT_DIR"
Building and deploying our AWS application
The next steps are building the SAM deployment package and deploying it to our favorite AWS region, using sam build
and sam deploy
CLI commands.
Building the SAM deployment package
We will first build and tag the Docker image based on Amazon Linux with GraalVM installed, and verify the image is correctly built.
$ ./aws-image/build-aws-image.sh
$ docker image ls tech.aaregall.lab/amazonlinux-graalvm
REPOSITORY TAG IMAGE ID CREATED SIZE
tech.aaregall.lab/amazonlinux-graalvm latest 27e8a73380b0 7 seconds ago 1.54GB
Note this step does not need to be repeated every time we want to deploy our SAM package, as long as the image is present.
Then we can instruct SAM to build our deployment package in a Docker container using our custom image with the following command:
sam build --use-container --build-image tech.aaregall.lab/amazonlinux-graalvm:latest
This command will start a Docker container and will build our application project inside of it with the instructions provided in the Makefile
.
By default, it will use the Gradle Wrapper of our project, which will be downloaded in the container itself. Alternatively, you could customize the image and install Gradle with a RUN
command on the Docker file, but using the Gradle wrapper simplifies and centralizes the whole process for this article's purpose.
The whole build process takes around 3 minutes to complete on a Mac M1 with 32GB of RAM.
Once finished, the .aws-sam
directory should have been generated with the following structure:
.aws-sam
βββ build
β βββ SpringCloudFunctionLambda
β β βββ aws-lambda-spring-cloud-function
β β βββ bootstrap
β βββ template.yaml
βββ build.toml
Deploying the SAM package to AWS
At this point, we should be ready to deploy the SAM deployment package to our AWS region.
First, we will export a couple of environment variables:
export AWS_REGION=...
export OPENAI_API_KEY=...
And secondly, run the sam deploy
specifying the AWS region and the OpenAiApiKey parameter:
sam deploy --region $AWS_REGION --parameter-overrides ParameterKey=OpenAiApiKey,ParameterValue=$OPENAI_API_KEY
I recommend running it with --guided
argument the first time, as the CLI will prompt us to configure some settings for the deployment, for example, if the deployment has the capability to create IAM roles or to auto-resolve the S3 bucket where our deployment package will be stored.
We can define a samconfig.toml
file with the following default deploy parameters to avoid being prompted for the same options every time we run sam deploy
:
version = 0.1
[default.deploy.parameters]
stack_name = "aws-lambda-spring-cloud-function"
resolve_s3 = true
s3_prefix = "aws-lambda-spring-cloud-function"
confirm_changeset = true
capabilities = "CAPABILITY_IAM"
disable_rollback = false
With these defaults, the CloudFormation resources will be added to the stack changeset, and after that, we will need to confirm to deploy the changes.
Once confirmed, CloudFormation events will start to refresh, and finally produce the expected Output: the URL of the API Gateway to our AWS Lambda! π
If we navigate to our AWS Console > CloudFormation section, we should see the application stack created with all the resources described above:
Also, you can inspect how the resources were generated, for example, our Lambda named "SpringCloudFunctionLambda":
Finally, we can test our application by calling the API Gateway URL like a regular endpoint, although we will need to provide the function definition using a special Spring Cloud Function HTTP header (spring.cloud.function.definition
):
POST https://abcdefghij.execute-api.xxx.amazonaws.com
Accept: application/json
Content-Type: application/json
spring.cloud.function.definition: forecast,forecastToQuestion,question
{
"latitude": 41.3874,
"longitude": 2.1686
}
HTTP/1.1 200 OK
Date: Sat, 01 Jul 2023 16:43:46 GMT
Content-Type: application/json
Connection: keep-alive
Apigw-Requestid: Ha82tghuliAEPvQ=
{
"answer": "The weather forecast for Barcelona (Spain) from July 1st to July 7th shows temperatures ranging from 20.7 ΒΊC to 29.5 ΒΊC, with no precipitation expected throughout the period."
}
If we prefer to only expose one concrete function, or a concrete function composition, and don't let the API Gateway consumer decide the function definition itself, we can always define the spring.cloud.function.definition
configuration property on our application.yml
: in that case, the request will not require the function definition header.
Limitations and troubleshooting
Worth to be mentioned, as of the time of this writing, the Spring Cloud Function AWS adapter had some limitations.
First, you need to know that if your approach is letting the client/event producer define the function to be executed (using spring.cloud.function.definition
header as we have seen), the AWS adapter library will set that value as a Spring Environment
property, meaning that while the lambda is running, that will be the function definition for every request/event that interacts with the lambda until the process shuts down and a new invocation comes.
This means two consecutive invocations to the same Lambda using a different function definition header, will effectively invoke the function defined by the very first invocation while the Lambda process is running. Please check the AWS Lambda runtime environment official docs and this GitHub issue if you are interested in knowing more about this bug. (Edit September 26th, 2023: the reported issue was fixed on Spring Cloud Function 4.0.5 β - the issue was only reproducible when using Kotlin).
Another limitation I found during my explorations is related to reactive types body mapping, meaning the AWS adapter is not fully able to map an array of GeoLocations
to a Flux<GeoLocation>
object.
Anyhow, with individual objects on the body as showcased, it works perfectly, which is fine for this article. Please check this GitHub issue if you are interested in this detail. (Edit July 13th, 2023: the reported issue was fixed on Spring Cloud Function 4.0.5 β )
If we want to troubleshoot or inspect the logs of our application running on AWS, we can easily do that from our terminal using the following command:
sam logs --stack-name aws-lambda-spring-cloud-function
I recommend enabling the DEBUG
level on Spring Cloud Functions AWS Adapter package to fully understand the lifecycle of invocations to our applications. It provides helpful insights about how everything works under the hood, and especially what actions the library performs for us, like function lookup, message conversion, etc.
logging:
level:
org.springframework.cloud.function.adapter.aws: DEBUG
Conclusions
This article demonstrated:
How to set up a Spring Cloud Function application project from scratch.
How to keep our focus on implementing functions with business logic taking benefit from Spring Framework abstractions and Spring Boot features, like dependency injection, automatic POJO JSON serialization, or auto-configuration.
How to integrate our Spring applications with different APIs using HTTP Interfaces.
How to write integration tests for Spring Cloud Function applications.
How to generate a native executable using GraalVM that we can deploy to a serverless platform.
How to set up an AWS SAM build process and how to deploy that package to AWS as an AWS Lambda using a custom native runtime - a process that is not straightforward and requires some attention from us as developers.
Know which are the limitations of Spring Cloud Function application when deployed as AWS Lambdas.
The source code of the application developed is available in the linked GitHub repository.
If you enjoyed the article, let me know by dropping a star βοΈ on the GitHub repository!
And of course, you can also give me feedback by writing a comment or reaching me directly through my linked social networks. βΊοΈ