From the course: Foundations of AI and Machine Learning for Java Developers

Accessing LLMs via stateless REST APIs - Java Tutorial

From the course: Foundations of AI and Machine Learning for Java Developers

Start my 1-month free trial

Accessing LLMs via stateless REST APIs

“

- [Instructor] Practically, all of the language models provide REST APIs to their services. This allows developers to access the language model or any of the other services via any programming language. So we could write programs that use gen AI services via Java, JavaScript, Python, C or C++, C#, or even a shell script. REST APIs certainly are an effective technique in a polyglot environment, which is typically true in large companies. This allows you to build gen AI-powered applications for mobile devices, laptops, notebooks, tablets, server-side environments, or even embedded devices like watches or automobiles. Chatbots clearly use these APIs when they hold human-like conversations with users. So these REST APIs are very important. Now, one thing about these language model REST APIs is that they are stateless. This means the network or the server does not maintain any persistent session information, that is state, about the client. Every request is independent and treated as a new connection. Most of us know that since there is no need to save state, the server architecture can provide higher degrees of scalability. And with huge numbers of users, the big gen AI providers use these architectures. So all of these REST APIs are stateless. There's no data saved. If the connection is broken, there is no persistent information of the prior connection. So that applies to all of these types of target applications, mobile devices, laptops, tables, embedded devices, and even chatbots. These all use these stateless APIs. But wait a minute. You may ask, what about a chatbot? When I use a chatbot, it remembers what I'm saying. Now, that is an excellent question. Chatbots clearly do remember the thread of a conversation. It remembers what you said as context of the conversation. This allows the chatbot to have a more human-like conversation with the user. But how is this possible if we are using stateless APIs? Chatbots certainly seem to be saving state. Well, it turns out that even chatbots use a stateless network connection to talk to a language model. When you build a chatbot, you want the chatbot to remember the prior conversation so it can respond within the context of that chat. Since the connection is stateless, well, somebody has to remember state. The answer is that remembering state is the responsibility of the developer. Java developers have to maintain a memory of the conversation. In fact, we have to remember the conversation and repeatedly send that information to the language model. This is our context. This context guides the language model to give us better completions. This is very important when we build chatbots. Now let's look a little deeper into the APIs.

Contents