Creating In-Session Memory Leveraging LLM for Rich Conversations in Ecommerce

An AI humanoid conversing with a human — Image from Dall-E

In the age of digital shopping, ecommerce businesses face the challenge of replicating the rich, interactive shopping experience typically found in physical stores. A critical element in overcoming this challenge lies in fostering intelligent, meaningful, and personalized conversations with customers throughout their shopping journey. With the evolution of AI, specifically language models like GPT from OpenAI and orchestration systems like LangChain, we now have the means to implement sophisticated conversational AI that can significantly enhance user experiences in ecommerce.

In this article, I will delve into how we can leverage what I’m calling Language Model Memory (LMM) and in-session memory to create rich and personalized conversations in ecommerce settings. I will also explain how you can use graph databases, such as Neo4j, in tandem with LLMS, such as GPT to empower your conversational agents.

A Primer on In-Session Memory and LLM

In-session memory

In the realm of ecommerce, creating personalized experiences for customers is crucial. A key aspect of personalization is remembering the context of a user’s ongoing session, or ‘in-session memory’. This mechanism stores information about a user during an active interaction session. It can include diverse details, such as a user’s browsing pattern, shopping preferences, or specific product inquiries. The importance of in-session memory is underlined in a research paper titled Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks where session-based recommendations for users are made possible using in-session memory.

Maintaining this memory allows the system to provide context-aware responses and recommendations. For instance, if a customer is searching for “black formal shoes” on an ecommerce website, the system stores this information in the in-session memory. If the customer then asks, “Do they come in leather?”, the system understands that ‘they’ refers to the ‘black formal shoes’ and provides an appropriate response.

In-session memory is especially important in cases where the customer doesn’t explicitly mention the context in every interaction. It significantly improves the user experience by facilitating personalized, relevant, and timely responses. You can read more about the importance of context in conversational AI in the paper Deep Reinforcement Learning for Dialogue Generation.

Language Language Models (LLM)

Large Language Models (LLMs) like GPT-4 from OpenAI and BARD from Google have revolutionized the field of Natural Language Processing (NLP). These models generate human-like text by predicting the likelihood of a word based on the preceding words in the text. In addition to these models are orchestration systems, such as LangChain, that can “chain” together different components to create more advanced use cases around LLMs.

GPT-4, or Generative Pretrained Transformer 4, is an AI model that uses machine learning to produce human-like text. It’s the latest iteration of the GPT series, which has been successful in a variety of natural language tasks, such as translation and summarization. You can find more about the GPT models in the original research paper.

On the other hand, LangChain is a distributed decentralized orchestration layer. It provides benefits of both on-chain (secure and immutable) and off-chain (scalable and efficient) computations. It is a widely leveraged framework specifically designed to streamline the creation of applications utilizing large language models (LLMs). Its use cases align broadly with those of language models in general, spanning from document analysis and summarization, chatbots, to code analysis.

Integration of LLMs and In-Session Memory

Integrating LLMs with in-session memory, to create Language Model Memory (LMM), is a powerful combination that enables AI models to maintain conversational context, leading to more intelligent and engaging interactions. When an AI model has access to the session’s context, it can generate responses that align with the user’s current activity, resulting in a natural, personalized conversation.

For instance, an AI model can remember the details of a product that a user is interested in and generate responses or recommendations based on that information. If the user adds a new product to their cart, the model can update the session context and adapt its responses accordingly.

This kind of advanced, context-aware conversational AI can transform ecommerce interactions, offering a personalized shopping experience that not only addresses customer queries but also anticipates their needs and preferences. For more details on how conversational AI is transforming ecommerce, check out this article.

By fusing the capabilities of LLMs and in-session memory, we can create conversational agents that offer seamless, intelligent, and engaging interactions, enhancing the customer experience across in-session and and out-of-session (i.e. future chats) experiences.

The Role of Graph Databases

Image Credit: Neo4J

Graph databases play a pivotal role in maintaining and leveraging in-session memory. These databases are excellent at storing and navigating complex relationships between entities, like the various factors involved in a shopping experience. In our context, they can efficiently store and retrieve data about a user’s ongoing session, including user preferences, browsing history, current items in the cart, and much more.

Neo4j, a prominent graph database, serves as a key asset in enriching ecommerce interactions and facilitating the effective integration of LLMs. Its strength lies in its capability to model and process intricate relational data efficiently, an essential attribute for maintaining in-session memory and delivering personalized user engagement.

Powering Ecommerce Interactions

In an ecommerce setting, Neo4j is instrumental in storing essential session data such as the user’s browsing history, their cart items, or their previous purchases. This information forms the in-session memory, enabling the conversational AI to recall the user’s current context and produce appropriate responses.

For example, if a user browses ‘Smartphones’ and places ‘Wireless Headphones’ in their cart, this can be depicted in Neo4j using nodes (User, Product, and Item) and relationships (:BROWSED, :ADDED_TO_CART).

Efficient Data Retrieval

The power of Neo4j’s graph model and its query language, Cypher, lies in their ability to effectively extract complex relational data. With a simple Cypher query, we can effortlessly retrieve a user’s session data:

session_data = graph.run("""
MATCH (u:User {name: 'John'})
OPTIONAL MATCH (u)-[:BROWSED]->(browsed_product:Product)
OPTIONAL MATCH (u)-[:ADDED_TO_CART]->(item_in_cart:Item)
RETURN browsed_product, item_in_cart
""").data()

The resulting session_data provides details about the products the user has viewed and the items they've placed in their cart. This can be used to establish the context for our language models.

Real-time Data Updates

As the user’s session continues, the graph database can be updated to mirror their recent activities. If a user adds another product to their cart, a new :ADDED_TO_CART relationship can be created between the User and the Item nodes.

Enriching LLM Integration

When integrating with LLMs like GPT and systems like LangChain, Neo4j’s efficient storage and retrieval of session data is crucial. The context derived from session data helps the models generate personalized and context-aware responses. Furthermore, the flexible data model offered by Neo4j can enable more advanced personalization strategies, including dynamic product recommendations and tailored marketing content.

Neo4j’s prowess in handling complex relational data, combined with sophisticated LLMs and in-session memory, opens up new vistas for enhancing ecommerce interactions. This powerful combination promises to revolutionize online shopping by bringing a more engaging, personal touch to digital conversations.

Creating LMM — Integrating OpenAI and LangChain with In-Session Memory

To build a smart conversational agent, we now need to integrate our LLM with the in-session memory stored in our graph database. Here’s an example of how it can be done using Python, OpenAI’s GPT, and LangChain:

First, we load our models:

from openai import GPT
from langchain import LangchainModel

gpt_model = GPT.load_model('path_to_your_gpt_model')
langchain_model = LangchainModel.load_model('path_to_your_langchain_model')

Then, we retrieve the relevant session data from our graph database:

from py2neo import Graph

# Initialize the graph database
graph = Graph("bolt://localhost:7687", auth=("neo4j", "password"))

# Query the database to retrieve current session data
current_session_data = graph.run("""
MATCH (u:User {name: 'John'})
OPTIONAL MATCH (u)-[:BROWSED]->(browsed_product:Product)
OPTIONAL MATCH (u)-[:ADDED_TO_CART]->(item_in_cart:Item)
RETURN browsed_product, item_in_cart
""").data()

Now, we can use this data to generate a context for our models:

# Define the context for the language models
contextual_info = f"""
John has browsed {current_session_data['browsed_product']['name']} and added {current_session_data['item_in_cart']['name']} to his cart.
"""

# Set the context for GPT and Langchain models
gpt_model.context = contextual_info
langchain_model.context = contextual_info

By setting the context based on the current user session, the language models can generate personalized and relevant responses, resulting in a more engaging and natural conversation. *Note, you can adjust to focus on all chat history if your goal is to build a recurring conversation*

At this point, our models are ready to generate responses based on the user’s session. Here’s how we can generate a response using the GPT model:

response = gpt_model.generate('What should I buy next?')

And similarly, with the Langchain model:

response = langchain_model.generate('What should I buy next?')

The models will take into account the context we provided and generate responses accordingly. For instance, if the user asks for product recommendations, the models can suggest products related to the ‘Smartphone’ John was browsing or the ‘Wireless Headphones’ he added to his cart.

Let’s consider a scenario where our models generate a response to the question, “What should I buy next?” taking into account the context of “John” browsing a ‘Smartphone’ and having ‘Wireless Headphones’ in his cart:

Input:

# For GPT model
response_gpt = gpt_model.generate('What should I buy next?')
print(response_gpt)

Output:

"Based on your interest in the Smartphone you were browsing, I'd recommend looking at this 'Portable Charger 10000mAh Power Bank'. It's highly rated and compatible with most smartphones. Also, since you have 'Wireless Headphones' in your cart, you might want to consider adding a 'Wireless Charging Pad' as well."

Similarly, for the Langchain model:

Input:

# For Langchain model
response_langchain = langchain_model.generate('What should I buy next?')
print(response_langchain)

Output:

"Considering your recent browsing of a Smartphone, you might find the 'Screen Protector for Smartphone' or 'Smartphone Car Mount Holder' to be useful accessories. Additionally, seeing that you've added 'Wireless Headphones' to your cart, a 'Headphones Case' could be a great addition to protect your new headphones."

These models generate responses based on the context provided, offering suggestions that align with the user’s recent activities and current interests. The language model has made these recommendations by considering “John’s” browsing and shopping behavior during the session, allowing for personalized and context-aware suggestions.

Enriching Ecommerce Interactions with LLMs and In-Session Memory

Utilizing Large Language Models (LLMs) and in-session memory enables us to design smart conversational agents capable of managing an array of user interactions. Here are some illustrative applications:

1. Tailored Product Suggestions: As we’ve seen, the conversational agent can recommend products that align with the user’s browsing history and items already in their shopping cart.
2. Real-time Order Updates: Should the user have an active order, the agent can access the current status from the graph database and provide timely updates.
3. Resolving Product Queries: The agent can address queries related to products by retrieving product details from the database, helping customers make informed decisions.
4. Customized Marketing: Drawing on insights from user preferences and shopping habits, the agent can deliver personalized marketing messages, enhancing user engagement.
5. Shopping Guidance: The agent can assist new or uncertain customers through their shopping journey, offering product suggestions and aiding in decision-making.

In summary, combining in-session memory with state-of-the-art language models coupled with the effective management of relational data via graph databases, paves the way for an immersive, personalized, and dynamic ecommerce conversation experience. This sophisticated approach is set to transform online shopping, bringing it closer to the engaging, personalized experience offered by physical stores, thereby enhancing customer satisfaction and fostering loyalty.

Matt Fisher is on LinkedIn