Nowadays Large language models (LLMs) with large context windows are capable of processing lengthy dialogue histories during prolonged interaction with users without additional memory modules; however, their responses tend to overlook or incorrectly recall information from the past. Such response failures are mainly due to (a) bias to the latest input (b) the absence of an important past event on the timeline.
Motivated by these, researchers have introduced THEANINE, a framework of timeline-augmented chain-of-thought reasoning for response generation in long-term conversations. THEANINE, augments LLMs’ response generation with memory timelines – series of memories that demonstrate the development and causality of relevant past events. THEANINE manages memories with a graph structure, and follows below phases.
In Phase I, THEANINE connects each memory in a graph based on their relationships, forming a network where similar memories are linked together. It leverages an LLM to dynamically link memories based on their temporal and cause-effect common sense relations.
In Phase II-1, during memory retrieval for generating responses, THEANINE retrieves the entire memory timeline that represents the sequence of relevant events. In Phase II-2, to bridge the gap between offline memory construction and online deployment, THEANINE refines this retrieved timeline to provide tailored information that is most relevant to the ongoing conversation.
Finally, in Phase III, for response generation, THEANINE leverages LLMs’ chain-of-thought (CoT) reasoning ability to reason over current conversation and refined timelines, conclude useful information, and generate the final response.
THEANINE was evaluated against Multi-Session Chat (MSC) and Conversation Chronicles (CC) long-term conversations datasets and under various Machine and Human Evaluation scenarios. In all such scenarios THEANINE responses that are more detail-rich, utilizes past memories efficiently and correctly, responses where human thinking makes sense and properly references past conversations.
Paper : https://arxiv.org/pdf/2406.10996