Retrieval-Augmented Generation based on Knowledge Graph (KG-RAG) for Providing Accurate Information to Dialogue Systems
- Speaker
- Tsuneo Kato (Doshisha University, Japan)
- Date
- Room
- Salle 306, bâtiment IMAG
Advancements in large language models (LLMs) make it easier to develop interactive information systems. However, LLMs have a problem of generating hallucinations, especially for closed information of organizations and frequently renewed information. We prototyped Doshisha University’s interactive information system (text-chat system) in which we focused on the correctness of the information supplied by retrieval-augmented generation using knowledge graph (KG-RAG) and bilingualism. We built a knowledge graph (KG) on the Neo4j database and developed a chat system that used the KG and OpenAI API for GPTs. The KG enables three types of search, i.e., KG search, full-text search, and vector search, by registering text information and its embedding vector into the nodes. The chat system calls the API for GPTs four times: 1) preprocessing, i.e., translating questions in English to those in Japanese and converting colloquial Japanese expressions to formal Japanese texts; 2) conversion from natural language to Cypher queries for KG search; 3) response generation based on KG-RAG; and 4) additional suggestion of next question for continuing the dialogue. We further developed a smart responding mechanism to inaccurate user inputs that contain wrong proper nouns, etc. We quantitatively evaluated the system from two points of view: 1) general accuracy of information provided by the responses in two languages and 2) accuracies of responses to inaccurate inputs. The addition of the first LLM for preprocessing and the smart responding mechanism significantly improved the general accuracy and the accuracy of responses to inaccurate inputs, respectively.
