A Hybrid Question Answering Model with Ontological Integration for Environmental Information

TY - JOUR

T1 - A Hybrid Question Answering Model with Ontological Integration for Environmental Information

AU - Sun, Tianda

AU - Carr, Jamie

AU - Kazakov, Dimitar

N1 - Publisher Copyright: © 2024 Copyright for this paper by its authors.

PY - 2024/10/19

Y1 - 2024/10/19

N2 - This paper presents a novel approach to constructing a Question Answering model for analysing Nationally Determined Contributions (NDC) reports within the environmental sector. The approach is based on Large Language Models (LLMs) equipped with Retrieval Augmented Generation (RAG) and enhanced by ontology integration. Acknowledging the challenges inherent in directly applying RAG, our approach begins with the development of a specialised ontology framework for NDC reports. This framework supports the construction of a knowledge graph that provides essential, verifiable information for a Question Answering (QA) model. In the next step, the model combines RAG embeddings with ontology-based queries, aiming to enhance the reliability of answers across various NDC reports. We evaluate the performance of our hybrid model through testing with a set of questions and human/AI evaluation across different LLMs. While the results indicate improvements in the efficiency of climate change-related QA models, they also underscore the complexity of achieving significant enhancements in this domain. Our findings contribute to ongoing discussions about the potential and limitations of integrating ontological methods with LLM for environmental information retrieval.

AB - This paper presents a novel approach to constructing a Question Answering model for analysing Nationally Determined Contributions (NDC) reports within the environmental sector. The approach is based on Large Language Models (LLMs) equipped with Retrieval Augmented Generation (RAG) and enhanced by ontology integration. Acknowledging the challenges inherent in directly applying RAG, our approach begins with the development of a specialised ontology framework for NDC reports. This framework supports the construction of a knowledge graph that provides essential, verifiable information for a Question Answering (QA) model. In the next step, the model combines RAG embeddings with ontology-based queries, aiming to enhance the reliability of answers across various NDC reports. We evaluate the performance of our hybrid model through testing with a set of questions and human/AI evaluation across different LLMs. While the results indicate improvements in the efficiency of climate change-related QA models, they also underscore the complexity of achieving significant enhancements in this domain. Our findings contribute to ongoing discussions about the potential and limitations of integrating ontological methods with LLM for environmental information retrieval.

KW - Knowledge Graph Construction

KW - Relation Extraction

KW - Retrieval Augmentation Generation

UR - http://www.scopus.com/inward/record.url?scp=85211099694&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85211099694

SN - 1613-0073

VL - 3833

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

T2 - 4th International Workshop on Data meets Ontologies in Explainable AI, DAO-XAI 2024

Y2 - 19 October 2024

ER -