1 Introduction↩︎

The use of AI in education has the potential to transform the way science and technology courses are taught. In scientific learning, students are expected to engage in problem-solving and exploration, yet traditional classroom methods often focus on the passive acquisition of established knowledge. This approach limits opportunities for students to experience the process of knowledge creation, leading to lower cognitive engagement. Cybersecurity is a problem-based learning domain where students must master complex tools, develop defense techniques, and uncover new threats, which necessitates a re-imagination of traditional education practices  [1].

Prior research highlights that managing uncertainty is a crucial component of the learning process, as students often struggle with acquiring new skills, applying diverse methodologies, and forming new understandings [2]. Educators can effectively manage this uncertainty by increasing it through the introduction of authentic, ambiguous challenges to stimulate critical thinking, maintaining it to encourage deeper exploration and problem-solving, and reducing it by identifying optimal solutions to help students integrate new insights with existing knowledge [3]. AI-driven question-answering (QA) systems can help manage this uncertainty in technical problem-solving by supporting self-paced learning, significantly enhancing cognitive engagement  [4].

In recent years, large language models (LLMs) have become central to AI-driven technologies. While LLM-powered QA systems hold great promise for enhancing learning, they also face challenges such as hallucination and limited domain knowledge, which can undermine their effectiveness [5]–[8]. In cybersecurity education, where precision is critical, ensuring the accuracy of AI-generated content is essential. For example, in tasks such as identifying vulnerabilities or interpreting security policies, inaccurate AI responses could lead to misinformation, compromising the learning experience and, potentially, real-world security [9]. A promising solution to address this challenge is the retrieval-augmented generation (RAG) approach [10], where the model generates responses by retrieving information from a validated knowledge base, thereby enhancing the accuracy and reliability.

Although the RAG approach helps reduce hallucinations and address domain knowledge issues to some extent, the reliability of LLM-generated answers remains a concern for achieving educational goals. Students may ask questions that fall outside the scope of the augmented cybersecurity knowledge base. In such cases, LLMs rely on their own parametric knowledge to generate responses, which can expose the QA system to risks of misinformation or misuse [9], [11], [12]. In an educational setting, it is also crucial to prevent students from manipulating the AI system for unintended purposes. There is a strong need to provide a validation system to ensure the accuracy and safety of LLM-generated responses. One potential solution is reinforcement learning from human feedback (RLHF) [13]. However, this method requires verification by cybersecurity experts, making it labor-intensive, costly, and time-consuming. Preferably, an automatic validation approach is needed. Domain-specific knowledge graphs, which structure expert knowledge and capture the interactions between key entities in alignment with domain rules [14], offer a promising direction. The knowledge graph ontology encodes these rules [15]. By leveraging this ontology, LLM responses can be validated by fact-checking against the predefined rules, ensuring greater accuracy and reliability without the need for constant human oversight.

In this paper, we propose CyberRAG, an ontology-aware RAG approach for developing a reliable QA system in cybersecurity education, comprising two key components. First, we utilize RAG methods to retrieve validated cybersecurity documents from a knowledge base, enhancing both the accuracy and relevance of the answers. Through finely crafted prompts, CyberRAG improves QA performance by leveraging both cybersecurity content and the natural language processing capabilities of LLMs. Additionally, we introduce an ontology-based validation approach that uses a cybersecurity knowledge graph ontology to automatically verify LLM-generated responses, thereby preventing potential risks of misuse and hallucination. We conduct comprehensive experiments on publicly available datasets to demonstrate the effectiveness of CyberRAG in delivering reliable and accurate answers. Our research explores the potential of integrating AI into education, emphasizing its transformative impact on traditional methods. This approach extends beyond cybersecurity and can also be applied to other educational subjects. Our contributions can be summarized as follows:

• We propose CyberRAG, a novel and reliable computational framework aimed at creating an interactive and secure environment for cybersecurity education.

• We design finely crafted prompts that enable CyberRAG to produce accurate and relevant answers by integrating a cybersecurity knowledge base with the vast prior knowledge present in LLMs.

• We explore the use of knowledge graph ontology to validate LLM-generated responses, ensuring accuracy and preventing potential misuse of QA systems in education.

2 Related Work↩︎

2.1 RAG in Education↩︎

Generative models have the potential to transform traditional education by enabling personalized learning and automating content creation, making education more adaptive and accessible [16]. The integration of LLMs in education has further enhanced conversational capabilities, supporting self-paced learning through AI-driven question-answering tools and bots [17], [18]. However, the use of LLMs also presents challenges, particularly the risks of hallucinations and inaccurate responses, which are critical concerns in educational contexts and must be carefully managed to ensure responsible implementation [19], [20]. To address these issues, retrieval-augmented generation (RAG) methods are used by combining LLMs with real-time knowledge base retrieval to improve response accuracy ensuring content is current and reliable in educational settings [21]–[23].

2.2 LLM-generated Answer Validation↩︎

However, most LLM approaches are currently limited to handling simple queries, as generating complex answers from extensive knowledge articles and course materials sourced from diverse knowledge bases remains challenging, often leading to issues with the correctness of the answers [24]. Jeong et al.  [25] proposed an adaptive QA approach to handle complex questions, while Li et al.  [26] developed an evaluation framework for grounded and modular assessment of RAG responses.

Various domains are using knowledge graphs to enhance LLMs’ reasoning and retrieval capabilities [27]–[29]. An ontology within a KG, which defines the rules of a specific domain, can also be used to validate LLM responses by checking the correctness of relationships between key entities [5]. This capability is particularly valuable in educational settings, where ensuring the accuracy of LLM-generated content is crucial. While some student questions may extend beyond the scope of the augmented course material or knowledge base, ontology-aware validation offers a promising solution. However, research in this area remains limited. Our ontology-aware methods mark a significant advancement in this field.

2.3 AI for Cybersecurity Education↩︎

Learning cybersecurity is crucial for national security, safeguarding critical infrastructure, and ensuring defense [30]–[32]. However, mastering this complex field requires deep knowledge of concepts, tools, and attack-defense simulations [33]–[35]. While AI shows promising potential in enhancing cognitive learning, research on its application in cybersecurity education remains limited [36]–[39]. Previous efforts include structured flow graphs from CTF texts for vulnerability analysis [40] and knowledge graphs for guiding student projects [41]–[43]. Additionally, a semi-automated approach has been used to create knowledge graphs from unstructured course material, improving student learning [44]. The AISecKG ontology[45] further advanced this by generating cybersecurity KG and the CyberQ dataset [46] for LLM-based question-answering.

However, a significant gap remains: the lack of a self-paced learning cybersecurity QA system. To address this, we leverage the CyberQ dataset to retrieve structured, authenticated cybersecurity content using RAG and validate LLM outputs with the AISecKG ontology, particularly for queries beyond the scope of the CyberQ dataset. Our approach enhances the accuracy, reliability, and safety of AI-driven education tools, effectively addressing this critical research gap.

3 Preliminaries↩︎

In this section, we will outline the fundamental concepts underlying our proposed CyberRAG framework.

3.1 Problem Formulation↩︎

We consider cybersecurity problem-based learning to be a question-answering (QA) problem. Specifically, students raise a series of questions \(Q = \{q_1, q_2,..., q_{|Q|}\}\), we expect the model to output corresponding responses as answers \(A = \{a_1, a_2,..., a_{|A|}\}\), where \(|Q| = |A|\) in our scenario.

3.2 RAG System↩︎

We employ a retrieval-augmented generation (RAG) system as a base framework to solve our problem because of its impressive capacity for hallucination mitigation. A classic RAG approach entails two parts:

• A retriever \(\mathcal{R}\) that can effectively retrieve reference documents \(D = \{d_1, d_2,..., d_{|D|}\}\) from the knowledge base \(K\) (e.g., knowledge graph, QA database, and text materials) based on query-document relevance.

• A generation model (e.g., large language model) \(\mathcal{G}\) that provides answers using the documents and human instructions \(I = \{i_1, i_2,..., i_{|I|}\}\).

3.3 Knowledge Graph Ontology↩︎

In the context of our CyberRAG framework, we incorporate knowledge graph ontology to validate the LLM-generated responses. A knowledge graph (KG) is a structured representation of knowledge in which entities (a.k.a, nodes) are connected by relationships (a.k.a, edges). Formally, we define a knowledge graph as a tuple \(G = (E, R)\), where \(E = \{e_1, e_2, ..., e_{|E|}\}\) represents the set of entities, and \(R = \{r_1, r_2, ..., r_{|R|}\}\) represents the set of relationships between these entities. Each relationship \(r \in R\) can be viewed as a directed edge connecting two entities \(e_i\) and \(e_j\) in the graph, thereby forming a triplet \((e_i, r, e_j)\).

To further formalize the structure and semantics of the domain knowledge represented in our system, we utilize an ontology \(O\). An ontology is a formal specification of a set of concepts within a domain and the relationships between those concepts. Formally, we define an ontology \(O\) as a tuple \(O = (C, R_C, H_C)\), where \(C = \{c_1, c_2, ..., c_{|C|}\}\) is a set of concepts, \(R_C = \{r_1^C, r_2^C, ..., r_{|R_C|}^C\}\) is a set of conceptual relationships, and \(H_C \subseteq C \times C\) is a hierarchical structure (e.g., a taxonomy) that organizes these concepts.

4 Method↩︎

In this section, we first introduce our approach to CyberRAG. The detailed design of each component is then discussed in the subsequent sections.

4.1 CyberRAG↩︎

Our proposed framework, named CyberRAG, comprises two key components: a retrieval-augmented generation (RAG) system and an ontology-based answer validation module. By leveraging the RAG system with carefully crafted prompts, CyberRAG effectively addresses students’ questions by utilizing cybersecurity knowledge materials and the natural language understanding capabilities of LLMs. The ontology-based validation approach ensures that the responses provided to students are both reliable and secure. An overview of CyberRAG is presented in Fig. 1.

4.2 Cybersecurity Knowledge Retrieval↩︎

To retrieve cybersecurity material from the knowledge base, the first step is to design an effective retriever. We utilize a dual-encoder dense retriever containing two encoders: a question encoder \(\mathcal{E}_Q\), which projects student queries into a shared semantic latent space, and a document encoder \(\mathcal{E}_D\), which embeds knowledge base documents into corresponding vector representations. These encoders are optimized to maintain semantic proximity between queries and relevant documents, ensuring that encoded representations are well-aligned in the latent space, thereby facilitating accurate retrieval. Note that the question encoder and the document encoder can be the same in some settings. Then, the retriever can retrieve the top \(k\) documents according to the query-document similarity.

Specifically, given a query \(q_i \in Q\) and a document \(d_j \in D\), they are encoded by query encoder \(\mathcal{E}_Q\) and document encoder \(\mathcal{E}_D\), respectively. \[h_{q_i} = \mathcal{E}_Q(q_i)\] \[h_{d_j} = \mathcal{E}_D(d_j)\] where \(h_{q_i}\) and \(h_{d_j}\) are latent representations of query \(q_i\) and document \(d_j\).

Then, the semantic relevance between the question and the document is defined by cosine similarity between their latent vectors: \[sim(h_{q_i}, h_{d_j}) = \frac{h_{q_i} \cdot h_{d_j}}{\|h_{q_i}\| \|h_{d_j}\|}\] where \(h_{q_i} \cdot h_{d_j}\) are inner product of two vectors and \(\|h_{q_i}\|\) and \(\|h_{d_j}\|\) are the magnitudes (a.k.a. Euclidean norms) of \(q_i\) and \(d_i\), respectively.

Based on the similarities. top-\(k\) relevant course documents can be retrieved. \[\label{eq:retriever} D = \mathcal{R}(Q, K)\tag{1}\]

4.3 Cybersecurity Answer Generation↩︎

After retrieving relevant cybersecurity documents, the next step is to prompt the generative model (i.e., LLMs) to generate answers. The most challenging aspect is designing a prompt that maximizes the effectiveness of both the retrieved information and the generative model. An ideal prompt should guide the model to summarize the information when the retrieved documents are highly relevant and encourage the model to generate new responses by leveraging its own knowledge when the retrieved information is insufficient or incomplete.

We design the prompt as illustrated in the Answer Generation Prompt. The proposed prompt consists of three key elements:

• Documents: The documents relevant to the student’s query are retrieved by the retriever from the knowledge base, using the method outlined in the previous section.

• Questions: These are queries raised by students, which the generative model is tasked with answering.

• Instructions: These are rules and prompts designed by domain experts. The first two instructions guide the generative model to answer students’ questions by referencing the retrieved documents. The third instruction encourages the model to generate responses based on its own knowledge when the documents are insufficient. Thus the proposed framework can address students’ queries, whether or not they are covered by the knowledge base.

The final prompt is given as input to the generative model to generate the answer to the student’s query: \[\label{eq:answer} A = \mathcal{G}(D, Q, I)\tag{2}\]

4.4 Ontology-based Answer Validation↩︎

For ontology validation, we used AISecKG [47], a cybersecurity education ontology that defines relationships between concepts, applications, and roles within the cybersecurity domain. AISecKG organizes these into three broad categories with 12 entity types. Concepts include features, functions, data, attacks, vulnerabilities, and techniques, while applications cover tools, systems, and apps. Roles consist of users, attackers, and security teams. The ontology defines nine core relationships between these entities, represented by 68 unique edges. For example, tuples such as (‘attacker’, ‘can_exploit’, ‘feature’) and (‘security team’, ‘can_analyze’, ‘feature’) illustrate entities and these relationships. These triples represent the fundamental domain rules that govern cybersecurity information at a schema level. By leveraging these ontology-based triples along with the natural language understanding capabilities of LLMs, an automatic answer validation system can be developed for the cybersecurity domain.

Specifically, given the question \(Q\) and the corresponding response \(A\) provided by the generative language model, the validation model \(\mathcal{V}\) takes the QA contexts, ontology rules denoted by \(O = \{o_1, o_2,..., o_{|O|}\}\), and validation human instruct \(I'\) as the input, then produces the validation result \(R\).

\[\label{eq:validation} R = \mathcal{V}(Q, A, O, I')\tag{3}\] Note that \(R \in [0, 1]\), and a higher score indicates the response aligns with the ontology well. The complete validation prompt is shown in the Ontology Validation Prompt.

By setting an appropriate threshold \(\sigma\), the pipeline can effectively filter out incorrect answers and identify potential misuse behavior. The CyberRAG algorithm is outlined in Algorithm 2.

5 Experiment and Discussion↩︎

5.1 Dataset↩︎

In this work, we use CyberQ  [46], an open-source cybersecurity dataset containing around 4,000 open-ended questions and answers on topics such as cybersecurity concepts, tool usage, setup instructions, attack analysis, and defense techniques. The questions vary in complexity, answer length, and vocabulary. The dataset was developed using facts from AISecKG [47], a cybersecurity knowledge graph, and a three-step LLM prompting method. The Zero-shot (ZS) method generated 1,027 QA pairs for simple WH-questions on cybersecurity entities. The Few-shot (FS) method produced 332 medium-complexity QA pairs related to setup and tools. The Ontology-Driven approach generated 2,171 high-complexity QA pairs covering attack and defense scenarios. The dataset includes 30 very short, 1,061 short, and 2,439 long questions. These cybersecurity questions and answers are comprehensive, challenging, and not straightforward, making them ideal for our goal of developing an interactive QA system to teach cybersecurity to students.

5.2 Experiment and Parameter Settings↩︎

The knowledge base for our experiments is the CyberQ dataset [46], and the cybersecurity knowledge graph ontology utilized in this work is AISecKG [47]. In future work, the knowledge base can be expanded to include other PDF documents containing cybersecurity-related coursework material. For brevity, we will refer to the CyberQ dataset as our KB and AISecKG as our ontology. We conduct the main experiments in two scenarios: within the knowledge base (In-KB) and outside the knowledge base (Out-of-KB). For the In-KB scenario, students’ queries can be answered using existing documents in the knowledge base, while the Out-of-KB scenario involves queries for which relevant documents are not present in the knowledge base.

We consider BERTScore [48], METEOR [49], ROUGE-1 [50], and ROUGE-2 [50] as metrics to evaluate the performance of the generated response. It is important to note that the ground truth answers in the CyberQ dataset are annotated by human cybersecurity experts. Unless otherwise specified, each experiment is run 10 times, with average scores and standard deviations recorded.

We use the Contriever [51] as the retriever for document retrieval, employing cosine similarity as the semantic distance metric to identify and retrieve the most relevant documents from the knowledge base. For the generative language model, we utilize LLaMA3-8B-Instruct [52], with an input text length of 256 and max-length padding. During the answer generation process, the number of beams for beam search is set to 4. The ontology-based validation model is formulated using another instance of LLaMA3-8B-Instruct [52], with an input text length of 512 and max-length padding. For all other parameters not explicitly mentioned, default settings are applied.

5.3 Research Questions↩︎

To assess our approach, we employ the following evaluation methodology and begin by posing the following research questions (RQs):

• RQ1: How does the integration of a knowledge base enhance the performance of the CyberRAG framework in generating accurate and relevant answers for both In-KB and Out-of-KB queries?

• RQ2: What is the impact of the knowledge base and the generative language model on the overall performance of CyberRAG, and how does their ablation affect the system’s ability to answer students’ questions?

• RQ3: How does the integration of a knowledge base influence the quality and reliability of answers generated by the CyberRAG framework, and what are the key benefits observed from the retrieval process?

• RQ4: How effective is the ontology-based validation model in ensuring the accuracy, relevance, and scope of answers generated by CyberRAG. Can this validate and prevent potential misuse behaviors?

5.4 Quantitative Results↩︎

To address RQ1, we design a comparative experiment. As shown in Table 1, a⃝ CyberRAG consistently achieves good performance across all datasets and scenarios. For instance, the In-KB question-answering on the FS dataset gives a BertScore of 0.9461, a METEOR of 0.8588, a ROUGE-1 of 0.7882, and a ROUGE-2 of 0.7195. These results demonstrate that the answers generated by CyberRAG not only align well with the semantic meaning of the ground truth but also closely match in exact word terms. b⃝ Compared with the Out-of-KB, the model shows relatively higher performance for In-KB questions. For example, in the ZS subset for In-KB questions, the model achieves a BERTScore of 0.9294, and for more complex questions in the OD subset, it reaches 0.9331. In contrast, for Out-of-KB questions, the BERTScore is 0.8720 in ZS and 0.8783 for the OD subset. This indicates that the integration of the knowledge base provides a solid reference for generating answers for open-ended questions in cybersecurity, enhancing the model’s performance across both simple and complex queries.. c⃝ CyberRAG provides semantically meaningful and high-quality answers for questions outside the knowledge base. For example, in the OD dataset with the Out-of-KB settings, the BERTScore is 0.8783, while the METEOR, ROUGE-1, and ROUGE-2 are 0.3912, 0.2587, and 0.1234, respectively. This suggests that while the generated answers may not exactly match the ground truth in wording, they are highly similar to expert-certified answers in meaning. d⃝ Furthermore, the results show a very low standard deviation (std) in both In-KB and the Out-of-KB settings, with std values consistently below 0.0011. This demonstrates that CyberRAG is both stable and reliable in producing accurate answers. Overall, these findings suggest that our proposed method is promising for answering cybersecurity-related questions.

5.5 Ablation Study↩︎

We conducted an ablation study to investigate RQ2; we consider two scenarios: (i) CyberRAG is built solely on a question-answering dataset, such as CyberQ, without utilizing the generative language model. In this case, the QA system is unable to answer questions beyond the scope of the training dataset, which is undesirable as it limits the breadth of cybersecurity education. (ii) CyberRAG without augmenting the knowledge base. In this case, the model essentially reverts to functioning as a generative language model, answering questions in a zero-shot manner. As shown in Fig. 1, the results under zero-shot settings are considerably inferior. Compared to the In-KB scenarios, the model under zero-shot setting lacks access to the ground truth information for reference, leading to significant drop in performances. When comparing the zero-shot results with Out-of-KB scenarios, the latter demonstrates a better performance. This is because, although Out-of-kb can not be answered directly using documents from the knowledge base, these documents still provide relevant and supplementary information that may help in formulating the answers. In conclusion, both the knowledge base and the generative language model play significant roles in enhancing the performance of a QA systems in cybersecurity education.

5.6 Retrieval Analysis↩︎

To research RQ3, we design a retrieval analysis. First, we acquire the documents retrieved from the knowledge base and then compare the response produced by CyberRAG with the ground truth to explore how CyberRAG benefits from the retrieval augmentation process. We consider the Faithfulness, Answer Relevancy, Context Precision, Context Recall, and Context Entity Recall as metrics and use the RAGAS [53] package to evaluate our method.

As shown in Table 2, our proposed method CyberRAG shows satisfactory results on all metrics, especially on Answer Relevancy, Context Precision, Context Recall, and Context Entity Recall. These results indicate that the retrieved documents are highly relevant and accurate for answer generation and the generated response adheres to the reference documents, with minimal hallucination. It is also worth mentioning that there is still some room for improvement in the Faithfulness metrics. The lower score can be attributed to the generative model introducing additional knowledge from itself to enrich the answers, which is otherwise beneficial for the Out-of-KB question-answering.

5.7 Answer Validation Analysis with Case Study↩︎

We designed an answer validation analysis to explore RQ4. We simulate a case-study with an out-of-domain query, such as, “How to make money in the stock market?" - a query that can be considered an unintended or misused case. As shown in Fig. 3, although the generative language model produces the corresponding answer, it is filtered out by the validation model because it fails to adhere to cybersecurity knowledge graph ontology. Conversely, when a relevant cybersecurity question is posed, it smoothly passes the validation test, highlighting the critical role the ontology-based validation model in ensuring accuracy and preventing potential misuse. Fig. 3 illustrates the data flow in the case study, demonstrating the transparency and reliability of CyberRAG.

6 Conclusion↩︎

The rapid advancement of AI is transforming education, particularly in technical fields like cybersecurity. AI-driven QA systems enhance cognitive engagement by managing uncertainty in problem-based learning. Our proposed CyberRAG introduces a novel, ontology-aware retrieval-augmented generation approach to create a reliable QA system for cybersecurity education. By leveraging domain knowledge and an ontology-based validation model, CyberRAG ensures relevance, accuracy, and safety of responses. Comprehensive experiments demonstrate its dependability in real-world scenarios, fostering a more interactive and secure learning environment. This research highlights the potential of AI to transform educational practices, not only in cybersecurity but across various subjects. As AI in education evolves, future research will explore innovative methods like virtual environments to further enhance students’ practical experiences using LLM agents.

References↩︎