📋 AI Review from DeepReviewer will be automatically processed
📋 AI Review from ZGCA will be automatically processed
The paper surveys Large Language Model (LLM)-empowered Knowledge Graph (KG) construction. It revisits the classical three-stage pipeline—ontology engineering (Section 3), knowledge extraction (Section 4), and knowledge fusion (Section 5)—and analyzes how LLMs reshape each stage. For ontology engineering, it contrasts top-down (LLMs as ontology assistants; Section 3.1, including CQ-based and natural language-based workflows) and bottom-up schema induction serving LLMs/RAG (Section 3.2). For extraction, it distinguishes schema-based approaches (static vs dynamic/adaptive schemas; Section 4.1) from schema-free approaches (structured generative extraction and open information extraction; Section 4.2). For fusion, it covers schema-level fusion, instance-level fusion, and hybrid frameworks (Section 5). The survey also outlines future directions (Section 6), including KG-based reasoning for LLMs, dynamic agent memory with KGs, multimodal KG construction, and roles for KGs beyond RAG.
Cross‑Modal Consistency: 34/50
Textual Logical Soundness: 26/30
Visual Aesthetics & Clarity: 10/20
Overall Score: 70/100
Detailed Evaluation (≤500 words):
1. Cross‑Modal Consistency
• Visual ground truth: Fig. 1 is a single‑pane mind‑map taxonomy. Central node “LLM for KGC” branches to: LLM‑Enhanced Ontology Construction (top‑down/bottom‑up), LLM‑Driven Knowledge Extraction (schema‑based: static/dynamic; schema‑free: structured generative/OIE), LLM‑Powered Knowledge Fusion (schema‑level/instance‑level/hybrid), and Future Applications, each listing exemplar works.
• Major 1: Fig. 1 is illegible at normal print size; most labels and citations cannot be read, blocking verification of text‑figure mappings. Evidence: Fig. 1 raster resolution 512×238 with dense small fonts; labels unreadable at 100% zoom.
• Minor 1: Section heading typo mismatches figure/category (“INSSTANCE‑LEVEL FUSION” vs. “Instance‑Level Fusion”). Evidence: Sec 5.2 header string “INSSTANCE‑LEVEL FUSION”.
• Minor 2: Duplicate insertion of the same taxonomy figure (appears twice), creating ambiguity about numbering. Evidence: Two separate image blocks, both captioned “Figure 1: Taxonomy of LLM for KGC”.
• Minor 3: Ambiguous naming “KG‑RAR (Wu et al., 2025)” not matched in refs title (“Graph‑Augmented Reasoning…”). Evidence: Sec 6.1 vs. reference entry “Wu et al., 2025”.
2. Text Logic
• No Major issues found.
• Minor 1: Incomplete bibliographic entry “Kommineni et al., 2024. Journal Name”. Evidence: References: “Journal Name, 2024.”
• Minor 2: Occasional typos and hard line breaks (e.g., “orga nization”). Evidence: Sec 4.2.1 split word around line “reasoning‑driven organiza tion.”
• Minor 3: Several future‑dated/arXiv‑only claims are reported without brief quantitative context (e.g., “comparable to novice modelers”), reducing evidential sharpness for a few assertions. Evidence: Sec 3.1.2 statements citing Lippolis et al., 2025b.
3. Figure Quality
• Major 1: Critical labels and citations in Fig. 1 are too small; the figure fails the “figure‑alone” test. Evidence: Fig. 1 text unreadable at print size; dense multi‑level bullets.
• Minor 1: Colour contrast is adequate but thin connector lines and tight spacing crowd content, slowing parsing. Evidence: Fig. 1 multi‑branch nodes with tightly packed item lists.
Key strengths:
Key weaknesses:
Recommendations:
📋 AI Review from SafeReviewer will be automatically processed
This paper provides a comprehensive survey of the evolving landscape of Knowledge Graph Construction (KGC) in the era of Large Language Models (LLMs). The authors meticulously trace the historical development of KGC methodologies, from traditional rule-based and statistical approaches to the current paradigm dominated by LLMs. The core contribution of this work lies in its systematic analysis of how LLMs are reshaping the three fundamental stages of KGC: ontology construction, knowledge extraction, and knowledge fusion. The paper introduces a useful taxonomy that categorizes LLM-based KGC methods into schema-based and schema-free paradigms, offering a clear framework for understanding the diverse approaches in this field. The authors delve into the technical details of various LLM-driven techniques, highlighting their strengths and limitations. For instance, in ontology construction, they discuss methods that leverage LLMs as assistants to human experts, as well as approaches that use KGs to enhance LLM reasoning. In knowledge extraction, they explore both methods that rely on predefined schemas and those that operate without such constraints. Similarly, in knowledge fusion, they examine techniques for integrating knowledge from different sources, addressing challenges such as entity alignment and conflict resolution. The paper also identifies key trends and future research directions, including the development of KG-based reasoning for LLMs, dynamic knowledge memory for agentic systems, and multimodal KG construction. The authors emphasize the potential of LLMs to overcome the limitations of traditional KGC methods, such as scalability issues, expert dependency, and pipeline fragmentation. They argue that LLMs enable a shift from rule-based pipelines to language-driven, generative frameworks, offering a more unified and adaptive approach to KGC. The paper concludes by highlighting the transformative impact of LLMs on the field of KGC, positioning them as a crucial technology for building intelligent, adaptive, and explainable knowledge systems. Overall, this survey serves as a valuable resource for researchers and practitioners interested in the intersection of LLMs and KGC, providing a thorough overview of the current state of the art and pointing towards promising avenues for future research.
This paper demonstrates several notable strengths that contribute to its overall value as a survey. First and foremost, the authors have conducted a thorough and comprehensive review of the relevant literature. They have successfully traced the evolution of KGC methodologies, providing a clear historical context for understanding the impact of LLMs on this field. The paper's systematic analysis of the three core stages of KGC—ontology construction, knowledge extraction, and knowledge fusion—is particularly commendable. By breaking down the problem into these distinct components, the authors are able to provide a more detailed and nuanced analysis of the various LLM-driven approaches. The introduction of a taxonomy that categorizes LLM-based KGC methods into schema-based and schema-free paradigms is another significant strength. This taxonomy provides a useful framework for understanding the diverse range of techniques in this field, helping readers to navigate the complex landscape of LLM-based KGC. Furthermore, the paper provides a detailed description of the technical mechanisms employed by various LLM-driven methods. The authors delve into the specifics of different approaches, highlighting their strengths and limitations. This level of detail is particularly valuable for researchers who are interested in implementing or building upon these methods. The paper also effectively identifies key trends and future research directions. The authors discuss the potential of LLMs to enable KG-based reasoning, dynamic knowledge memory, and multimodal KG construction. These insights are particularly valuable for guiding future research in this field. Finally, the paper is generally well-organized and clearly written. The authors present complex concepts in a clear and accessible manner, making the paper easy to follow even for readers who are not experts in this field. The use of examples and case studies further enhances the clarity of the paper, helping readers to understand the practical implications of the various LLM-driven approaches.
Despite its strengths, this paper exhibits several weaknesses that warrant careful consideration. One significant limitation is the lack of in-depth analysis regarding the practical challenges associated with using LLMs for KGC. While the paper mentions issues like hallucinations and reliability, it does not delve into the specific mechanisms through which these problems manifest in the context of KGC. For instance, the paper fails to explore how the inherent biases in LLM training data can lead to skewed or inaccurate knowledge graphs. It also lacks a discussion on the potential for LLMs to generate contradictory information, which can undermine the consistency and reliability of the constructed KGs. This lack of detailed analysis is particularly concerning given the known limitations of LLMs in this area. The paper also lacks a thorough evaluation of the practical implications of using LLMs for KGC. While the paper mentions the computational cost of LLMs, it does not provide a detailed analysis of the trade-offs between accuracy, efficiency, and cost. It fails to discuss how the computational cost of LLMs scales with the size and complexity of the knowledge graphs being constructed. Furthermore, the paper does not address the practical challenges of deploying LLM-based KGC systems in real-world applications, such as the need for continuous updates and maintenance. This lack of practical consideration limits the paper's usefulness for researchers and practitioners who are interested in implementing these methods. My analysis confirms that the paper does not include any empirical evaluations of the presented methods. This is a major weakness, as it makes it difficult to assess the real-world performance of LLM-based KGC. The paper primarily focuses on describing existing methods and does not include any experiments or benchmarks to evaluate their effectiveness. This lack of empirical validation makes it difficult to compare the performance of different LLM-based KGC methods and to assess their strengths and weaknesses in different scenarios. The paper also lacks a detailed comparison of LLM-based KGC methods with traditional knowledge graph construction techniques. While the paper introduces a taxonomy of LLM-based methods, it does not provide a systematic comparison of these methods with traditional approaches. This makes it difficult to understand the relative advantages and disadvantages of using LLMs for KGC. The paper does not discuss the specific scenarios where LLM-based methods are more suitable than traditional methods, and vice versa. This lack of comparative analysis limits the paper's ability to provide practical guidance for researchers and practitioners who are considering using LLMs for KGC. Furthermore, the paper does not adequately address the issue of evaluating the quality of LLM-constructed KGs. While the paper mentions the need for evaluation, it does not delve into the specific challenges of evaluating the quality of LLM-constructed KGs, especially given the potential for hallucination and inconsistency. The paper fails to discuss the need for new evaluation metrics that go beyond simple accuracy measures and consider the faithfulness and trustworthiness of the generated knowledge. Finally, the paper's organization could be improved. While the paper follows a logical structure, the flow between sections could be smoother, with more explicit connections and transitions. The paper also lacks a dedicated section that consolidates the limitations of LLM-based KGC methods. This makes it difficult for readers to get a clear overview of the challenges associated with these methods. My analysis of the paper confirms these weaknesses, and I have high confidence in these findings, as they are directly supported by the paper's content and the absence of certain discussions.
To address the identified weaknesses, I recommend several concrete improvements. First, the paper should include a more in-depth analysis of the practical challenges associated with using LLMs for KGC. This analysis should delve into the specific mechanisms through which problems like hallucinations and biases manifest in the context of KGC. The authors should explore how the inherent biases in LLM training data can lead to skewed or inaccurate knowledge graphs. They should also discuss the potential for LLMs to generate contradictory information and how this can undermine the consistency and reliability of the constructed KGs. Furthermore, the paper should provide a more thorough evaluation of the practical implications of using LLMs for KGC. This evaluation should include a detailed analysis of the trade-offs between accuracy, efficiency, and cost. The authors should discuss how the computational cost of LLMs scales with the size and complexity of the knowledge graphs being constructed. They should also address the practical challenges of deploying LLM-based KGC systems in real-world applications, such as the need for continuous updates and maintenance. To address the lack of empirical validation, the paper should include a more detailed discussion of evaluation metrics and benchmarks. The authors should explore the use of metrics that go beyond simple accuracy measures and consider the faithfulness and trustworthiness of the generated knowledge. They should also discuss the need for standardized benchmarks that allow for a fair comparison of different LLM-based KGC methods. This would help to establish a more rigorous and reliable way to assess the performance of these methods. The paper should also include a more detailed comparison of LLM-based KGC methods with traditional knowledge graph construction techniques. This comparison should highlight the specific scenarios where LLM-based methods are more suitable than traditional methods, and vice versa. The authors should discuss the specific advantages and disadvantages of each approach, providing a more balanced and nuanced perspective on the field. To improve the paper's organization, the authors should consider restructuring the paper to better highlight the key contributions and insights. The current structure, while comprehensive, makes it difficult to discern the core arguments and novel findings. A more effective approach would be to begin with a clear problem statement, outlining the specific challenges in KG construction that LLMs aim to address. This should be followed by a detailed discussion of the proposed taxonomy, explicitly stating the criteria for categorizing different LLM-based KGC methods. The authors should also consider adding a dedicated section that consolidates the limitations of LLM-based KGC methods. This would provide a more balanced perspective on the field and help readers to understand the challenges associated with these methods. Finally, the paper should include a more thorough discussion of the ethical implications of using LLMs for KGC. This discussion should address issues such as data privacy, bias, and the potential for misuse of the constructed knowledge graphs. By addressing these ethical concerns, the paper would provide a more complete and responsible analysis of the field.
Several key questions arise from my analysis of this paper. First, given the known limitations of LLMs, such as their tendency to hallucinate and their reliance on potentially biased training data, what specific techniques can be employed to mitigate these issues in the context of KGC? I am particularly interested in understanding how techniques like fine-tuning, prompt engineering, or the use of external knowledge sources can be used to improve the reliability and accuracy of LLM-constructed KGs. Second, how can we develop more robust and reliable evaluation metrics for LLM-constructed KGs? The paper acknowledges the need for evaluation but does not delve into the specific challenges of evaluating the quality of LLM-constructed KGs, especially given the potential for hallucination and inconsistency. I am curious to know what new evaluation metrics could be developed that go beyond simple accuracy measures and consider the faithfulness and trustworthiness of the generated knowledge. Third, what are the most promising directions for future research in LLM-based KGC? The paper mentions several future directions, including KG-based reasoning, dynamic knowledge memory, and multimodal KG construction. However, I would like to hear the authors' thoughts on which of these directions they believe holds the most potential for overcoming the current limitations of LLM-based KGC. Fourth, how can we address the computational cost and scalability challenges associated with using LLMs for KGC? The paper acknowledges the computational cost of LLMs but does not provide a detailed analysis of how this cost scales with the size and complexity of the knowledge graphs being constructed. I am interested in understanding what techniques can be used to reduce the computational cost of LLM-based KGC and to make it more scalable for real-world applications. Finally, what are the ethical implications of using LLMs for KGC? The paper does not address the ethical concerns associated with using LLMs for KGC, such as data privacy, bias, and the potential for misuse of the constructed knowledge graphs. I would like to know what steps can be taken to ensure that LLM-based KGC is conducted in an ethical and responsible manner.