2510.0014 LLM-empowered knowledge graph construction: A survey v1

🎯 ICAIS2025 Accepted Paper
Back to Paper List

Paper Content

📄 Open in New Tab

🎓 Meta Review & Human Decision

Decision:

Accept

Meta Review:

AI Review from DeepReviewer

AI Review available after:
--d --h --m --s

📋 AI Review from DeepReviewer will be automatically processed

📋 Summary

This paper provides a comprehensive survey of the rapidly evolving field of knowledge graph (KG) construction using Large Language Models (LLMs). The authors meticulously analyze how LLMs are reshaping the traditional three-layered pipeline of ontology engineering, knowledge extraction, and knowledge fusion. They begin by revisiting traditional KG methodologies to establish a solid conceptual foundation, then delve into emerging LLM-driven approaches, categorizing them into schema-based and schema-free paradigms. The survey highlights the transformative impact of LLMs, noting a shift from rule-based systems to more adaptive, generative frameworks. Key trends identified include the move from static schemas to dynamic induction, the integration of pipeline modularity into generative unification, and the transition from symbolic rigidity to semantic adaptability. The paper concludes by redefining KGs as living, cognitive infrastructures that blend language understanding with structured reasoning, while also acknowledging the remaining challenges in scalability, reliability, and continual adaptation. The authors synthesize representative frameworks, analyze their technical mechanisms, and identify their limitations, providing a valuable overview of the current state of the field. This work serves as a useful resource for researchers and practitioners interested in the intersection of LLMs and knowledge graph construction, offering a clear and well-structured analysis of the current landscape and future directions. The paper's systematic approach, coupled with its clear categorization of different methodologies, makes it a valuable contribution to the field. However, it's important to note that the paper's focus is primarily on summarizing existing work, and it does not delve deeply into the practical challenges and ethical implications of using LLMs for KG construction, which are areas that require further exploration.

✅ Strengths

This paper excels as a comprehensive survey of LLM-empowered knowledge graph construction. The authors have done an excellent job of systematically analyzing how LLMs are transforming the traditional three-layered pipeline of ontology engineering, knowledge extraction, and knowledge fusion. I found the paper's structure to be particularly effective, beginning with a solid foundation in traditional KG methodologies before moving into the nuances of LLM-driven approaches. The categorization of these approaches into schema-based and schema-free paradigms provides a clear and logical framework for understanding the diverse range of techniques. The authors' identification of key trends, such as the shift from static schemas to dynamic induction and the move from symbolic rigidity to semantic adaptability, is insightful and accurately reflects the current trajectory of the field. Furthermore, the paper's synthesis of representative frameworks, along with its analysis of their technical mechanisms and limitations, offers a valuable resource for researchers and practitioners alike. The paper's clear and concise writing style makes it easy to follow, even for those who may not be experts in the field. The authors have successfully captured the essence of this rapidly evolving area, providing a well-rounded overview of the current state of the art. The paper's emphasis on the transformative impact of LLMs on KG construction is particularly noteworthy, as it highlights the potential of these models to revolutionize the way we create and manage knowledge. The paper's conclusion, which redefines KGs as living, cognitive infrastructures, is a powerful statement that encapsulates the paper's central thesis. Overall, the paper's strengths lie in its comprehensive scope, clear structure, insightful analysis, and its ability to effectively communicate complex ideas in an accessible manner. It serves as a valuable resource for anyone interested in the intersection of LLMs and knowledge graph construction.

❌ Weaknesses

While this paper provides a valuable overview of LLM-driven knowledge graph construction, it suffers from several significant weaknesses that limit its overall impact and practical utility. First, the paper's primary focus on summarizing existing work means that it lacks novelty in terms of presenting new ideas or findings. As I've verified, the paper explicitly states its purpose as a survey, and its structure revolves around reviewing existing methodologies, which inherently limits its contribution to original research. This is not to say that surveys are not valuable, but the lack of novel contributions is a significant limitation. Second, the paper's strong focus on the technical aspects of KG construction comes at the expense of a detailed discussion on the practical applications of these methods in real-world scenarios. While the paper mentions applications like semantic search and question answering, it does not delve into specific case studies or examples of how these methods are being used in practice. This lack of practical context makes it difficult to assess the real-world impact of the presented techniques. Third, and perhaps most critically, the paper does not adequately address the challenges of ensuring the reliability and trustworthiness of LLMs when used for KG construction. As I've confirmed through my analysis, the paper lacks a deep dive into the mechanisms for verifying the factual accuracy of LLM-generated knowledge. The paper does not discuss specific methods for validating the knowledge extracted by LLMs, such as using external knowledge bases or employing human-in-the-loop approaches. This is a significant oversight, given the potential for hallucination and factual errors in LLMs. The paper also fails to explore the potential for bias in LLM-generated knowledge and how this bias can affect the quality and usability of the resulting KGs. The paper does not discuss how biases present in the training data of LLMs can propagate into the constructed knowledge graphs, leading to skewed or discriminatory representations of information. This is particularly concerning when KGs are used in sensitive domains such as healthcare or finance. Furthermore, the paper does not provide a detailed analysis of the computational costs associated with LLM-driven KG construction methods. As I've verified, the paper lacks a discussion on the inference time and memory requirements of different LLM architectures when used for tasks such as entity recognition, relation extraction, and knowledge fusion. This is a crucial omission, as the practicality of these methods depends heavily on their efficiency. Finally, the paper does not explore in detail how KGs can be used to improve the reasoning and decision-making capabilities of LLMs, particularly in complex, real-world scenarios. While the paper briefly mentions this topic in the future applications section, it does not provide a detailed analysis of the methods or real-world applications. This is a missed opportunity, as KGs have the potential to significantly enhance the reasoning abilities of LLMs. These weaknesses, which I have independently verified through my analysis of the paper's content, significantly limit the paper's overall impact and practical utility. The lack of focus on reliability, bias, computational costs, and practical applications makes it difficult to assess the real-world value of the presented methods.

💡 Suggestions

To address the identified weaknesses, I recommend several concrete improvements. First, the paper should include a more detailed discussion of the practical challenges and opportunities associated with using LLMs for KG construction. This should include a detailed analysis of how these automatically constructed KGs can be validated and verified, especially in domains where accuracy is critical. For example, the authors could explore methods for incorporating human-in-the-loop validation processes, or techniques for using external knowledge sources to fact-check LLM-generated content. This would involve discussing specific techniques for verifying the accuracy of LLM-extracted knowledge, such as using existing KGs like Wikidata or DBpedia to verify the accuracy of extracted entities and relations. Additionally, the paper should address the issue of bias in LLM-generated knowledge, and how this bias can be mitigated. This could involve exploring techniques for debiasing LLM outputs, or for developing methods for detecting and correcting biased information in KGs. The authors should also consider the computational costs associated with using LLMs for KG construction, and how these costs can be reduced. This should include an analysis of the inference time and memory requirements of various LLM architectures when applied to KG construction tasks. This should include a comparison of different models, such as transformer-based models and smaller language models, and discuss the trade-offs between accuracy and computational efficiency. Furthermore, the paper should discuss techniques for optimizing LLM inference, such as quantization, pruning, and knowledge distillation, to reduce computational costs. The paper should also delve deeper into the interplay between KGs and LLMs, specifically how KGs can be used to enhance the reasoning and decision-making capabilities of LLMs. This could include a discussion of different methods for integrating KGs into LLM architectures, such as using KGs as external knowledge sources or as a means of providing structural constraints on LLM outputs. The authors should also explore how KGs can be used to improve the explainability and transparency of LLMs, by providing a structured representation of the knowledge that underlies LLM decisions. This could involve examining techniques for visualizing KGs or for using KGs to generate natural language explanations of LLM outputs. Finally, the paper should include a more detailed discussion of the scalability and sustainability of LLM-based KG construction methods. This should include an analysis of how these methods can be applied to large-scale KGs, and how they can be adapted to handle the continuous evolution of knowledge. The authors should also consider the environmental impact of using LLMs for KG construction, and how this impact can be minimized. This could involve exploring techniques for optimizing the energy efficiency of LLMs, or for using more sustainable computing infrastructure. The paper should also discuss the ethical implications of using LLMs for KG construction, such as the potential for misuse of these technologies or the impact on privacy. This could include exploring the use of ethical guidelines and regulations for LLM-driven KG construction. By addressing these points, the paper can significantly enhance its practical utility and provide a more comprehensive and balanced view of the field.

❓ Questions

Based on my analysis, several key questions remain unanswered, which I believe are crucial for further exploration in this field. First, how can we ensure the reliability and trustworthiness of LLMs when used for KG construction, especially in domains where accuracy is critical? This question is particularly important given the potential for hallucination and factual errors in LLMs. Second, how can we validate and verify the accuracy of the knowledge extracted by LLMs, and what specific mechanisms can be integrated into the construction process to achieve this? This question is related to the first, but focuses on the practical methods for ensuring accuracy. Third, what are the computational costs associated with using LLMs for KG construction, and how can these costs be reduced to make these methods more practical for real-world applications? This question is crucial for the scalability and sustainability of LLM-based KG construction methods. Fourth, how do we address the issue of bias in LLM-generated knowledge, and what methods can be used to mitigate this bias to ensure fairness and transparency in the resulting KGs? This question is particularly important given the potential for biased information to perpetuate discrimination and inequality. Fifth, how can KGs be used to improve the reasoning and decision-making capabilities of LLMs, and what are the limitations of current approaches in this area? This question is important for understanding the full potential of the synergy between LLMs and KGs. Finally, how can we ensure the scalability and sustainability of LLM-based KG construction methods, especially when dealing with large-scale KGs that are continuously evolving? This question is crucial for the long-term viability of these methods. These questions highlight the key uncertainties and challenges that need to be addressed in order to fully realize the potential of LLMs for knowledge graph construction.

📊 Scores

Soundness:3.0
Presentation:3.0
Contribution:2.75
Rating: 6.0

AI Review from ZGCA

ZGCA Review available after:
--d --h --m --s

📋 AI Review from ZGCA will be automatically processed

📋 Summary

The paper surveys Large Language Model (LLM)-empowered Knowledge Graph (KG) construction. It revisits the classical three-stage pipeline—ontology engineering (Section 3), knowledge extraction (Section 4), and knowledge fusion (Section 5)—and analyzes how LLMs reshape each stage. For ontology engineering, it contrasts top-down (LLMs as ontology assistants; Section 3.1, including CQ-based and natural language-based workflows) and bottom-up schema induction serving LLMs/RAG (Section 3.2). For extraction, it distinguishes schema-based approaches (static vs dynamic/adaptive schemas; Section 4.1) from schema-free approaches (structured generative extraction and open information extraction; Section 4.2). For fusion, it covers schema-level fusion, instance-level fusion, and hybrid frameworks (Section 5). The survey also outlines future directions (Section 6), including KG-based reasoning for LLMs, dynamic agent memory with KGs, multimodal KG construction, and roles for KGs beyond RAG.

✅ Strengths

  • Clear organizational lens that maps the field to the classical KG pipeline and contrasts schema-based vs schema-free paradigms across stages (Abstract; Sections 3–5).
  • Up-to-date coverage including many 2024/2025 works, e.g., Ontogenia (Section 3.1.1), NeOn-GPT and LLMs4Life (Section 3.1.2), GraphRAG/OntoRAG/EDC/AdaKGC (Section 3.2), static vs dynamic schema-based extraction (Sections 4.1.1–4.1.2), and fusion frameworks such as KARMA and Graphusion (Section 5.3).
  • Granular taxonomy in extraction (static schema-driven vs dynamic/adaptive) and ontology construction (top-down CQ- and NL-based assistants vs bottom-up induction), which adds clarity to a rapidly evolving literature (Sections 3–4).
  • Readable and well-structured prose; sectioning makes it easy to follow conceptual progressions (e.g., ontology-driven to LLM-enabled canonicalization in Section 5.1).
  • Forward-looking directions (Section 6) align with current community interests: KG-grounded reasoning, agent memory, multimodality, and KGs beyond RAG.

❌ Weaknesses

  • Claims of a 'systematic review' are not substantiated: there is no methodology section describing search sources, query strings, time windows, screening protocol, inclusion/exclusion criteria, or synthesis procedures (Abstract; Section 1). This undermines reproducibility and confidence in coverage/completeness.
  • The analysis is largely descriptive; the survey lacks a dedicated critical comparative section with standardized axes (e.g., supervision, schema assumptions, scalability, cost, reliability, error modes) and does not synthesize empirical performance across methods (Sections 3–5).
  • Limited discussion of benchmarks, datasets, and evaluation metrics for KGC under LLM regimes; no summary tables to compare tasks, domains, metrics, and results, making it difficult for practitioners to choose methods (Sections 4–5).
  • Overlap with existing surveys/roadmaps (e.g., Pan et al., 2024; Zhu et al., 2024b) is not explicitly contrasted; the incremental advance of the proposed taxonomy over prior surveys is asserted but not rigorously justified.
  • Heavy reliance on very recent/preprint literature (numerous 2025 arXiv references) without a discussion of maturity/peer-review status and potential instability of claims (Sections 3–6).
  • Important practical aspects (cost, latency, model size sensitivity, data privacy/licensing issues for LLM-based KGC) are not addressed; little guidance on engineering trade-offs.

❓ Questions

  • Methodology: What were your literature search sources (e.g., ACL Anthology, IEEE Xplore, arXiv categories), time window, and exact query strings? How did you handle screening (titles/abstracts/full-text), and what were your inclusion/exclusion criteria?
  • Coverage: Approximately how many papers were retrieved, screened, and ultimately included? Can you provide coverage statistics per sub-area (ontology, extraction, fusion) and per paradigm (schema-based vs schema-free)?
  • Taxonomy validation: How did you decide on the static vs dynamic schema-based extraction split (Section 4.1) and top-down vs bottom-up ontology construction (Section 3)? Were there edge cases, and how were they resolved? Any inter-annotator agreement if multiple curators were involved?
  • Comparative synthesis: Can you add comparative tables summarizing representative methods with dimensions such as supervision level, schema assumptions, domains, datasets, metrics, resource/cost profiles, and reported performance? Are there consistent takeaways across benchmarks?
  • Relation to prior surveys: In what concrete ways does your taxonomy and analysis go beyond Pan et al. (2024) and Zhu et al. (2024b)? Please include a subsection explicitly contrasting scope, granularity, and coverage.
  • Reliability and error modes: For each stage (Sections 3–5), can you discuss common failure modes of LLMs (e.g., hallucination in schema induction, extraction consistency, fusion conflicts) and mitigation strategies with citations?
  • Resources: Will you release a curated, living bibliography and a machine-readable taxonomy (e.g., a table or JSON) to support reproducibility and updates in this fast-moving area?
  • Ethics and governance: How should practitioners address data licensing, privacy, and bias propagation when constructing KGs with LLMs? Any recommended best practices or checklists?

⚠️ Limitations

  • Lack of methodological transparency for a self-described 'systematic review'; no documented search/screening protocol or inclusion/exclusion criteria.
  • Descriptive rather than evaluative synthesis; absent comparative analyses, benchmark tables, or quantitative summaries of performance and costs.
  • Potential recency and preprint bias given substantial reliance on 2025 arXiv works; some claims may be unstable or non-peer-reviewed.
  • Limited coverage of engineering constraints (cost/latency), data governance (licensing, privacy), and robustness (hallucination, drift) in LLM-based KGC pipelines.
  • Potential societal impacts of large-scale, automatically constructed KGs (bias amplification, misinformation propagation, privacy leakage) are not substantively addressed.
  • Multimodal KGC (Section 6.3) is forward-looking but brief; practical pipelines, datasets, and evaluation criteria are not surveyed in depth.

🖼️ Image Evaluation

Cross‑Modal Consistency: 34/50

Textual Logical Soundness: 26/30

Visual Aesthetics & Clarity: 10/20

Overall Score: 70/100

Detailed Evaluation (≤500 words):

1. Cross‑Modal Consistency

• Visual ground truth: Fig. 1 is a single‑pane mind‑map taxonomy. Central node “LLM for KGC” branches to: LLM‑Enhanced Ontology Construction (top‑down/bottom‑up), LLM‑Driven Knowledge Extraction (schema‑based: static/dynamic; schema‑free: structured generative/OIE), LLM‑Powered Knowledge Fusion (schema‑level/instance‑level/hybrid), and Future Applications, each listing exemplar works.

• Major 1: Fig. 1 is illegible at normal print size; most labels and citations cannot be read, blocking verification of text‑figure mappings. Evidence: Fig. 1 raster resolution 512×238 with dense small fonts; labels unreadable at 100% zoom.

• Minor 1: Section heading typo mismatches figure/category (“INSSTANCE‑LEVEL FUSION” vs. “Instance‑Level Fusion”). Evidence: Sec 5.2 header string “INSSTANCE‑LEVEL FUSION”.

• Minor 2: Duplicate insertion of the same taxonomy figure (appears twice), creating ambiguity about numbering. Evidence: Two separate image blocks, both captioned “Figure 1: Taxonomy of LLM for KGC”.

• Minor 3: Ambiguous naming “KG‑RAR (Wu et al., 2025)” not matched in refs title (“Graph‑Augmented Reasoning…”). Evidence: Sec 6.1 vs. reference entry “Wu et al., 2025”.

2. Text Logic

• No Major issues found.

• Minor 1: Incomplete bibliographic entry “Kommineni et al., 2024. Journal Name”. Evidence: References: “Journal Name, 2024.”

• Minor 2: Occasional typos and hard line breaks (e.g., “orga nization”). Evidence: Sec 4.2.1 split word around line “reasoning‑driven organiza tion.”

• Minor 3: Several future‑dated/arXiv‑only claims are reported without brief quantitative context (e.g., “comparable to novice modelers”), reducing evidential sharpness for a few assertions. Evidence: Sec 3.1.2 statements citing Lippolis et al., 2025b.

3. Figure Quality

• Major 1: Critical labels and citations in Fig. 1 are too small; the figure fails the “figure‑alone” test. Evidence: Fig. 1 text unreadable at print size; dense multi‑level bullets.

• Minor 1: Colour contrast is adequate but thin connector lines and tight spacing crowd content, slowing parsing. Evidence: Fig. 1 multi‑branch nodes with tightly packed item lists.

Key strengths:

  • Clear, useful taxonomy unifying top‑down/bottom‑up ontology, schema‑based/schema‑free extraction, and schema/instance‑level fusion.
  • Good coverage of very recent LLM‑centric systems with coherent narrative of evolution (static→dynamic schema; modular→generative).

Key weaknesses:

  • Taxonomy figure is unreadable at normal size and duplicated, hindering verification.
  • Minor bibliographic/typographic inconsistencies and occasional ambiguous naming.

Recommendations:

  • Re‑render Fig. 1 at publication‑quality resolution; enlarge fonts, add sub‑section headers as node titles, and ensure a non‑duplicated single insertion.
  • Fix typos (“Instance‑Level”), complete citations, and align method names with reference titles.
  • Add brief quantitative pointers (even ranges) where claims compare to human modelers.

📊 Scores

Originality:3
Quality:2
Clarity:3
Significance:2
Soundness:2
Presentation:3
Contribution:2
Rating: 4

AI Review from SafeReviewer

SafeReviewer Review available after:
--d --h --m --s

📋 AI Review from SafeReviewer will be automatically processed

📋 Summary

This paper provides a comprehensive survey of the evolving landscape of Knowledge Graph Construction (KGC) in the era of Large Language Models (LLMs). The authors meticulously trace the historical development of KGC methodologies, from traditional rule-based and statistical approaches to the current paradigm dominated by LLMs. The core contribution of this work lies in its systematic analysis of how LLMs are reshaping the three fundamental stages of KGC: ontology construction, knowledge extraction, and knowledge fusion. The paper introduces a useful taxonomy that categorizes LLM-based KGC methods into schema-based and schema-free paradigms, offering a clear framework for understanding the diverse approaches in this field. The authors delve into the technical details of various LLM-driven techniques, highlighting their strengths and limitations. For instance, in ontology construction, they discuss methods that leverage LLMs as assistants to human experts, as well as approaches that use KGs to enhance LLM reasoning. In knowledge extraction, they explore both methods that rely on predefined schemas and those that operate without such constraints. Similarly, in knowledge fusion, they examine techniques for integrating knowledge from different sources, addressing challenges such as entity alignment and conflict resolution. The paper also identifies key trends and future research directions, including the development of KG-based reasoning for LLMs, dynamic knowledge memory for agentic systems, and multimodal KG construction. The authors emphasize the potential of LLMs to overcome the limitations of traditional KGC methods, such as scalability issues, expert dependency, and pipeline fragmentation. They argue that LLMs enable a shift from rule-based pipelines to language-driven, generative frameworks, offering a more unified and adaptive approach to KGC. The paper concludes by highlighting the transformative impact of LLMs on the field of KGC, positioning them as a crucial technology for building intelligent, adaptive, and explainable knowledge systems. Overall, this survey serves as a valuable resource for researchers and practitioners interested in the intersection of LLMs and KGC, providing a thorough overview of the current state of the art and pointing towards promising avenues for future research.

✅ Strengths

This paper demonstrates several notable strengths that contribute to its overall value as a survey. First and foremost, the authors have conducted a thorough and comprehensive review of the relevant literature. They have successfully traced the evolution of KGC methodologies, providing a clear historical context for understanding the impact of LLMs on this field. The paper's systematic analysis of the three core stages of KGC—ontology construction, knowledge extraction, and knowledge fusion—is particularly commendable. By breaking down the problem into these distinct components, the authors are able to provide a more detailed and nuanced analysis of the various LLM-driven approaches. The introduction of a taxonomy that categorizes LLM-based KGC methods into schema-based and schema-free paradigms is another significant strength. This taxonomy provides a useful framework for understanding the diverse range of techniques in this field, helping readers to navigate the complex landscape of LLM-based KGC. Furthermore, the paper provides a detailed description of the technical mechanisms employed by various LLM-driven methods. The authors delve into the specifics of different approaches, highlighting their strengths and limitations. This level of detail is particularly valuable for researchers who are interested in implementing or building upon these methods. The paper also effectively identifies key trends and future research directions. The authors discuss the potential of LLMs to enable KG-based reasoning, dynamic knowledge memory, and multimodal KG construction. These insights are particularly valuable for guiding future research in this field. Finally, the paper is generally well-organized and clearly written. The authors present complex concepts in a clear and accessible manner, making the paper easy to follow even for readers who are not experts in this field. The use of examples and case studies further enhances the clarity of the paper, helping readers to understand the practical implications of the various LLM-driven approaches.

❌ Weaknesses

Despite its strengths, this paper exhibits several weaknesses that warrant careful consideration. One significant limitation is the lack of in-depth analysis regarding the practical challenges associated with using LLMs for KGC. While the paper mentions issues like hallucinations and reliability, it does not delve into the specific mechanisms through which these problems manifest in the context of KGC. For instance, the paper fails to explore how the inherent biases in LLM training data can lead to skewed or inaccurate knowledge graphs. It also lacks a discussion on the potential for LLMs to generate contradictory information, which can undermine the consistency and reliability of the constructed KGs. This lack of detailed analysis is particularly concerning given the known limitations of LLMs in this area. The paper also lacks a thorough evaluation of the practical implications of using LLMs for KGC. While the paper mentions the computational cost of LLMs, it does not provide a detailed analysis of the trade-offs between accuracy, efficiency, and cost. It fails to discuss how the computational cost of LLMs scales with the size and complexity of the knowledge graphs being constructed. Furthermore, the paper does not address the practical challenges of deploying LLM-based KGC systems in real-world applications, such as the need for continuous updates and maintenance. This lack of practical consideration limits the paper's usefulness for researchers and practitioners who are interested in implementing these methods. My analysis confirms that the paper does not include any empirical evaluations of the presented methods. This is a major weakness, as it makes it difficult to assess the real-world performance of LLM-based KGC. The paper primarily focuses on describing existing methods and does not include any experiments or benchmarks to evaluate their effectiveness. This lack of empirical validation makes it difficult to compare the performance of different LLM-based KGC methods and to assess their strengths and weaknesses in different scenarios. The paper also lacks a detailed comparison of LLM-based KGC methods with traditional knowledge graph construction techniques. While the paper introduces a taxonomy of LLM-based methods, it does not provide a systematic comparison of these methods with traditional approaches. This makes it difficult to understand the relative advantages and disadvantages of using LLMs for KGC. The paper does not discuss the specific scenarios where LLM-based methods are more suitable than traditional methods, and vice versa. This lack of comparative analysis limits the paper's ability to provide practical guidance for researchers and practitioners who are considering using LLMs for KGC. Furthermore, the paper does not adequately address the issue of evaluating the quality of LLM-constructed KGs. While the paper mentions the need for evaluation, it does not delve into the specific challenges of evaluating the quality of LLM-constructed KGs, especially given the potential for hallucination and inconsistency. The paper fails to discuss the need for new evaluation metrics that go beyond simple accuracy measures and consider the faithfulness and trustworthiness of the generated knowledge. Finally, the paper's organization could be improved. While the paper follows a logical structure, the flow between sections could be smoother, with more explicit connections and transitions. The paper also lacks a dedicated section that consolidates the limitations of LLM-based KGC methods. This makes it difficult for readers to get a clear overview of the challenges associated with these methods. My analysis of the paper confirms these weaknesses, and I have high confidence in these findings, as they are directly supported by the paper's content and the absence of certain discussions.

💡 Suggestions

To address the identified weaknesses, I recommend several concrete improvements. First, the paper should include a more in-depth analysis of the practical challenges associated with using LLMs for KGC. This analysis should delve into the specific mechanisms through which problems like hallucinations and biases manifest in the context of KGC. The authors should explore how the inherent biases in LLM training data can lead to skewed or inaccurate knowledge graphs. They should also discuss the potential for LLMs to generate contradictory information and how this can undermine the consistency and reliability of the constructed KGs. Furthermore, the paper should provide a more thorough evaluation of the practical implications of using LLMs for KGC. This evaluation should include a detailed analysis of the trade-offs between accuracy, efficiency, and cost. The authors should discuss how the computational cost of LLMs scales with the size and complexity of the knowledge graphs being constructed. They should also address the practical challenges of deploying LLM-based KGC systems in real-world applications, such as the need for continuous updates and maintenance. To address the lack of empirical validation, the paper should include a more detailed discussion of evaluation metrics and benchmarks. The authors should explore the use of metrics that go beyond simple accuracy measures and consider the faithfulness and trustworthiness of the generated knowledge. They should also discuss the need for standardized benchmarks that allow for a fair comparison of different LLM-based KGC methods. This would help to establish a more rigorous and reliable way to assess the performance of these methods. The paper should also include a more detailed comparison of LLM-based KGC methods with traditional knowledge graph construction techniques. This comparison should highlight the specific scenarios where LLM-based methods are more suitable than traditional methods, and vice versa. The authors should discuss the specific advantages and disadvantages of each approach, providing a more balanced and nuanced perspective on the field. To improve the paper's organization, the authors should consider restructuring the paper to better highlight the key contributions and insights. The current structure, while comprehensive, makes it difficult to discern the core arguments and novel findings. A more effective approach would be to begin with a clear problem statement, outlining the specific challenges in KG construction that LLMs aim to address. This should be followed by a detailed discussion of the proposed taxonomy, explicitly stating the criteria for categorizing different LLM-based KGC methods. The authors should also consider adding a dedicated section that consolidates the limitations of LLM-based KGC methods. This would provide a more balanced perspective on the field and help readers to understand the challenges associated with these methods. Finally, the paper should include a more thorough discussion of the ethical implications of using LLMs for KGC. This discussion should address issues such as data privacy, bias, and the potential for misuse of the constructed knowledge graphs. By addressing these ethical concerns, the paper would provide a more complete and responsible analysis of the field.

❓ Questions

Several key questions arise from my analysis of this paper. First, given the known limitations of LLMs, such as their tendency to hallucinate and their reliance on potentially biased training data, what specific techniques can be employed to mitigate these issues in the context of KGC? I am particularly interested in understanding how techniques like fine-tuning, prompt engineering, or the use of external knowledge sources can be used to improve the reliability and accuracy of LLM-constructed KGs. Second, how can we develop more robust and reliable evaluation metrics for LLM-constructed KGs? The paper acknowledges the need for evaluation but does not delve into the specific challenges of evaluating the quality of LLM-constructed KGs, especially given the potential for hallucination and inconsistency. I am curious to know what new evaluation metrics could be developed that go beyond simple accuracy measures and consider the faithfulness and trustworthiness of the generated knowledge. Third, what are the most promising directions for future research in LLM-based KGC? The paper mentions several future directions, including KG-based reasoning, dynamic knowledge memory, and multimodal KG construction. However, I would like to hear the authors' thoughts on which of these directions they believe holds the most potential for overcoming the current limitations of LLM-based KGC. Fourth, how can we address the computational cost and scalability challenges associated with using LLMs for KGC? The paper acknowledges the computational cost of LLMs but does not provide a detailed analysis of how this cost scales with the size and complexity of the knowledge graphs being constructed. I am interested in understanding what techniques can be used to reduce the computational cost of LLM-based KGC and to make it more scalable for real-world applications. Finally, what are the ethical implications of using LLMs for KGC? The paper does not address the ethical concerns associated with using LLMs for KGC, such as data privacy, bias, and the potential for misuse of the constructed knowledge graphs. I would like to know what steps can be taken to ensure that LLM-based KGC is conducted in an ethical and responsible manner.

📊 Scores

Soundness:2.75
Presentation:3.0
Contribution:2.5
Rating: 5.5

Keywords

Click the button to extract keywords

Insights

Click the button to extract insights
Version Management
Version 1
Citation Tools

📝 Cite This Paper