Skip to content

MAKGED is the first multi-agent framework for collaborative error detection in knowledge graphs.

Notifications You must be signed in to change notification settings

kse-ElEvEn/MAKGED

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

37 Commits
 
 
 
 
 
 
 
 

Repository files navigation

MAKGED: Multi-Agent Framework for Knowledge Graph Error Detection

License: MIT arXiv GitHub Stars

image

πŸ“Œ Overview

MAKGED is the first multi-agent framework for collaborative error detection in knowledge graphs, addressing two critical challenges in KG quality management:

  1. Multi-Perspective Analysis: Overcomes single-view limitations through bidirectional subgraph analysis
  2. Transparent Decision-Making: Implements explainable error detection via structured agent discussions

Key innovations:

  • 🎯 4 Specialized Agents: Head/Tail Γ— Forward/Backward agent architecture
  • πŸ” Hybrid Embeddings: Combines GCN-based structural features with LLM semantic features
  • πŸ€– LLM-Powered Collaboration: Implements 3-round discussion protocol with tiebreaker mechanism
  • 🏭 Industrial Proven: Validated with China Mobile's KGs

🧠 Framework Architecture

Core Components

  1. Bidirectional Subgraph Construction

    • Head_Forward/Backward Subgraphs
    • Tail_Forward/Backward Subgraphs
  2. Hybrid Embedding Generator

    graph TD
      A[Subgraph Structure] --> B[3-Layer GCN]
      C[Triple Text] --> D[Llama-2 Embedding]
      B --> E[Concatenation Layer]
      D --> E
      E --> F[Unified Representation]
    
    Loading
  3. Multi-Agent Discussion Protocol

    • Phase 1: Independent Analysis
    • Phase 2: 3-Round Discussion
    • Phase 3: Summarizer Voting (for ties)

πŸ“Š Datasets

Dataset Triples Entities Relations Error Rate
FB15K 44,000 14,541 237 30.2%
WN18RR 33,134 40,943 11 30.7%

πŸ“ˆ Benchmark Results

Performance Comparison (FB15K)

Models FB15K Accuracy FB15K F1-Score FB15K Precision FB15K Recall WN18RR Accuracy WN18RR F1-Score WN18RR Precision WN18RR Recall
Embedding-Based Methods
TransE 0.6373 0.6312 0.6410 0.6531 0.3813 0.2927 0.6255 0.5083
DistMult 0.5938 0.5132 0.5261 0.5204 0.6401 0.5157 0.5965 0.5449
ComplEx 0.6268 0.4781 0.5413 0.5172 0.6414 0.4450 0.6464 0.5217
CAGED 0.6091 0.4574 0.5028 0.4552 0.6544 0.5064 0.5532 0.5013
KGTtm 0.6828 0.4078 0.6172 0.3045 0.6911 0.4487 0.6589 0.3402
PLM-based Methods
KG-BERT 0.7675 0.6280 0.7371 0.5470 0.8162 0.7222 0.8177 0.6468
StAR 0.7350 0.6017 0.6900 0.5420 0.7012 0.6100 0.6572 0.5645
CSProm-KG 0.7078 0.5509 0.6139 0.4997 0.7116 0.6025 0.6138 0.4997
Contrastive Learning-based Methods
SeSICL 0.5950 0.4600 0.5513 0.5172 0.5050 0.4073 0.4421 0.5711
CCA 0.7456 0.6810 0.7123 0.6537 0.7621 0.7134 0.7568 0.6912
LLM-based Methods
Llama2 0.7420 0.6010 0.7250 0.6851 0.7100 0.6271 0.7021 0.6344
GPT-3.5 0.7445 0.6117 0.7185 0.6555 0.7603 0.7496 0.7120 0.6260
Llama3 0.7558 0.6264 0.7357 0.7148 0.7654 0.7522 0.7185 0.6327
Our Methods
MAKGED 0.7748 0.7367 0.7686 0.7252 0.8283 0.7909 0.8832 0.7704

Industrial Case Study

image

πŸ“š Citation

If you find our paper and resource useful in your research, please consider giving a star ⭐ and citation πŸ“.

@article{li2025harnessing,
  title={Harnessing Diverse Perspectives: A Multi-Agent Framework for Enhanced Error Detection in Knowledge Graphs},
  author={Li, Yu and Huang, Yi and Qi, Guilin and Feng, Junlan and Hu, Nan and Zhai, Songlin and Xue, Haohan and Chen, Yongrui and Shen, Ruoyan and Wu, Tongtong},
  journal={arXiv preprint arXiv:2501.15791},
  year={2025}
}

πŸ“§ Contact

For technical inquiries:
Yu Li - Southeast University


⬆ Back to Top

About

MAKGED is the first multi-agent framework for collaborative error detection in knowledge graphs.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published