Skip to main content
Springer Nature Link
Log in

Adaptive heterogeneous graph reasoning for relational understanding in interconnected systems

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Traditional graph neural networks (GNNs) have proven effective on homogeneous graph data. However, due to the increased complexity and diversity in the structure and node features of heterogeneous graphs, existing GNNs struggle to effectively capture relationships between different node types, resulting in inefficient learning. Additionally, the semantic information in heterogeneous graphs is often underutilized, limiting their overall performance. To address these challenges, we propose the heterogeneous graph attention joint network (HGAJ) framework, which leverages graph attention neural networks to jointly learn the structure and node features of heterogeneous graphs. We introduce heterogeneous structure learning by assigning attention weights to different types of relational subgraphs, enhancing the fusion and generation of heterogeneous feature subgraphs. To further improve the performance of classification tasks, we employ a bidirectional propagation method to optimize and update parameters. Moreover, we integrate the Galactica large language model (LLM) into the HGAJ framework to utilize its semantic reasoning capabilities in scientific scenarios, assisting in relational reasoning on heterogeneous graphs. We extensively evaluate our framework on the Association for Computing Machinery (ACM), Digital Bibliography & Library Project (DBLP), and Yelp datasets, demonstrating its superiority in relational reasoning on heterogeneous graph data compared to current state-of-the-art (SOTA) models. Code is available at https://github.com/AmbitYuki/HGAJ.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2
Fig. 3
Algorithm 3
Algorithm 4
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

No datasets were generated or analyzed during the current study.

Notes

  1. https://dl.acm.org/

  2. https://dblp.uni-trier.de

  3. https://github.com/AndyJZhao/HGSL/tree/main/data

References

  1. Abbasimehr H, Paki R (2022) Improving time series forecasting using lstm and attention models. J Ambient Intell Humanized Comput 13:1–19. https://doi.org/10.1007/s12652-020-02761-x

    Article  Google Scholar 

  2. Cai X, Shang J, Hao F, et al (2021) Hmsg: heterogeneous graph neural network based on metapath subgraph learning. arXiv preprint arXiv:2109.02868https://doi.org/10.48550/arXiv.2109.02868

  3. Chang Y, Chen C, Hu W et al (2022) Megnn: meta-path extracted graph neural network for heterogeneous graph representation learning. Knowl-Based Syst 235:107611. https://doi.org/10.1016/j.knosys.2021.107611

    Article  Google Scholar 

  4. Cirstea RG, Yang B, Guo C (2019) Graph attention recurrent neural networks for correlated time series forecasting. In: MileTS19@ KDD, pp 1–6, https://doi.org/10.48550/arXiv.2103.10760

  5. Cui Y, Zhang S, Liu Y et al (2021) Geometric attentional dynamic graph convolutional neural networks for point cloud analysis. Neurocomputing 432:300–310. https://doi.org/10.1016/j.neucom.2020.12.067

    Article  Google Scholar 

  6. Ding K et al (2022) Data augmentation for deep graph learning: a survey. ACM SIGKDD Explor Newslett 24(2):61–77. https://doi.org/10.1145/3575637.3575646

    Article  Google Scholar 

  7. Ding Y et al (2022) A novel time-frequency transformer based on self-attention mechanism and its application in fault diagnosis of rolling bearings. Mech Syst Signal Process 168:108616. https://doi.org/10.1016/j.ymssp.2021.108616

    Article  Google Scholar 

  8. Dong K, Zhang S (2022) Deciphering spatial domains from spatially resolved transcriptomics with an adaptive graph attention auto-encoder. Nature commun 13(1):1739. https://doi.org/10.1038/s41467-022-29439-6

    Article  Google Scholar 

  9. Dong X, Zhang Y, Pang K et al (2022) Heterogeneous graph neural networks with denoising for graph embeddings. Knowl-Based Syst 238:107899. https://doi.org/10.1016/j.knosys.2021.107899

    Article  Google Scholar 

  10. Dong Y, Chawla NV, Swami A (2017) Metapath2vec: Scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp 135–144, https://doi.org/10.1145/3097983.3098036

  11. Dong Y, Liu K, Li Y et al (2022) Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification. IEEE Trans Image Process 31:1559–1572. https://doi.org/10.1109/TIP.2022.3144017

    Article  Google Scholar 

  12. Fang U et al (2022) Gomic: multi-view image clustering via self-supervised contrastive heterogeneous graph co-learning. World Wide Web 26:1–17. https://doi.org/10.1007/s11280-022-01110-6

    Article  Google Scholar 

  13. Franceschi L, Niepert M, Pontil M, et al (2019) Learning discrete structures for graph neural networks. In: International Conference on Machine Learning, PMLR, pp 1972–1982, https://doi.org/10.48550/arXiv.1903.11960

  14. Fu X, Wang J, Mao W et al (2020) Magnn: metapath aggregated graph neural network for heterogeneous graph embedding. Proceed Web Conf. https://doi.org/10.1145/3366423.3380297

    Article  Google Scholar 

  15. Hernández A, Amigó JM (2021) Attention mechanisms and their applications to complex systems. Entropy 23(3):283. https://doi.org/10.3390/e23030283

    Article  MathSciNet  Google Scholar 

  16. Hu B, Fang Y, Shi C (2019) Adversarial learning on heterogeneous information networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM, pp 2357–2365, https://doi.org/10.1145/3292500.3330970

  17. Hu L, Cui P, Chen W, et al (2019) Heterogeneous graph attention networks for semi-supervised short text classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 1159–1169, https://doi.org/10.18653/v1/D19-1488

  18. Hu Y, Xiao F (2022) Network self attention for forecasting time series. Appl Soft Comput 124:109092. https://doi.org/10.1016/j.asoc.2022.109092

    Article  Google Scholar 

  19. Hu Z, Dong Y, Wang K et al (2020) Heterogeneous graph transformer. Proceed Web Conf 2020:276–286. https://doi.org/10.1145/3366423.3380027

    Article  Google Scholar 

  20. Jin W, Ma Y, Liu X, et al (2020) Graph structure learning for robust graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 66–74, https://doi.org/10.1145/3394486.3403049

  21. Jung J, Heo HS, Tak H, et al (2022) Aasist: Audio anti-spoofing using integrated spectro-temporal graph attention networks. In: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp 6367–6371, https://doi.org/10.1109/ICASSP43922.2022.9747766

  22. Kinderkhedia M (2019) Learning representations of graph data–a survey. arXiv preprint arXiv:1906.02989https://doi.org/10.48550/arXiv.1906.02989

  23. Li W et al (2022) Collaborative representation learning for nodes and relations via heterogeneous graph neural network. Knowl-Based Syst 255:109673. https://doi.org/10.1016/j.knosys.2022.109673

    Article  Google Scholar 

  24. Lin H, et al. (2021) Rumor detection on twitter with claim-guided hierarchical graph attention networks. arXiv preprint arXiv:2110.04522https://doi.org/10.48550/arXiv.2110.04522

  25. Liu W et al (2022) Mhsa-ec: an indoor localization algorithm fusing the multi-head self-attention mechanism and effective csi. Entropy 24(5):599. https://doi.org/10.3390/e24050599

    Article  MathSciNet  Google Scholar 

  26. Lo WW, Lim MH, Ong KL, et al (2022) E-graphsage: A graph neural network based intrusion detection system for iot. In: NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium, IEEE, pp 1–6, https://doi.org/10.1109/NOMS54207.2022.9789878

  27. Mo X, Huang Z, Xing Y et al (2022) Multi-agent trajectory prediction with heterogeneous edge-enhanced graph attention network. IEEE Trans Intell Transp Syst 23(7):9554–9567. https://doi.org/10.1109/TITS.2022.3146300

    Article  Google Scholar 

  28. Ng I, et al. (2022) Masked gradient-based causal structure learning. In: Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), Society for Industrial and Applied Mathematics, https://doi.org/10.1137/1.9781611977172.48

  29. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp 701–710, https://doi.org/10.1145/2623330.2623732

  30. Qi Z, Tan X, Qu C, et al (2023) SaFER: A robust and efficient framework for fine-tuning BERT-based classifier with noisy labels. In: Sitaram S, Beigman Klebanov B, Williams JD (eds) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track). Association for Computational Linguistics, Toronto, Canada, pp 390–403, https://doi.org/10.18653/v1/2023.acl-industry.38, https://aclanthology.org/2023.acl-industry.38

  31. Qian J, Qiu X, Tan X et al (2023) An attentive lstm based approach for adverse drug reactions prediction. Appl Intell 53(5):4875–4889

    Google Scholar 

  32. Scarselli F, Gori M, Tsoi AC et al (2008) The graph neural network model. IEEE Trans Neural Networks 20(1):61–80. https://doi.org/10.1109/TNN.2008.2005605

    Article  Google Scholar 

  33. Schlichtkrull M, Kipf TN, Bloem P, et al (2018) Modeling relational data with graph convolutional networks. In: The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings, Springer International Publishing, pp 593–607, https://doi.org/10.1007/978-3-319-93417-4_38

  34. Shen Y et al (2022) Remaining useful life prediction of rolling bearing based on multi-head attention embedded bi-lstm network. Measurement 202:111803. https://doi.org/10.1016/j.measurement.2022.111803

    Article  Google Scholar 

  35. Shi C, Liu J, Zhang X et al (2016) A survey of heterogeneous information network analysis. IEEE Trans Knowl Data Eng 29(1):17–37. https://doi.org/10.1109/TKDE.2016.2598561

    Article  Google Scholar 

  36. Sun F, Liu J, Wu S, et al (2019) Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 1441–1450, https://doi.org/10.1145/3357384.3357895

  37. Sun Q, Liu Y, Zhang C, et al (2022) Graph structure learning with variational information bottleneck. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 1–9, https://doi.org/10.1609/aaai.v36i4.20335

  38. Taylor R, Lample G, Baevski A, et al (2022) Galactica: A large language model for science. arXiv preprint arXiv:2211.09085https://doi.org/10.48550/arXiv.2211.09085

  39. Thanapalasingam T, Wang Y, Xu Z et al (2022) Relational graph convolutional networks: a closer look. Peer J Comp Sci 8:e1073. https://doi.org/10.7717/peerj-cs.1073

    Article  Google Scholar 

  40. Velickovic P, Fedus W, Hamilton W, et al (2019) Deep graph infomax. In: ICLR (Poster), p 4, https://doi.org/10.48550/arXiv.1809.10341

  41. Wang K, Yu Y, Huang C et al (2022) Heterogeneous graph neural network for attribute completion. Knowl-Based Syst 251:109171

    Article  Google Scholar 

  42. Wang X, Ji H, Shi C, et al (2019) Heterogeneous graph attention network. In: The World Wide Web Conference, ACM, pp 2022–2032, https://doi.org/10.1145/3308558.3313562

  43. Wang X et al (2022) A survey on heterogeneous graph embedding: methods, techniques, applications and sources. IEEE Trans Big Data 9(2):415–436. https://doi.org/10.1109/TBDATA.2022.3177455

    Article  Google Scholar 

  44. Wang Z, Liu C, Gombolay M (2022) Heterogeneous graph attention networks for scalable multi-robot scheduling with temporospatial constraints. Autonomous Robots pp 1–20. https://doi.org/10.1007/s10514-021-09997-2

  45. Xiong Z, Ji S, Wang X et al (2019) Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. J Med Chem 63(16):8749–8760. https://doi.org/10.1021/acs.jmedchem.9b00959

    Article  Google Scholar 

  46. Yang C et al (2022) Logic synthesis optimization sequence tuning using rl-based lstm and graph isomorphism network. IEEE Trans Circuits Syst II: Express Briefs 69(8):3600–3604. https://doi.org/10.1109/TCSII.2022.3168344

    Article  Google Scholar 

  47. Yang Y, Guan Z, Li J et al (2023) Interpretable and efficient heterogeneous graph convolutional network. IEEE Trans Knowl Data Eng 35(2):1637–1650. https://doi.org/10.1109/TKDE.2021.3101356

    Article  Google Scholar 

  48. Yu G, Luo Y, Deng R (2022) Automatic segmentation of golden pomfret based on fusion of multi-head self-attention and channel-attention mechanism. Comp Electron Agric 202:107369. https://doi.org/10.1016/j.compag.2022.107369

    Article  Google Scholar 

  49. Yu L, Sun L, Du B et al (2023) Heterogeneous graph representation learning with relation awareness. IEEE Trans Knowl Data Eng 35(6):5935–5947. https://doi.org/10.1109/TKDE.2022.3160208

    Article  Google Scholar 

  50. Yu P, et al (2022) Multiplex heterogeneous graph convolutional network. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, https://doi.org/10.1145/3534678.3539482

  51. Zhang C (2020) Learning from heterogeneous networks: methods and applications. In: Proceedings of the 13th International Conference on Web Search and Data Mining, ACM, pp 119–120, https://doi.org/10.1145/3336191.3372182

  52. Zhang C, Chen D, Wang Y, et al (2019) Heterogeneous graph neural network. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM, pp 793–803, https://doi.org/10.1145/3292500.3330961

  53. Zhang J, Li S, Yang B et al (2020) A novel graph attention adversarial network for predicting disease-related associations. Methods 179:81–88. https://doi.org/10.1016/j.ymeth.2020.05.010

    Article  Google Scholar 

  54. Zhang M, Wang X, Zhu M, et al (2022) Robust heterogeneous graph neural networks against adversarial attacks. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 4363–4370

  55. Zhang S, Yao L, Sun Y et al (2019) Graph convolutional networks: a comprehensive review. Comput Social Netw 6(1):1–23. https://doi.org/10.1186/s40649-019-0069-y

    Article  Google Scholar 

  56. Zhang T, Guo G (2020) Graph attention lstm: a spatiotemporal approach for traffic flow forecasting. IEEE Intell Trans Syst Magazine 14(2):190–196. https://doi.org/10.1109/MITS.2020.2990165

    Article  Google Scholar 

  57. Zhao J, Zhang C, Chen D, et al (2021) Heterogeneous graph structure learning for graph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 3955–3963, https://doi.org/10.1609/aaai.v35i5.16600

  58. Zheng S, Guan D, Yuan W (2022) Semantic-aware heterogeneous information network embedding with incompatible meta-paths. World Wide Web pp 1–21. https://doi.org/10.1007/s11280-021-00903-5

  59. Zhu H, Zhang W, Zhang Y, et al (2019) Multistep flow prediction on car-sharing systems: A multi-graph convolutional neural network with attention mechanism. International Journal of Software Engineering and Knowledge Engineering 29(11n12):1727–1740. https://doi.org/10.1142/S0218194019400187

  60. Zhu Y, Xu Y, Cui H, et al (2022) Structure-enhanced heterogeneous graph contrastive learning. In: Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), Society for Industrial and Applied Mathematics, pp 82–90, https://doi.org/10.1137/1.9781611977172.10

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China, “Science and Technology Innovation Action Plan” Shanghai Natural Science Foundation (Grant No.62102241, No.23ZR1425400).

Author information

Author notes

  1. Bin Li, Haoyu Wang and Xaoyu Tan have contributed equally to this work.

Authors and Affiliations

  1. School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, China

    Bin Li, Haoyu Wang, Jue Chen & Xihe Qiu

  2. INF Technology (Shanghai) Co., Ltd., Shanghai, China

    Xaoyu Tan

  3. School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China

    Qiong Li

Authors
  1. Bin Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Haoyu Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xaoyu Tan
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Qiong Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Jue Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Xihe Qiu
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

All authors discussed the ideas. B.L. and H.W wrote the code, H.W., X.Q. and X.T. drafted the initial manuscript. H.W., X.Q. and X.T. conducted the experimental analysis. X.Q, X.T., J.C. and Q. L. reviewed all the figures. B.L. and H.W. created all the figures. All authors reviewed the manuscript.

Corresponding author

Correspondence to Xihe Qiu.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, B., Wang, H., Tan, X. et al. Adaptive heterogeneous graph reasoning for relational understanding in interconnected systems. J Supercomput 81, 112 (2025). https://doi.org/10.1007/s11227-024-06623-7

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-024-06623-7

Keywords

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy