Dynamic Variational Walk and Vertex Attributes-Driven Laplacian Space Optimization for Network Representation Learning

Forbes Richard^*
*Correspondence: Forbes Richard, Department of Information Science, University of Pennsylvania, Philadelphia, PA 19104, USA, Email:

Introduction

For mining complex object interactions, network data analysis is critical. Random walk and neural-language-model-based network representation learning (NRL) approaches have been widely used for network data analysis in recent years. These NRL approaches, however, have the following flaws: First, because the random walk procedure is based on symmetric node similarity and a fixed probability distribution, the sampled vertices' sequences may lose local community structure information; second, because the shallow neural language model's feature extraction capacity is limited, they can only extract the local structural features of networks; and third, these approaches require specially designed mechanisms for various downstream tasks to integrate vertex attributes.

Description

We conducted extensive research to address the aforementioned issues and propose a novel general NRL framework called dynamic structure and vertex attribute fusion network embedding, which first defines an asymmetric similarity and h-hop dynamic random walk strategy to guide the random walk process in walked vertex sequences in order to preserve the network's local community structure. Then, using the walked vertex sequences [1-3], we train a self-attention-based sequence prediction model to simultaneously learn the vertices' local and global structural features. Finally, we present an attributesdriven Laplacian space optimization method for bringing the structural feature extraction and attribute feature extraction processes together. The proposed approach is thoroughly evaluated using node visualisation and classification on multiple benchmark datasets and it outperforms baseline approaches. Complex inter-entity relationships, such as social networks between users, e-commerce networks between users and products, citation networks between publications, biological networks, and so on, can be effectively modelled using network structure.

These networks have been used in a variety of data mining applications, such as vertex classification, link prediction, user search, and recommendation systems. Traditional supervised matrix decomposition-based network analysis approaches bind the second and third phases tightly and use specialised algorithms for a wide range of downstream tasks and network types. However, as information technology advances, networks become more diverse, extensive, and sparse, making it impractical and time-consuming to label all vertices and perform matrix decomposition. Furthermore, approach migration is hampered by the strong link between algorithms and downstream tasks. Unsupervised network representation learning (NRL) approaches based on random walk and shallow neural language models have been extensively researched and have achieved great success in recent years.

These approaches distinguish the second and third phases by concentrating on the extraction of representative low-dimensional latent features for vertices. The resulting features can then be directly incorporated into subsequent vector-based data mining algorithms. In general, random walks on a network are used to generate vertex sequences [4,5] that contain structural information about the network, and then a neural language model is trained by modelling the co-occurrence of vertices pairs on the sequences to extract structural features for the vertices. Despite the fact that these methods have been shown to be effective, they have the three flaws listed below: First, because the random walk is a Markov process based on a static probability distribution, the sampled vertex sequences are prone to losing vertex neighbourhood structure information; second, because the shallow language model's feature extraction capability is limited, only local structural features can be learned, while global structural features are ignored; and third, in order to incorporate the vertex attribute information.

Conclusion

To address the aforementioned issues, we propose dynamic structure and vertex attributes fusion network embedding, a general NRL approach (dSAFNE). First, we design an h-hop weighted dynamic random walk strategy based on a newly defined asymmetric second-order vertex approximation, which incorporates a series of previously walked vertices to dynamically calculate the sampling probability of each vertex and assign a higher walk probability to the more similar ones. A self-attention-based sequence prediction model is then applied to the sampled vertex sequences. A fake task that predicts the next walked vertex based on the previous vertex sequence can be used to learn vertex representations that preserve both local and global structural characteristics.

References

1. Vrančić, Christian, Anna Fomichova and Norbert Gretz. "Continuous glucose monitoring by means of mid-infrared transmission laser spectroscopy in vitro." Analyst 136 (2011): 1192-1198.

Google Scholar, Crossref, Indexed at

2. Kasahara, Ryosuke, Saiko Kino and Shunsuke Soyama. "Noninvasive glucose monitoring using mid-infrared absorption spectroscopy based on a few wavenumbers." Biomed Opt Express 9 (2018): 289-302.

Google Scholar, Crossref, Indexed at

3. Jintao, Xue, Ye Liming and Liu Yufei. "Noninvasive and fast measurement of blood glucose in vivo by near infrared (NIR) spectroscopy." Spectrochim Acta A Mol Biomol Spectrosc 179 (2017): 250-254.

Google Scholar, Crossref, Indexed at

4. Enejder, Annika M.K., Thomas G. Scecina and Jeankun Oh, et al. "Raman spectroscopy for noninvasive glucose measurements." J Biomed Opt 10 (2005): 031114.

Google Scholar, Crossref, Indexed at

5. Pandey, Rishikesh, Santosh Kumar Paidi and Tulio A. Valdez. "Noninvasive monitoring of blood glucose with Raman spectroscopy." Acc Chem Res 50 (2017): 264-272.

Google Scholar, Crossref, Indexed at