Semantic Node Similarity in Graphs Using Transformer-Based Embeddings

SimGAT: structure-aware graph attention network with multi-scale structural embedding

SimGAT, a structure-aware graph attention model built on SimRank-derived structural embeddings, is proposed, which computes structural similarity in the SimRank2Vec embedding space and injects it as a topological prior into the graph attention mechanism, enabling neighborhood aggregation to be jointly guided by node attributes and global structural relationships.

Constructing embeddings for textual causality via representation regularization and semantic metrics

ACKGE: knowledge graph embedding with probabilistic semantic bounds for adaptive contrastive learning

Unsupervised Graph Representation Learning with Complementary View Alignment

Unsupervised graph representation learning aims to derive meaningful node embeddings by capturing both structural and attribute information without relying on labeled data. Existing methods, such as GAEs, have demonstrated effectiveness but typically rely on message-passing mechanisms that assume homophily, leading to performance degradation on heterophilous graphs, where connected nodes exhibit dissimilar features. This homophily bias results in the loss of critical high-frequency components that are essential for identifying heterophilous patterns. To address these challenges, we propose \textsc{AlignGAE}, a novel extension of \textit{MaskGAE} that preserves the full frequency spectrum through complementary view alignment. Our framework introduces a dual-encoder architecture that separately processes structural and attribute information, incorporates node positional encoding to approximate Neighborhood Identity Distribution (NID), and employs dual reconstruction tasks for both edges and node attributes. We further propose theoretically grounded NID alignment strategies that ensure semantic consistency across views while preserving their distinct characteristics. Through comprehensive spectral analysis, we demonstrate that \textsc{AlignGAE} achieves optimal representation properties when the alignment loss converges. Extensive experiments across 12 benchmark datasets validate our approach, showing that \textsc{AlignGAE} outperforms state-of-the-art methods by up to 18.7\% on heterophilous graphs in node classification, while maintaining competitive performance on homophilous graphs. Our results establish a new paradigm for frequency-aware graph representation learning.

Hierarchy-Aware Semantic Losses for Knowledge Graph Link Prediction

Knowledge graphs are often accompanied by ontological class hierarchies that encode valuable semantic information, yet many link prediction methods either ignore such hierarchies or incorporate them indirectly through additional graph edges. Recent work introduced hierarchy-aware graph neural networks (GNNs), which use semantic losses derived from box embeddings to encourage satisfaction of subclass relationships during GNN-based representation learning. While this approach has shown promise for biological regression tasks, its effectiveness for knowledge graph link prediction has not been investigated. In this paper we evaluate hierarchy-aware semantic losses on link prediction across three benchmark datasets: AIFB, CoDEx, and BioKG. We combine graph neural network encoders with box-embedding-based semantic losses that encourage learned representations to better satisfy ontology-derived class hierarchies, and compare this approach to both standard link prediction models and models incorporating subclass relations as graph edges. Across all datasets, hierarchy-aware semantic losses significantly improve mean reciprocal rank (MRR) and consistently outperform models that incorporate hierarchy information through additional subclass edges. Relative to the baseline GNN models, MRR improved by 7.6%, 2.4%, and 15.5% on AIFB, CoDEx, and BioKG, respectively. Furthermore, semantic losses consistently outperform the alternative of augmenting the graph with subclass edges. These results are consistent with ontology-derived class hierarchies providing complementary information to graph structure, and suggest that encouraging hierarchical consistency through semantic losses is an effective and comparatively parameter-efficient mechanism for improving knowledge graph link prediction.

From Word Embeddings to Semantic Projections: Interpretability and Context in Web-Scale Semantic Analysis

Recent developments in NLP and web-scale document analysis have increasingly emphasized the importance of interpretability and contextual dependence in semantic representations. Although modern word embeddings achieve remarkable empirical performance, their semantic structure is often difficult to interpret, since meaning is encoded through latent geometric relations in high-dimensional spaces. This paper discusses an alternative conceptual framework based on explicit contextual semantic relations. Building on ideas from distributional semantics, co-occurrence analysis, and fuzzy set theory, the study revisits semantic projections and related count-based representations as interpretable directional semantic structures for semantic analysis in document corpora and web-based information environments. In this setting, several classical association measures, including PMI and related transformations, may be understood as derived from simpler conditional semantic projections. The methodology is illustrated through a comparative analysis of semantic associations related to “ChatGPT” across general web-scale data and specialized scientific repositories. Our results demonstrate that semantic projections effectively capture persistent contextual structures while remaining sensitive to corpus-specific discourse communities. The resulting perspective emphasizes interpretability, asymmetry, contextual dependence, and direct empirical meaning as central principles for semantic representation.