¹

1 Introduction↩︎

Knowledge Graphs (KGs) encode human knowledge as triplets \((s, r, o)\), representing subject, relation, and object. Because real-world facts evolve over time, Temporal Knowledge Graphs (TKGs) augment each triplet with a timestamp, yielding \((s, r, o, t)\). The central challenge in TKG reasoning is extrapolation, namely reasoning beyond the observed time horizon. We focus on two extrapolative tasks that jointly characterize a future event: predicting what will happen, known as future link prediction \((s, r, ?, t)\), and predicting when it will happen, known as occurrence-time estimation \((s, r, o, ?)\). The latter task remains much less explored, despite its importance for downstream systems that must act on forecasts.

1.0.0.1 Related work.

Existing TKG reasoning methods fall into four broad paradigms. Quad-based methods such as TTransE [1], ChronoR [2], and ATiSE [3] inject time directly into static entity and relation embeddings. Graph-based methods, including RE-NET [4], and CENET [5], aggregate structural evolution across temporal snapshots. Path-based methods such as TLogic [6] and CluSTeR [7] explicitly construct multi-hop reasoning chains. Transformer-based architectures, including SToKE [8] and ECEformer [9], leverage self-attention to capture contextual dependencies. All four paradigms achieve strong performance, yet they share a common limitation: they operate on discrete time steps or static snapshots, oversimplifying the continuous, stochastic nature of real-world events.

Temporal Point Processes (TPPs), and in particular the self-exciting Hawkes process [10]–[12], offer a principled probabilistic alternative. Deep extensions of TPPs [13]–[17] have shown strong results on generic event streams, and pioneering work, such as Know-Evolve [18] GHNN [19] and the Transformer-integrated GHT [20], first imported continuous point processes into TKG representation learning. More recent models, such as HTCCN [21] and THCN [22], adapt Hawkes processes to TKG extrapolation through temporal causal convolutions.

Despite this progress, three challenges remain unresolved across both discrete and point-process paradigms: (i) Insufficient temporal dynamics. Snapshot-based models ignore the fluid nature of time and struggle to capture the lingering excitation of distant history. (ii) Neglected mutual excitation. Most frameworks model entities in isolation and cannot express how the occurrence of one event chain triggers or inhibits another. (iii) Deterministic, miscalibrated time prediction. Time estimation has received far less attention than link prediction, and the methods that do exist rely on mean-based regression that is unreliable under heavy-tailed inter-arrival distributions.

1.0.0.2 Our contribution.

We propose the Group Attention Neural Hawkes Process (GAttNHP), a unified framework that addresses these three challenges with three matched components.

(a) Continuous-time self-excitation via attentional point processes. We cast each subject–relation event chain as a marked TPP, and encode its history with a self-attention mechanism. This bridges discrete snapshots and continuous time, capturing long-range dependencies and dynamically simulating the evolution of the intensity function. (Addresses challenge (i).)

(b) Cross-chain mutual excitation via semantic soft grouping. We dynamically infer a soft group membership for each chain, and translate a globally learnable, group-level Hawkes prior \((\Phi,\gamma)\) into an analytical additive cross-attention mask. Exponentiated inside the softmax, this mask contributes an exact group-Hawkes excitation–decay factor that modulates the semantic query–key similarity, so the model captures mutual excitation among chains and learns shared group-level excitation patterns without exhaustive pairwise computation. (Addresses challenge (ii).)

(c) Calibrated time prediction via Non-Crossing Quantile regression. We replace mean-based regression with an NCQ head that estimates a full set of monotonically ordered quantiles of the inter-arrival distribution. The construction guarantees no quantile crossing and remains stable under heavy tails, yielding both a robust point estimate (the median) and explicit uncertainty quantification. (Addresses challenge (iii).)

These components are jointly optimized end-to-end. Section 2 reviews the Hawkes process. Section 3 develops GAttNHP. Section 4 presents experiments on TKG benchmarks. Section 5 concludes.

2 Background: Hawkes Processes↩︎

A Hawkes process [10] is a doubly stochastic point process whose conditional intensity is \[\lambda(t) = \mu + \sum_{i:\,t_i < t} g(t - t_i), \label{eq:hawkes}\tag{1}\] where \(\mu \geq 0\) is the base intensity and \(g(\cdot)\) is a non-negative, decaying kernel (e.g., exponential or power-law). Each past event lifts the instantaneous likelihood of future events, and that influence decays with time.

A multivariate Hawkes process generalizes Eq. 1 to \(K\) event types: \[\lambda_k(t) = \mu_k + \sum_{j=1}^{K} \sum_{t_{j\ell} \in \mathcal{H}_{j,t}} g_{kj}(t - t_{j\ell}), \qquad k = 1, \ldots, K, \label{eq:multivariate-hawkes}\tag{2}\] where \(\mu_k\) is the baseline intensity for type \(k\), \(\mathcal{H}_{j,t}\) is the history of type-\(j\) events before \(t\), and \(g_{kj}(\cdot)\) is the excitation kernel describing how past type-\(j\) events influence the intensity of type \(k\). A common choice is the exponential kernel \(g_{kj}(u) = \alpha_{kj} e^{-\beta_{kj} u} \mathbf{1}\{u > 0\}\), in which \(\alpha_{kj} \geq 0\) controls the excitation magnitude and \(\beta_{kj} > 0\) governs the decay rate.

The classical Hawkes formulation has two limitations relevant to TKGs. First, the kernel only allows past events to excite, never inhibit, future occurrences. Second, parametric kernels are too rigid to capture the rich, context-dependent dynamics of real event streams. Neural Hawkes processes address both limitations by parameterizing the conditional intensities with neural networks. For an event sequence on \((0, T]\), the total intensity is \[\lambda(t) = \sum_{k=1}^{K} \lambda_k(t) = \sum_{k=1}^{K} f_k\!\left(w_k^\top h(t)\right), \qquad t \in (0, T],\] where \(h(t)\) is a continuous-time hidden state summarizing event history, \(w_k\) is a learnable weight vector for type \(k\), and \(f_k(\cdot)\) is a non-negative activation. A common choice is the temperature-scaled softplus \(f_k(c) = \tau_k \log(1 + \exp(c/\tau_k))\) with \(\tau_k > 0\). We adopt this neural formulation as the foundation of GAttNHP.

3 Methodology↩︎

This section develops GAttNHP. Section 3.1 formalizes TKG extrapolation as a marked point process. Section 3.2 then specifies the conditional intensity as the superposition of three components—base, self-excitation, and group excitation—that the rest of the section operationalizes: the self-excitation term is realized in Section 3.3 by an attention-based history encoder, the group excitation term in Section 3.4 by semantic soft grouping with an analytical Hawkes mask, and the full intensity is assembled in Section 3.5. Section 3.6 then introduces the Non-Crossing Quantile head for time prediction, and Section 3.7 states the joint training objective.

3.1 Problem formulation↩︎

3.1.0.1 TKG reasoning as a marked point process.

Given an observed dataset \(\mathcal{D}= \{(s, r, o, t)\}\), we associate one event chain \(\mathcal{S}_u\) with each subject–relation pair \(u = (s, r)\). The chain consists of object–time marks \((o_i, t_i)\), and its history up to time \(t\) is \(\mathcal{H}_t^u = \{(o_i, t_i) \in \mathcal{S}_u : t_i < t\}\). We model \(\mathcal{S}_u\) as a marked TPP with conditional intensity \[\lambda(o, t \mid s, r) = \lim_{\Delta \downarrow 0}\frac{\mathbb{P}\!\left(\text{event with mark } o \text{ in } [t, t+\Delta) \mid \mathcal{H}_t^u\right)}{\Delta},\] i.e., the instantaneous rate of observing object \(o\) at time \(t\) for the chain \(u\), given the past.

3.1.0.2 Downstream tasks.

Once the intensity is learned, we consider two standard TKG reasoning tasks. (i) Entity prediction at time \(t\). Given a query \((s, r, ?, t)\), infer the missing object as \[\hat{o} = \mathop{\mathrm{arg\,max}}_{o \in \mathcal{V}} \lambda(o, t \mid s, r, \mathcal{H}_t). \label{eq:entity-prediction}\tag{3}\] (ii) Time prediction for a given fact. Given \((s, r, o, ?)\), forecast the next occurrence as \[\hat{t} = \mathbb{E}\!\left[T \mid T > t_{\text{now}},\, s, r, o, \mathcal{H}_{t_{\text{now}}}\right]. \label{eq:time-prediction}\tag{4}\]

3.2 Group neural Hawkes intensity↩︎

Following the Group Network Hawkes Process(GNHP) [23], we decompose the conditional intensity for a chain \(u = (s, r)\) as \[\lambda_u(t) = \phi\!\Big(\,\underbrace{\mu_u(t)}_{\text{base}}\;+\;\underbrace{\nu_{\text{self}}(t, \mathcal{H}^u)}_{\text{self-excitation}}\;+\;\underbrace{\nu_{\text{group}}(t, \mathcal{H}^{\text{group}})}_{\text{group-excitation}}\Big),\] where \(\phi(\cdot)\) is a non-negative activation (we use softplus) ensuring \(\lambda_u(t) > 0\). The three terms target three distinct sources of intensity: \(\mu_u\) captures the baseline popularity of chain \(u\); \(\nu_{\text{self}}\) captures the inertia of \(u\)’s own history; and \(\nu_{\text{group}}\) captures cross-chain mutual excitation. We now realize these terms in turn.

3.3 Self-excitation via attention (AttNHP)↩︎

To realize \(\nu_{\text{self}}\), we must capture long-range temporal dependencies within the history of a single chain. Following [17], we adopt an attention-based encoder that builds a history-aware temporal embedding.

3.3.0.1 Setup.

Fix a chain \(u = (s, r)\). Suppose we observe \(\Gamma - 1\) historical events on \([0, t_\Gamma)\), \(\mathcal{H}_{t_\Gamma}^u = \{(o_1, t_1), \ldots, (o_{\Gamma-1}, t_{\Gamma-1})\}\) with \(0 < t_1 < \cdots < t_{\Gamma-1} < t_\Gamma\). Given a target time \(t\) (typically \(t = t_\Gamma\)), our goal is a temporal embedding \(h_u(t) \in \mathbb{R}^D\).

3.3.0.2 Layer-wise attention updates.

We stack \(L\) attention layers and concatenate the layer-wise representations: \(h_u(t) = \mathop{\mathrm{Concat}}\big(h_u^{(0)}(t), h_u^{(1)}(t), \ldots, h_u^{(L)}(t)\big)\). For each layer \(\ell > 0\), \[h_u^{(\ell)}(t) = \underbrace{h_u^{(\ell-1)}(t)}_{\text{residual}} + \tanh\!\left(\frac{\sum_{(o_i, t_i) \in \mathcal{H}_t^u} a^{(\ell)}\!\big((o_i, t_i), t\big)\, v^{(\ell)}(o_i, t_i)}{1 + \sum_{(o_j, t_j) \in \mathcal{H}_t^u} a^{(\ell)}\!\big((o_j, t_j), t\big)}\right). \label{eq:attn-update}\tag{5}\] The attention weight between target time \(t\) and historical event \((o_i, t_i)\) is \[a^{(\ell)}\!\big((o_i, t_i), t\big) = \exp\!\left(k^{(\ell)}(o_i, t_i)^\top q^{(\ell)}(t)\,/\,\sqrt{D}\right). \label{eq:attn-weight}\tag{6}\] Here \(q^{(\ell)}(t)\) is the query for the target state, and \(k^{(\ell)}(o_i, t_i)\) and \(v^{(\ell)}(o_i, t_i)\) are the key and value for the historical event, all produced via learned linear projections \(Q^{(\ell)}, K^{(\ell)}, V^{(\ell)} \in \mathbb{R}^{D \times D}\). For any time \(\tau \in \{t, t_i\}\), \[\begin{align} q^{(\ell)}(\tau) &= Q^{(\ell)}\big(1;\, [\tau];\, h_u^{(\ell-1)}(\tau)\big), \\ k^{(\ell)}(\tau) &= K^{(\ell)}\big(1;\, [\tau];\, h_u^{(\ell-1)}(\tau)\big), \\ v^{(\ell)}(\tau) &= V^{(\ell)}\big(1;\, [\tau];\, h_u^{(\ell-1)}(\tau)\big), \end{align}\] where \([1; [\tau]; \cdot]\) concatenates a bias term, a time encoding, and the previous-layer representation.

3.3.0.3 Base case and time encoding.

As the base case, \(h^{(0)}(t) \stackrel{\text{def}}{=} [o]^{(0)}\) is the learned embedding of the object at time \(t\). Because continuous time \(t \in \mathbb{R}\) cannot be embedded directly, we use sinusoidal time encodings \([t] \in \mathbb{R}^D\): \[[t]_d = \sin\!\left(\frac{t}{m \cdot \theta^{d/D}}\right) \mathbf{1}(d \text{ even}) + \cos\!\left(\frac{t}{m \cdot \theta^{d/D}}\right) \mathbf{1} (d \text{ odd}), \label{eq:time-encoding}\tag{7}\] for \(0 \leq d < D\), with \(m\) and \(\theta\) controlling the timescale.

The output of this module is a per-event self-excitation embedding \(h_u(t)\) that summarizes the chain’s own history. It enters the full intensity in Section 3.5.

3.4 Group excitation via semantic soft grouping↩︎

We now realize \(\nu_{\text{group}}\). The challenge is to capture mutual excitation across chains without exhaustive pairwise computation. Our key idea is threefold: (i) infer a soft group assignment for each chain; (ii) learn group-level Hawkes priors; and (iii) translate those priors into an analytical attention mask, so that a single multi-head attention layer realizes the full Hawkes excitation kernel.

3.4.0.1 Semantic soft group assignment.

For chain \(u = (s, r)\), we extract static embeddings \(e_s\) and \(e_r\) of the subject and relation, and map them into a distribution over \(G\) latent groups via an MLP with a GELU activation: \[w_u = \mathop{\mathrm{Softmax}}\!\left(\frac{W_2\, \mathop{\mathrm{GELU}}(W_1 [e_s \,\|\, e_r] + b_1) + b_2}{\tau}\right), \label{eq:group-assignment}\tag{8}\] where \(w_u \in [0, 1]^G\) is the soft assignment, \(\|\) denotes concatenation, and \(\tau\) is a temperature controlling the sharpness of the distribution. For a mini-batch of \(B\) sequences, the stacked weights form \(W \in \mathbb{R}^{B \times G}\).

3.4.0.2 Group-level Hawkes priors.

We introduce two globally learnable parameters: a raw group-to-group excitation matrix \(\Phi \in \mathbb{R}^{G \times G}\) and a group-wise decay vector \(\gamma \in \mathbb{R}^G\). Here \(\Phi_{m,n}\) controls how strongly an event in group \(n\) excites future events in group \(m\), and \(\gamma_m\) governs the temporal decay rate within group \(m\).

Given the soft assignments \(W\), we instantiate the expected sequence-to-sequence excitation \(\bar{\Phi} \in \mathbb{R}^{B \times B}\) and decay \(\bar{\gamma} \in \mathbb{R}^{B}\) via \[\bar{\Phi} = W \Phi W^\top, \qquad \bar{\gamma} = W \gamma. \label{eq:expected-priors}\tag{9}\] This soft-group construction reduces the cost of cross-sequence excitation from \(O(B^2)\) raw parameters to \(O(G^2)\), while preserving the ability to express asymmetric, semantically-driven interactions between any two chains in the batch.

3.4.0.3 Hawkes-derived analytical attention mask.

Consider a query event indexed by \(q\) (occurring at \(t_q\) in chain \(u\)) and a historical key event indexed by \(k\) (occurring at \(t_k\) in chain \(v\)), with \(\Delta t_{q,k} = t_q - t_k\). We define an event-level attention mask \[M_{q,k} = \begin{cases} \log(\bar{\Phi}_{u,v}) - \bar{\gamma}_u\, \Delta t_{q,k}, & \Delta t_{q,k} > 0, \\ -\infty, & \text{otherwise.} \end{cases} \label{eq:hawkes-mask}\tag{10}\] This mask is not arbitrary: when the standard multi-head attention applies the exponential during its softmax, the mask collapses exactly into a Group Hawkes intensity kernel, \[\exp(M_{q,k}) = \bar{\Phi}_{u,v}\, \exp(-\bar{\gamma}_u\, \Delta t_{q,k}).\] Through the exponential inside the softmax, the semantic similarity (the \(q^\top k\) term) is modulated by the group-Hawkes factor \(\bar{\Phi}_{u,v}\exp(-\bar{\gamma}_u\Delta t)\) and then normalized. The aggregated \(z_u(t)\) thus combines semantic feature matching with a macro-level mutual-excitation prior.

3.4.0.4 Cross-chain feature aggregation.

With the mask defined, we aggregate features across chains. Let \(H_{\text{base}} \in \mathbb{R}^{B \times L \times D}\) stack the base context features of the current batch (constructed in Eq. 12 below by concatenating local temporal features with static semantics). To enable global inter-chain communication, we flatten the keys and values into a shared memory pool \(H_{\text{shared}} \in \mathbb{R}^{S \times D}\), where \(S = B \times L\). The refined contextual representations are \[\hat{H} = \mathop{\mathrm{LayerNorm}}\!\Big(\mathop{\mathrm{MHA}}\big(Q = H_{\text{base}},\, K = H_{\text{shared}},\, V = H_{\text{shared}},\, \text{mask} = M + P\big)\Big),\] where \(P\) is a Boolean key-padding mask filtering out invalid or padded historical events. Each event is therefore mutually excited and semantically enriched by the relevant history across the entire memory pool, strictly governed by the learned group dynamics.

3.5 Full conditional intensity↩︎

We now assemble the final intensity. Let \(z_u(t)\) denote the refined representation for chain \(u\) at time \(t\)—this is the output of the Hawkes-guided cross-attention from Section 3.4, applied on top of the self-attention encoder from Section 3.3. Concretely, \[z_u(t) = \mathop{\mathrm{LayerNorm}}\!\left(h_{\text{base}}^{(u)}(t) + \sum_{v \in B} \sum_{t_k < t} \alpha_{u,v}(t, t_k)\, v_v(t_k)\right), \label{eq:zu}\tag{11}\] where the foundational representation explicitly concatenates the micro-level history embedding \(h_u(t)\) (from Section 3.3) with the static semantic embeddings: \[h_{\text{base}}^{(u)}(t) = \big(h_u(t) \,\|\, e_s \,\|\, e_r\big), \label{eq:hbase}\tag{12}\] and the cross-attention weight \(\alpha_{u,v}\) is modulated by the Hawkes mask \(M\): \[\alpha_{u,v}(t, t_k) = \mathop{\mathrm{Softmax}}\!\left(\frac{q_u(t)^\top k_v(t_k)}{\sqrt{D}} + M_{u,v}(t, t_k)\right). \label{eq:alpha}\tag{13}\] Through the exponential inside the softmax, semantic similarity (the \(q^\top k\) term) is multiplied by the physical Hawkes kernel \(\bar{\Phi}_{u,v} \exp(-\bar{\gamma}_u \Delta t)\). The aggregated \(z_u(t)\) therefore simultaneously captures semantic feature matching and macro-level mutual excitation.

To compute the conditional intensity of observing a specific object \(o \in \mathcal{V}\), we apply a linear projection followed by softplus to ensure positivity: \[\lambda_u(o, t) = \mathop{\mathrm{Softplus}}\!\left(w_o^\top z_u(t) + b_o\right), \label{eq:lambda-final}\tag{14}\] where \(w_o\) and \(b_o \in \mathbb{R}\) are object-specific parameters: \(b_o\) acts as a learned baseline popularity, while \(z_u(t)\) provides the dynamic, history-aware drive.

3.5.0.1 Optimization objective.

We train by minimizing the negative log-likelihood (NLL) of the observed event streams. For a dataset of chains \(\mathcal{U}\), the entity-prediction loss is \[\mathcal{L}_{\text{event}} = -\sum_{u \in \mathcal{U}} \left(\sum_{(o_i, t_i) \in \mathcal{H}^u} \log \lambda_u(o_i, t_i) - \int_0^T \sum_{o \in \mathcal{V}} \lambda_u(o, t)\, dt\right). \label{eq:event-loss}\tag{15}\] We approximate the integral term using the rectangle rule, computed as the product of the time interval and the aggregated intensity at the target time step.

3.6 Non-Crossing Quantile regression for time prediction↩︎

Equations 3 and 4 require predicting when a future event will occur, not just which entity. We treat this as a distributional regression problem on the inter-arrival time \(\Delta_u = t_{\text{next}} - t_{\text{now}}\), using a Non-Crossing Quantile (NCQ) module adapted from [24].

3.6.0.1 Why quantile regression.

Inter-arrival times in TKGs are heavy-tailed: a small fraction of very long gaps drag the mean far from any typical value. Mean-squared-error (MSE) regression therefore yields biased and unstable point estimates, and provides no uncertainty information. Quantile regression sidesteps both problems by estimating the conditional CDF directly. However, naively predicting a set of quantiles independently can produce crossings—e.g., \(\hat{\Delta}(0.1) > \hat{\Delta}(0.9)\)—that violate CDF monotonicity. NCQ enforces monotonicity by construction.

3.6.0.2 Architecture: value and delta layers.

Sharing the same input context \(z_u(t)\), we use two independent dense layers—a Value Layer \(v(\cdot)\) and a Delta Layer \(d(\cdot)\)—that decouple the central tendency from the distributional spread. Given \(\Gamma\) sorted target quantiles \(\mathcal{T}= \{\tau_1, \tau_2, \ldots, \tau_\Gamma\}\), the prediction model is \[\text{NCQ}(z_u(t)) = v(z_u(t); \theta_v) \oplus d(z_u(t); \theta_\delta), \label{eq:ncq}\tag{16}\] where \(\oplus\) denotes broadcast addition, and \(\theta_v, \theta_\delta\) are learnable parameters. The Value Layer outputs a scalar \(v_t \in \mathbb{R}\) representing the central level of the predicted quantiles. The Delta Layer outputs a vector \(c_t \in \mathbb{R}^\Gamma\) that shapes the spread.

3.6.0.3 Monotonicity via positive increments.

To guarantee \(\hat{\Delta}_u(\tau_1) < \cdots < \hat{\Delta}_u(\tau_\Gamma)\), we must ensure positive gaps between adjacent quantiles. We pass the Delta Layer outputs through softplus, \[\delta_\gamma = \mathop{\mathrm{Softplus}}([c_t]_\gamma) = \log(1 + e^{[c_t]_\gamma}) > 0, \qquad \gamma = 1, \ldots, \Gamma,\] and construct the \(\gamma\)-th quantile by adding a centered cumulative sum to the central value: \[\hat{\Delta}_u(\tau_\gamma) = v_t + \left[\sum_{j=1}^{\gamma} \delta_j - \Gamma^{-1} \sum_{j=1}^{\Gamma} (\Gamma + 1 - j)\, \delta_j\right]. \label{eq:quantile-construction}\tag{17}\] The bracketed term centers the quantiles so that their average equals \(v_t\). By construction, \(\hat{\Delta}_u(\tau_\gamma) - \hat{\Delta}_u(\tau_{\gamma-1}) = \delta_\gamma > 0\), so monotonicity is guaranteed structurally rather than by penalty.

3.6.0.4 Optimization.

The module minimizes the standard average pinball loss, \[\mathcal{L}_{\text{time}} = \Gamma^{-1} \sum_{\gamma=1}^{\Gamma} \rho_{\tau_\gamma}\!\left(\Delta_{\text{true}} - \hat{\Delta}_u(\tau_\gamma)\right), \label{eq:time-loss}\tag{18}\] where \(\rho_\tau(y) = y(\tau - \mathbf{1} (y < 0))\) is the check function. At inference, we use the median quantile (\(\tau = 0.5\)) as a robust point estimate.

3.7 Joint training↩︎

We jointly optimize for what and when by combining Eqs. 15 and 18 : \[\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{event}} + \beta\, \mathcal{L}_{\text{time}}, \label{eq:total-loss}\tag{19}\] where \(\beta\) balances the two tasks. Because \(\mathcal{L}_{\text{time}}\) uses the bounded check function instead of squared error, its gradients remain well-scaled even on heavy-tailed inter-arrival distributions, yielding a stable joint signal.

We optimize the total objective \(\mathcal{L}_{\text{total}}\) using the Adam optimizer with an initial learning rate of \(10^{-3}\). Training is performed on mini-batches; each batch consists of \(B\) sampled event chains, each containing a sequence of \(L\) events. For every event in the sequence, we compute the conditional intensity \(\lambda_u(o, t)\) and quantile predictions \(\hat{y}_{\tau}\) in a single forward pass. All models are trained for a maximum of 30 epochs, and the model with the minimum validation loss is kept as the final model.

4 Experiments↩︎

We evaluate GAttNHP on three event-based TKG benchmarks in the main paper—ICEWS18 [5], ICEWS14 [5], and ICEWS05-15 [6]. Results on three additional datasets (GDELT, WIKI, YAGO) and comprehensive ablations are deferred to Appendix 6. We answer two questions: (Q1) does GAttNHP improve future link prediction over state-of-the-art baselines? (Q2) does the NCQ head deliver more accurate and better-calibrated time prediction than mean-based regression, external TKG baselines, and alternative point-process heads?

4.0.0.1 Baselines.

For entity prediction, we compare against three families of methods: (i) static KG models—TransE [25], DistMult [26], ComplEx [27]; (ii) TKG interpolation models—TTransE [1], DE-DistMult, DE-SimplE [28], TNTComplEx [29]; and (iii) TKG extrapolation models—CyGNet [30], RE-GCN [31], CEN [32], CENET [5], TLogic [6], GHT [20], ECEformer [9]. For time prediction, we compare our Non-Crossing Quantile (NCQ) head not only against an internal ablative variant of GAttNHP that replaces the NCQ with a standard mean-squared-error (MSE) regressor, but also against state-of-the-art TKG extrapolation models (e.g., GHT [20] and GHNN [19]) and other widely-adopted continuous-time point process baselines (RQS-QF [33] and RMTPP [13]).

4.1 Temporal link prediction↩︎

Table 1 reports time-aware raw MRR and Hits@1/3/10 on the three ICEWS datasets. Across every metric and every dataset, GAttNHP substantially outperforms prior methods. On ICEWS14 in particular, GAttNHP attains an MRR of 0.5068 (\(\pm\) 0.0064), an absolute improvement of 8.25 points over the previous best, ECEformer (0.4243); its Hits@1 of 0.4119 (\(\pm\) 0.0056) likewise outpaces the runner-up TLogic (0.3186) by nearly ten points. Similar gains hold on ICEWS18 and ICEWS05-15. These results directly validate our two architectural claims: the self-attention encoder captures continuous-time history that snapshot-based baselines miss, and the semantic-soft-grouping mechanism adds cross-chain mutual excitation that no prior TKG model expresses.

3pt

4.2 Time prediction↩︎

Time-prediction evaluation proceeds along two complementary axes. External TKG baselines. We first compare GAttNHP-NCQ against two external TKG occurrence-time methods, GHT [20] and GHNN [19], together with an internal mean-based variant (GAttNHP-MSE). As shown in Table 2 (MAE in days; we use the median \(\tau=0.5\) as the point estimate), GAttNHP-NCQ attains the lowest MAE on all three benchmarks, improving over GAttNHP-MSE by \(47\%\)/\(50\%\)/\(70\%\) and over the stronger external baseline GHT by \(55\%\)/\(73\%\)/\(29\%\) on ICEWS14/ICEWS18/ICEWS05-15. The advantage over MSE widens on the more volatile datasets: heavy-tailed inter-arrival times amplify the squared-error penalty into very large gradients on outliers that destabilize training, whereas NCQ’s bounded pinball loss avoids this pathology. Decoder-head comparison. To isolate the contribution of the NCQ head itself, we keep the GAttNHP encoder fixed and replace only the time-prediction decoder, comparing NCQ against the rational-quadratic-spline quantile head RQS-QF [33] and the RMTPP head [13]. Since all variants share the identical embedding learning, data, and protocol, any difference reflects the decoder head alone. Table 3 shows that NCQ again attains the lowest MAE and the best-calibrated intervals across datasets: its empirical \(90\%\)-interval coverage stays close to nominal (\(0.85\)–\(0.90\)), whereas RQS-QF is severely under-covered (\(0.21\)–\(0.58\)) with far larger interval scores, and RMTPP lies in between. We further emphasize that the MSE variant produces no distributional output at all, underscoring the value of the quantile formulation.

4pt

5 Conclusion↩︎

We introduced GAttNHP, a unified framework for extrapolative reasoning on temporal knowledge graphs that bridges structural representation learning with continuous-time point processes. Built on a self-attention neural Hawkes encoder, GAttNHP captures long-range dependencies within each event chain. Its central contribution is the macro-level group interaction module, which—through a semantic soft-grouping strategy that translates globally learnable Hawkes priors into an analytical attention mask—lets event chains share excitation patterns through their latent group memberships rather than through pairwise interaction. A Non-Crossing Quantile head completes the picture, providing calibrated, monotonically ordered time predictions that remain stable under heavy-tailed inter-arrival distributions and, as a side benefit, supply cleaner gradients for the joint objective. Across six benchmarks, GAttNHP achieves state-of-the-art results, with its largest improvements concentrated on the long-tail event chains where existing TKG methods fail most. Future work will explore adaptive group structures and extend the point-process formulation to multi-hop reasoning.

6 Appendix↩︎

The appendix is preserved from the original manuscript: Datasets and Evaluation (dataset statistics, evaluation metrics), experimental Setup (Implementation Details, Hyperparameter Search and Tuning Budget, Baseline Reproduction, Efficiency and scalability), additional results on GDELT/WIKI/YAGO, ablation studies (component analysis, sensitivity to the number of latent groups \(G\)), the frequency-aware analysis demonstrating that the group-interaction module’s largest gains arise on long-tail event chains, and the qualitative case study on ICEWS14.

6.1 Datasets and Evaluation↩︎

6.1.1 Dataset Statistics↩︎

We evaluate our proposed method on six benchmark datasets, including four event-centric datasets (ICEWS14, ICEWS05-15, ICEWS18, GDELT) and two knowledge-centric datasets (WIKI, YAGO). The detailed statistics for each dataset are summarized in Table 4.

6.1.2 Evaluation Metrics↩︎

We evaluate over all test events \(i=1,\dots,N\). For entity prediction, \(\mathrm{rank}_i\) is the rank of the ground-truth object; for time prediction, \(\Delta_i\) is the true inter-arrival time and \(q_i(\alpha)\) the predicted \(\alpha\)-quantile, with the median \(q_i(0.5)\) as the point estimate.

6.1.2.1 Entity Prediction Metrics.

We evaluate entity prediction performance using two standard metrics: Mean Reciprocal Rank (MRR) and Hits@k. Let \(\mathcal{S}\) denote the set of query samples (e.g., the test set). For the \(j\)-th sample, let \(\mathcal{Z}_j\) represent the set of ground-truth entities, and \(|\mathcal{Z}_j|\) be the number of true labels. We denote the rank of a candidate entity \(q\) in the prediction list as \(\text{rank}_q\).

Hits@k measures the proportion of instances where the true entity appears among the top \(k\) ranked candidates: \[\text{Hits@k} = \frac{1}{\sum_{j \in \mathcal{S}} |\mathcal{Z}_j|} \sum_{j \in \mathcal{S}} \sum_{q \in \mathcal{Z}_j} \mathbb{I}(\text{rank}_q \leq k),\] where \(\mathbb{I}(\cdot)\) is the indicator function.

Mean Reciprocal Rank (MRR) represents the average of the reciprocal ranks of the inferred true entities: \[\text{MRR} = \frac{1}{\sum_{j \in \mathcal{S}} |\mathcal{Z}_j|} \sum_{j \in \mathcal{S}} \sum_{q \in \mathcal{Z}_j} \frac{1}{\text{rank}_q}.\]

6.1.2.2 Time prediction metrics.

For point accuracy we use the Mean Absolute Error and the Symmetric MAPE: \[\mathrm{MAE}=\frac{1}{N}\sum_{i=1}^{N}\big|\Delta_i-q_i(0.5)\big|, \qquad \mathrm{SMAPE}=\frac{1}{N}\sum_{i=1}^{N} \frac{2\,|\Delta_i-q_i(0.5)|}{|\Delta_i|+|q_i(0.5)|}.\] To assess the full predictive distribution we additionally report quantile-based scores. The quantile (pinball) score at level \(\alpha\) and its mean over the evaluated levels \(\mathcal{A}=\{0.05,0.25,0.5,0.75,0.95\}\) are \[\mathrm{QS}(\alpha)=\frac{1}{N}\sum_{i=1}^{N} 2\big(\mathbf{1}\{\Delta_i\le q_i(\alpha)\}-\alpha\big)\big(q_i(\alpha)-\Delta_i\big), \qquad \mathrm{QSm}=\frac{1}{|\mathcal{A}|}\sum_{\alpha\in\mathcal{A}}\mathrm{QS}(\alpha).\] We report \(\mathrm{QS}(0.95)\) as a tail score; note that \(\mathrm{QS}(0.5)\) coincides with the MAE and is therefore omitted. Calibration is measured by the Mean Absolute Calibration Error \[\mathrm{MACE}=\frac{1}{|\mathcal{A}|}\sum_{\alpha\in\mathcal{A}} \Big|\,\alpha-\frac{1}{N}\sum_{i=1}^{N}\mathbf{1}\{\Delta_i\le q_i(\alpha)\}\,\Big|.\] For a central \((1-\rho)\times 100\%\) prediction interval \([\,l_i,u_i\,]=[\,q_i(\rho/2),\,q_i(1-\rho/2)\,]\), we report the empirical coverage and the interval score \[\begin{align} \mathrm{Cov@}p &=\frac{1}{N}\sum_{i=1}^{N}\mathbf{1}\{\,l_i\le \Delta_i\le u_i\,\}, \qquad p=(1-\rho)\times 100\%, \\ \mathrm{IS} &=\frac{1}{N}\sum_{i=1}^{N}\Big[(u_i-l_i) +\tfrac{2}{\rho}(l_i-\Delta_i)\mathbf{1}\{\Delta_i<l_i\} +\tfrac{2}{\rho}(\Delta_i-u_i)\mathbf{1}\{\Delta_i>u_i\}\Big]. \end{align}\] We use the \(50\%\) interval (\(\rho=0.5\), i.e.\([q_i(0.25),q_i(0.75)]\)) and the \(90\%\) interval (\(\rho=0.1\), i.e.\([q_i(0.05),q_i(0.95)]\)), reported as Cov@50/Cov@90 and IS50/IS90. For all metrics except coverage, lower is better; for coverage, values close to the nominal level (\(0.50\), \(0.90\%\)) are better.

6.2 sec:Experimental32Setup↩︎

6.2.1 Implementation Details↩︎

All models are implemented in PyTorch and optimized with Adam at a learning rate of \(10^{-3}\). Each dataset is split chronologically into training / validation / test sets (\(80\%/10\%/10\%\)). Unless otherwise stated, GAttNHP uses a hidden size of \(64\), a \(16\)-dimensional time embedding, \(2\) attention layers with \(4\) heads, dropout \(0.1\), \(G=4\) latent groups, a batch size of \(16\), and is trained for \(30\) epochs. The time-loss weight \(\beta\) is set to \(0.05\) for the denser ICEWS14, ICEWS05-15, ICEWS18,WIKI and YAGO, and to \(0.001\) for GDELT. We keep the checkpoint with the lowest training loss and report test-set metrics. For reproducibility, we fix all random seeds and enable deterministic CuDNN / algorithm settings. All GAttNHP experiments run on a single NVIDIA RTX 4080 (16 GB); the largest baseline (ECEformer) requires a 48 GB GPU.

6.2.2 Hyperparameter Search and Tuning Budget↩︎

We grid-search the batch size over \(\{4,8,16,32\}\), the maximum number of epochs over \(\{30,50\}\), the number of latent groups \(G\) over \(\{1,2,4,8,16\}\), and the time-loss weight \(\beta\) over \(\{1,0.5,0.1,0.05,0.01,0.001\}\); the remaining architectural settings (learning rate, hidden size, time-embedding size, number of layers and heads, dropout) are fixed. Every configuration is selected by validation performance. The selected setting (\(G=4\), batch size \(16\), \(30\) epochs) is used in all main experiments.

6.2.3 Baseline Reproduction↩︎

For a fair comparison, results for the entity-prediction baselines [TransE [25], DistMult [26], TTransE [1]², DE-DistMult, DESimplE [28]³,ComplEx [27],TNTComplEx [29]⁴; CyGNet [30]⁵, RE-GCN [31]⁶, CEN [32]⁷, CENET [5]⁸, TLogic [6]⁹, GHT [20]¹⁰, ECEformer [9]¹¹] are re-run with their official code on our chronological splits. For occurrence-time prediction, the external baselines GHT [20] and GHNN [19]¹² are re-run; GAttNHP-MSE is an internal variant that shares our encoder and only replaces the NCQ head with a mean-squared-error regressor. The additional time-prediction heads (RQSQF [33]¹³, RMTPP, Exponential) are integrated on top of the identical GAttNHP encoder and trained with their own native objectives, so that observed differences reflect the time-prediction head alone.

6.2.3.1 Efficiency and scalability.

Table 5 reports the trainable parameter count, peak GPU memory, and per-epoch training / inference time of GAttNHP across all six benchmarks, together with the parameter count of the strongest baseline, ECEformer. GAttNHP is markedly lightweight:it uses \(2.8\)–\(8.5\)M parameters, i.e.roughly \(12\)–\(35\times\) fewer than ECEformer’s \(88\)–\(107\)M, and its peak memory stays at \(2.5\)–\(3.1\) GB throughout, so every experiment fits on a single 16 GB GPU (RTX 4080), whereas ECEformer requires a 48 GB GPU. Notably, the memory footprint remains essentially flat even on GDELT (over \(1.7\)M training facts), because cross-chain excitation is pooled within each mini-batch rather than over the entire graph. Training time scales gracefully with dataset size, from \(6.8\) s/epoch on ICEWS14 to \(172\) s/epoch on the much larger GDELT, with inference taking a few seconds at most. These savings are a direct consequence of the \(\mathcal{O}(G^2)\) semantic soft-grouping, which replaces the \(\mathcal{O}(B^2)\)/\(\mathcal{O}(N^2)\) pairwise excitation that a naive cross-chain Hawkes model would require—making GAttNHP both accurate and practical to train under modest hardware.

6.3 Additional Results↩︎

To comprehensively evaluate the generalization and robustness of our proposed method, we present additional temporal link prediction results across three widely-used benchmark datasets: GDELT, WIKI, and YAGO. We compare GAttNHP against a diverse set of strong baselines, ranging from traditional static knowledge graph embeddings to recent state-of-the-art continuous-time point process models. The overall evaluation results are summarized in Table 6.

6.3.0.1 Performance on Large-Scale and Noisy Data (GDELT & WIKI)

GAttNHP exhibits exceptional robustness on large-scale and noisy datasets. On GDELT, it achieves an MRR of 0.2643, outperforming the second-best method, Eceformer, by a remarkable margin of 6.25 percentage points. Likewise, on WIKI, GAttNHP establishes a new state-of-the-art performance with an MRR of 0.5128 and a Hits@1 of 0.4445. These results strongly highlight its superior capability in capturing both complex temporal dynamics and topological evolution within densely interacting temporal knowledge graphs.

6.3.0.2 Performance on Knowledge-Centric Data (YAGO)

On the YAGO dataset, GAttNHP remains highly competitive, achieving an MRR of 0.5130. While it significantly outperforms strong baselines like RE-GCN and CyGNet, it trails slightly behind CENET. We attribute this performance gap to the intrinsic nature of the YAGO dataset: it is dominated by long-term, relatively stable “facts” (e.g., entity attributes) rather than instantaneous, event-like interactions. Since our Hawkes-process-based architecture is fundamentally designed to capture mutual excitation effects and bursty dynamics, it naturally yields the most substantial gains on event-driven datasets (e.g., ICEWS and GDELT), while offering a more modest advantage on static-fact-driven benchmarks. Nonetheless, the overall empirical results consistently demonstrate that explicitly modeling group-wise intensity and addressing quantile crossing can effectively improve predictive performance across diverse scenarios.

6.4 Ablation Study↩︎

6.4.1 Analysis of Model Components↩︎

To rigorously quantify the contribution of each component in our framework, we conduct an ablation study on ICEWS14, ICEWS18, and ICEWS05-15. We construct four model variants to isolate (i) the effect of the self-attention temporal encoder, (ii) the benefit of the proposed group-wise mutual excitation mechanism, and (iii) the impact of alternative time-prediction objectives. The results are summarized in Figure 2.

The model variants are defined as follows:

(i) Base (Self-Attn Only): This variant uses only the self-attention temporal encoder to model self-excitation from an event chain’s own history. It ignores cross-chain interactions and does not include the auxiliary time prediction task.

(ii) w/ Group Mutual Excitation: Built on the Base model, this variant adds the proposed Neural Group-wise Mutual Excitation module to explicitly capture interactions across frequency-based groups, providing an efficient approximation to full-rank mutual excitation.

(iii) Full Model (w/ Time Pred – NCQ): The complete framework, which further incorporates the auxiliary time prediction task using our proposed Non-Crossing Quantile (NCQ) objective.

(iv) Variant (w/ Time Pred – MSE): A contrastive variant that replaces NCQ with standard mean squared error (MSE) for time prediction, used to assess the necessity of the quantile-based formulation.

6.4.1.1 Effect of group-wise mutual excitation.

As illustrated in Figure 2, the group-wise mutual excitation mechanism is the primary driver of performance gains across all benchmarks. By comparing the Base (attnhp) model with the gattnhp variant, we observe a substantial leap in predictive accuracy. For instance, on ICEWS18, the MRR rises from 0.2757 to 0.3956 (+11.99 percentage points), and on ICEWS14, it increases from 0.4431 to 0.5138.

These results empirically validate that events in temporal knowledge graphs are not isolated; rather, they are governed by complex cross-chain dependencies. While standard self-attention (Base) only captures historical self-excitation within a single dyad, our proposed grouping strategy allows event chains to borrow statistical strength from their latent semantic neighbors. This macro-level modeling effectively captures the hidden interaction topologies and alleviates the sparsity issues inherent in individual sequences, leading to more robust and comprehensive temporal representations.

6.4.1.2 NCQ vs. MSE for multi-task time prediction.

We further investigate the impact of the auxiliary time prediction objective on the primary entity forecasting task. As shown in the comparison between gattnhp+time+MLPCNQ and gattnhp+time+MLPMSE, the choice of loss function is critical for stable multi-task learning.

The proposed NCQ objective maintains, and in some cases further enhances, the entity prediction performance (e.g., achieving the highest MRR of 0.3863 on ICEWS18 and 0.5122 on ICEWS05-15). This indicates that our quantile-based probabilistic modeling provides a compatible and enriching auxiliary signal that aligns well with the intensity-based event prediction.

In sharp contrast, the MSE variant triggers a catastrophic performance collapse across all datasets. For example, on ICEWS18, the MRR plummets from 0.3863 to a mere 0.2415, and on ICEWS14, it drops from 0.5118 to 0.1124. This drastic degradation confirms our theoretical intuition: because inter-event times in TKGs follow heavy-tailed distributions with extreme outliers, the squared-error penalty of MSE generates excessively large, "toxic" gradients during backpropagation. These gradients dominate the optimization process and distort the shared latent representation space, effectively destroying the model’s ability to reason about entity relationships. Consequently, our NCQ formulation is essential for achieving a successful joint optimization of "what" and "when" in TKG extrapolation.

6.4.2 Sensitivity to the number of latent groups (\(G\)).↩︎

To investigate the scalability and effectiveness of our semantic soft grouping strategy, we evaluate the model’s performance under varying numbers of latent groups \(G \in \{1, 2, 4, 8, 16\}\). As shown in Figure 3, the choice of \(G\) plays a critical role in balancing the primary entity prediction task (MRR) and the auxiliary time prediction task (MAE).

Setting \(G=1\) corresponds to a configuration where all events are assigned to a single, unified group. In this scenario, the mutual excitation matrix effectively learns a global, macro-level interaction pattern across the entire knowledge graph. While this global prior provides a robust baseline and yields respectable performance (e.g., an MRR of 0.4565 on ICEWS14), it fundamentally lacks the capacity to capture fine-grained semantic topologies. As \(G\) increases, the model explicitly disentangles diverse event patterns from the global average, enabling more precise and specialized mutual excitations.

Interestingly, we observe a crucial trade-off between the primary entity prediction task (MRR/Hits@K) and the auxiliary time prediction task (MAE). For instance, while setting \(G=2\) yields slightly higher MRR scores on some datasets, it suffers from a marked degradation in time prediction (e.g., MAE spikes significantly). Conversely, \(G=4\) strikes the best balance across all three datasets, maintaining highly competitive MRR scores while substantially reducing the time prediction error. We therefore fix \(G=4\) in all experiments.

However, excessively large values (e.g., \(G=16\)) consistently lead to a performance drop across both tasks. We attribute this to group-level sparsity: over-partitioning the semantic space dilutes the statistical strength shared among event chains, hindering the Hawkes process from learning robust excitation priors.

6.5 Why Group Interaction Matters: A Frequency-Aware Analysis↩︎

To isolate where the performance gains of our proposed group interaction module originate, we conduct a fine-grained analysis by partitioning the test set into three categories based on the training frequency of each event chain \(u=(s,r)\). Specifically, chains with training frequencies below the 90^th percentile are classified as Low-Freq (Tail), those above the 99^th percentile as High-Freq, and the remainder as Mid-Freq. We then compare our full model, GAttNHP, against its ablated variant, AttNHP, which retains the self-attention encoder but removes the cross-sequence mutual excitation entirely.

Table 7 reports the MRR and Hits@\(K\) for each frequency group. The results reveal a striking pattern: the benefits of the group interaction module are not uniformly distributed. While high-frequency chains already enjoy abundant training data and achieve reasonable performance without cross-sequence interaction, the most substantial gains consistently emerge in the Mid-Freq and Low-Freq (Tail) groups. For instance, on the tail group, GAttNHP achieves an MRR of 0.5415 and a Hits@10 of 0.6944, representing massive absolute improvements of 10.68 and 10.93 percentage points over AttNHP, respectively.

This phenomenon perfectly aligns with the intended mechanism of our semantic soft-grouping strategy. Low-frequency chains suffer from severe data sparsity, making it difficult to learn reliable temporal dynamics solely from their limited individual histories. Through the group-level mutual excitation module, these tail chains dynamically aggregate statistical strength from other chains within the same latent semantic group—specifically, chains that share similar subject-relation semantics and excitation patterns. In effect, the module acts as an implicit data augmentation mechanism. Rare events are no longer modeled in isolation; instead, they benefit from the collective dynamics of their semantic peers.

In contrast, high-frequency chains gain only marginal improvements from the group module, as their own rich histories already provide sufficient training signals for the self-attention encoder. This explains why GAttNHP’s overall MRR improvement is primarily driven by its superior predictions on mid- and tail-frequency events—precisely the long-tail entities that existing TKG models struggle with most.

6.5.0.1 Takeaway.

The group interaction module does not merely add capacity; through cross-chain group interaction, data-scarce chains borrow statistical strength from data-rich ones, which is precisely why its gains concentrate on the mid- and low-frequency chains.

6.5.1 Case Study: The Impact of Group Interaction↩︎

To qualitatively understand how the group-level mutual excitation module corrects the bias of relying solely on individual event histories, we conduct a case study on two representative queries from the ICEWS14 dataset. We compare the top-3 ranking predictions of our full model, GAttNHP, against its ablated counterpart, AttNHP (which strictly relies on self-attention over isolated event chains). Table 8 reports the predictions, with the ground-truth answers underlined.

6.5.1.1 Discussion and Insights.

The results in Table 8 perfectly illustrate the “information borrowing” capability of GAttNHP. In Q1, the query asks who NATO would consult in November 2014. AttNHP predicts Yunus Qanuni (an Afghan politician), likely overfitting to NATO’s localized historical presence in Afghanistan. However, by incorporating group-level semantics, GAttNHP successfully captures the macro-level geopolitical alignment and correctly ranks John Kerry (then US Secretary of State) at the top, even placing other relevant US figures (Barack Obama, Joe Biden) highly.

Similarly, in Q2, an armed gang in Syria uses unconventional violence. AttNHP incorrectly predicts Hezbollah as the primary target, driven by raw historical frequency in the region. In contrast, GAttNHP successfully identifies the Lebanese Military. This indicates that the group interaction module helps the model recognize the broader structural pattern of Syrian armed gangs spilling over to clash with neighboring state militaries, rather than memorizing a single, highly active entity like Hezbollah. Both cases demonstrate that macro-level group priors effectively prevent the model from being trapped in narrow historical biases.

References↩︎