¹

1 Introduction↩︎

Large-scale Critical Infrastructure Networks (CINs) are characterized by physically interdependent subsystems of varying network sizes that are operated by a diverse group of utility stakeholders governed by a regulatory entity. Data-driven cyberattacks targeting key operational technology (OT) components, such as industrial control systems (ICS), of several utilities, have been shown to cause devastating cascading failures that threaten overall network stability [1]. In the United States, there have been several attempts aimed at establishing information-sharing and analysis centers (ISACs) as a means for detecting network-wide ICS attacks through secure aggregation of sensor data as well as local cyber-incident alarms [2]. However, efforts to establish regulatory compliance initiatives like ISACs have severely fallen short of expectations primarily due to the tepid response from stakeholders [3]. Privacy concerns of utilities as well as lack of trust and credibility of reported information form the core set of obstacles that threaten the feasibility of ISAC-like compliance frameworks. Therefore, in this paper, we introduce PRECISE a differentially private subsystem-level ICS attack detection framework that relies on privacy-preserving disclosures of underlying datasets by utilities. Additionally, PRECISE enables ISAC-like entities to verify compliance of reported detection outcomes with respect to the disclosed datasets, leading to increased trust and credibility in regulatory outcomes.

Enhancing the cyber resilience of interdependent CINs in order to limit disruptions from cascading impacts is a key research priority, as outlined by the Cybersecurity and Infrastructure Security Agency (CISA) [4]. In such cases, regulatory entities like ISACs play a critical role in stemming the impacts of data-driven ICS attacks by coordinating cyber-incident response [3] accompanied by timely dissemination of insights. However, the capabilities of ISACs are only as good as the quality of the alarms and the associated datasets reported from the utility stakeholders. In fact, poor quality data collected by ISACs without sufficient context impedes situational awareness, hampers decision-making, and increases false positives occasionally resulting in unnecessary system upgrades [5]. Therefore, enabling ISACs to verify that the reported alarms comply with the underlying datasets of the utilities can help pave the way for improved situational awareness and reduced false alarm rates across the entire network.

The ability to verify compliance of alarms on the basis of underlying datasets is challenging due to privacy concerns of utility stakeholders. Privacy concerns also impede real-time information sharing among CIN stakeholders [6]. It has been demonstrated that operational data from CINs can be used to identify industrial customer demands [7], reveal operational costs of strategic CIN [8], and identify systemic vulnerabilities [9]. There is also considerable trepidation among stakeholders regarding the perceived misuse of information obtained from cyber information-sharing programs [6] by governmental agencies. In some cases CIN stakeholders are also concerned about risks to their organizational reputation [10] in the event of data leaks. As a result, the need of the hour is a private, trustworthy framework for detecting data-driven ICS attacks [11] while enabling data-driven compliance verification of the reported alarms.

Conventionally, ICS frameworks capture IoT and sensor data from assets [12] across each utility. ICS frameworks utilize state-space models to characterize the operational aspect of utilities [13] and detect anomalies and deviations from steady state conditions. A majority of these anomaly detection frameworks utilize statistical hypothesis tests on state space residuals computed using sensor data [14]. As a result, alarms at the utility level are an outcome of anomaly detection frameworks that is inherently characterized by local sensor data as well as the associated covariance matrices. Statistical analysis of residuals can also yield significant insights into the nature of detected anomalies such as attack diagnosis and distinguishing attacks from routine equipment failure [14]. As a result, statistical methods are widely used as a first line of defense in identifying potential attack driven anomalies in ICS frameworks.

In this paper, we consider the interaction between the utility stakeholders and their regulatory counterparts such as ISACs. The goal of the utility is to convince an ISAC regarding alarm validity through privacy-preserving disclosures of the underlying residuals and the detection algorithm used for alarms.

Our proposed approach therefore uses differential privacy (DP) driven disclosures of data as a means to drive the transparent validation of the alarms. A schematic of our proposed approach is depicted in Figure 1. However, two major bottlenecks arise with this approach. First, statistical hypothesis tests, commonly used for data-driven ICS attack detection, require privacy-preserving disclosures of local covariance matrices to generate alarms [12]. Second, using differentially private data disclosures can itself severely affect the alarm outcomes leading to misclassification with respect to original alarm statistics [15], [16]. Consequently, ensuring compliance and transparency through data sharing introduces a high degree of statistical complexity driven by privacy constraints.

In this paper, we target both these bottlenecks by adopting a two phase DP mechanism to introduce transparency and regulatory compliance for ICS based attack detection. In the first phase, we focus on DP based disclosures of covariance matrices in order to characterize the utility-level detection framework. The second phase involves privacy-preserving disclosures of temporal state-space residual values by leveraging Gaussian differential privacy frameworks. We derive \((\epsilon,\delta)\) differential privacy guarantees on data-driven ICS attack detection frameworks. Our contributions can be summarized as follows:

• We develop an algorithmic framework for privacy-preserving hypothesis tests that are compatible with DP disclosures of covariance and state-space residuals.

• We develop two distinct implementation modes that enable regulatory bodies like ISACs to independently verify utility detection outcomes using DP disclosures of high-dimensional ICS data.

• We derive strong privacy guarantees governing residual and covariance disclosures in order to obtain DP equivalent levels of significance and test statistic distributions for the corresponding hypothesis tests.

• We theoretically characterize the impact of DP on detection quality by analyzing the DP-induced levels of significance and associated test statistics.

Our proposed framework is evaluated using a generalizable state-space modeling framework that utilizes a non-linear Kalman Filter based approach. Our experimental results are demonstrated using real-world ICS data [17] consisting of diverse attack scenarios on several heterogenous subsystems. The key takeaway of our research is that DP based disclosures offer a viable alternative for establishing regulatory compliance standards, help achieve higher degree of situational awareness, trust and credibility in CINs while providing strong privacy guarantees for utility stakeholders.

2 Related Work↩︎

Attacks like DoS, DDoS, and phishing, which specifically target IT systems, can often be effectively detected and isolated by monitoring network traffic [18]–[20]. However, data-driven attacks on ICS are specifically designed to evade such conventional defense mechanisms, including network monitoring and protocol-based intrusion detection [21], [22]. These attacks form a more significant threat to ICSs due to their ability to impact information, communication, and the underlying physical systems [23], leading to significant damage to critical infrastructure. Unlike IT-layer attacks, sensor data is manipulated in order to effect damage through malicious control actions and incorrect state estimations, resulting in degraded asset performance.

Several model-based detection mechanisms have been proposed for ICS attacks involving data manipulation [24]–[36]. A popular state-space estimation modeling framework used for ICS attack detection is the Kalman Filter based technique [12]–[14], [37], [38]. The Kalman Filter based detection algorithms rely on residuals computed on the basis of the estimated (or predicted) and observed measurements or states followed by a statistical testing procedure [39], [40]. Due to the presence of robust physics-based models in industrial IoT, the Kalman Filter based methods form a powerful class of techniques for anomaly detection. Using a Kalman Filter based state space estimation, one can conduct attack diagnosis [14], distinguish between routine faults and attacks using degradation models [37] and use decentralized methodologies to raise network-wide alarms [38]. However, detecting ICS attacks across multi-stakeholder CINs requires modeling spatiotemporal interdependencies among utilities [1], [41], introducing significant privacy concerns when operational data must be shared.

On the other hand, recent works have investigated the use of DP as a means to circumvent the privacy-related impediments to enable data sharing in CINs [9]. DP is a widely used method to protect the privacy of data sets intended to be communicated through public domains [42], [43]. DP-driven approaches involve injecting a randomized noise in order to obfuscate the real underlying data record [44], [45]. The injected randomized noise can be designed so as to facilitate theoretical guarantees bounding the loss of privacy [42]. DP thereby ensures that the probability of extracting the real value from a noisy data set by any external entity remains remarkably low. Most DP approaches applied in the context of CINs are geared toward the public release of operational data for benchmarking purposes [7], [9], [46]–[50]. The use of DP for protecting the input signals in a Kalman Filter based state space modeling framework has also received considerable attention as well [51]–[53].

Additionally, there have been several approaches that have focused on differential privacy in the context of statistical hypothesis tests [15], [54]. Such methods have typically relied on developing DP versions of popular statistical hypothesis tests such as Wilcoxon-signed test [15], goodness-of-fit tests[16], [54], [55], F-test for linear regression significance [56]. However, a majority of them target categorical datasets [16], [55] or are nonparametric in nature [15]. Therefore, while most of these methods provide strong methodological foundations, they are not geared towards temporal datasets that result from state-space modeling approaches such as Kalman Filter. As a result, there exists a critical methodological gap for DP based approaches that can specifically cater to anomaly detection for temporal state-space models.

Secure multiparty computation (MPC), homomorphic encryption (HE), and federated learning (FL) have all been explored for privacy-preserving data sharing in CINs. However, each comes with practical limitations that make them unsuitable for our setting. MPC and HE fundamentally require coordinated computation between parties, which introduces significant communication overhead that is difficult to sustain in real-time ICS monitoring environments. Privacy-preserving federated learning paradigms, while well-suited for distributed model training, offer no benefit for enabling compliance between the regulators and stakeholders in the context of anomaly detection in CINs. Moreover, federated learning paradigms are unable to provide provable worst-case guarantees on the privacy of individual data when considering regulatory disclosures. Differential privacy, on the other hand, provides mathematically rigorous bounds on privacy loss, operates without requiring coordination between utilities, and is computationally lightweight enough to handle sequential temporal data — making it the most appropriate choice for privacy-preserving disclosures of state-space residuals and covariance matrices in the regulatory compliance setting considered in this paper.

3 State Space Modeling for ICS↩︎

Operational modeling of utility stakeholder level ICS is the preliminary step for building a robust detection framework that can be ultimately leveraged for regulatory compliance. In that regard, an exceptional operational model must possess two critical estimation capabilities. First, operational models of utility ICS must incorporate transition functions that can help estimate the future state of the system based on existing sensor data. Second, such models must also be capable of yielding stable and accurate estimations of process and sensor noise distributions which are critical in helping make future state estimations more robust. The core idea is that an exceptional operational model can be utilized to statistically distinguish between normal and anomalous ICS behavior enabling accurate local attack detection which will in turn drive regulatory compliance. For ease of reference, Table 1 provides a summary of the key notation used throughout this paper.

3.1 Non-Linear State Space Formulation↩︎

For instituting a local ICS attack detection framework, we begin by discussing a generalizable state-space modeling framework for characterizing utility level ICS operations that are non-linear in nature. Our generalizable state-space model considers a sensor-driven non-linear system at time \(t\), where \(x_t\in \mathbb{R}^{m}\) represents the latent space embedding, \(u_t\in \mathbb{R}^{m}\) represents the control action and \(y_t\in \mathbb{R}^{d}\) represents noisy sensor measurements from asset sensors. \[\begin{gather} x_{t+1}= g(x_{t-1},u_{t-1}) + v_t,\tag{1}\\ y_t=h(x_t)+w_t, \tag{2} \end{gather}\] In Equations 1 , 2 , \(\boldsymbol{g}, \boldsymbol{h}\) are the state transition and observation functions respectively. Collectively, \(\boldsymbol{g}, \boldsymbol{h}\) represent the relationship between the measurements \(y_t\) and the state \(x_t\). The process and measurement noises at time \(t\) are denoted by \(v_t\in \mathbb{R}^{m}\), and \(w_t\in \mathbb{R}^{m}\) respectively. The process and measurement noises follow multivariate normal distributions with zero mean implying that \(v_t \sim N(0,Q_t), w_t \sim N(0,R_t)\), where \(Q_t,R_t\) represent the covariance matrices respectively. Such a type of modeling framework has also been used extensively in prior work [26], [30], [57]. A Non-Linear Kalman Filter can be used to model the state space of the stakeholder ICS as represented by Equations 1 and 2 .

3.2 Non-linear Kalman Filter Estimation↩︎

In the non-linear model given in Equations 1 , 2 , an extended Kalman Filer based model denoted by \(\mathcal{K}\) can be formulated using the following equations. \[\begin{gather} \hat{x}_{t|t-1}=g(\hat{x}_{t-1|t-1},u_{t-1}) \tag{3},\\ r_t=y_t-h(\hat{x}_{t|t-1}) \tag{4},\\ P_{t|t-1} = G_tP_{t-1|t-1}G^T_t + Q_{t-1} \tag{5},\\ S_t = H_tP_{t|t-1}H^T_t+R_t \tag{6},\\ K_t = P_{t|t-1}H^{T}_{t}S_t^{-1} \tag{7}\\ \hat{x}_{t|t}=\hat{x}_{t|t-1}+K_tr_t \tag{8}\\ P_{t|t} = (I - K_tH_t)P_{t|t-1} \tag{9} \end{gather}\] In Equations 5 ,7 , \(P_{t|t-1},P_{t|t}\) represent the predicted and the updated covariance estimates respectively, while \(S_t\) represents the residual covariance at \(t\). The state transition and the observation matrices at \(t\) given by \(G_t, H_t\) respectively are computed using the Jacobians of \(g,h\) as denoted by the following equations. \[\begin{gather} G_t = \left. \frac{\partial g}{\partial x} \right|_{x_{t-1|t-1},u_t}\text{ and } H_t = \left. \frac{\partial h}{\partial x} \right|_{x_{t|t-1}} \end{gather}\]

3.3 Learning the Non-Linear Kalman Filter↩︎

The stakeholder level ICS system can be modeled as the Non-Linear Kalman Filter (NLKF) described in Section 3.2. The NLKF model requires a generalizable framework that is known a priori and can characterize the stakeholder level ICS system dynamics including the estimation of process and measurement noise covariance estimation based on current and historic sensor measurements and state estimates.

However, constructing a generalizable framework that efficiently captures ICS system dynamics is challenging due to the tedious nature of estimating and fitting parametrized probability distributions to process and measurement noise with covariance \(Q_t,R_t\) respectively. Without accurate knowledge of these covariance matrices, the Kalman gain cannot be computed precisely resulting in erroneous posteriori state estimates predictions which can compromise the detection quality for data driven attacks.

Therefore, we utilize a machine learning framework comprising of Long and Short Term Memory (LSTM) based recurrent neural networks complemented by feed-forward layers that attempt to cast the ICS process towards a Non-Linear Kalman Filter based setting. More precisely, the Non-Linear Kalman Filter based LSTM (NLKF) design involves the posteriori state estimates of the prior time step \(\hat{x}_{t|t}\) in addition to the observed sensor measurements \(y_t\) to predict the process and measurement noise covariance \(Q_t,R_t\).

Figure 2 depicts the NLKF framework that is used for learning the state space model. The NLKF framework consists of three LSTM (\(LSTM_g\), \(LSTM_Q\) and \(LSTM_R\)) and one feed-forward Linear layers (\(Linear_h\)) that are collectively used to learn the non-linear interdependencies between the sensor measurements \(y_t\), and the priori and posteriori state estimates \(\hat{x}_{t|t-1}\) and \(\hat{x}_{t|t}\). Specifically, \(LSTM_g\) learns the relationship between the current state estimate \(\hat{x}_{t|t}\) and the a priori state estimate \(\hat{x}_{t|t+1}\). \(LSTM_Q\) and \(LSTM_R\) predict the state and measurement covariance matrices based on \(\hat{x}_{t|t}\) and \(y_t\) respectively. We use \(Linear_h\) to map the lower dimensional a priori state estimates to the high dimensional sensor data space in order to predict \(y^{pred}_t\) such that residual \(r_t = y_t - y^{pred}_t\) can be computed. Using the autograd functionality in machine learning frameworks like PyTorch, we compute the Jacobians \(H_t\) and \(G_t\) for the \(LSTM_g\) and \(Linear_h\) respectively. Finally, the estimated Jacobians, sensor noise, state noise and state covariance matrices enable us to calculate the extended Kalman gain matrix \(K_t\) and the posterior state covariance \(P_{t+1|t+1}\) according to Equations 5 7 . Therefore, the NLKF framework provides us with the fundamental ability to generate residual \(r_t\) estimates along with temporally sound predictions of the covariance matrices \(Q_t,R_t,S_t\) as well.

3.4 Hypothesis Tests for Anomaly Detection↩︎

Estimation of covariance matrices enables us to implement statistical hypothesis tests on the corresponding covariates themselves. As a result, we can leverage the NLKF framework, for orchestrating the \(\chi^2\) hypothesis tests, that can detect covert attacks [12] using \(S_t\) and \(r_t\). Additionally, the same methodology can also be used for detecting false data injection attacks using sensor data measurements \(y_t\) and covariance \(R_t\)[13].

To establish the theoretical underpinnings of the \(\chi^2\) hypothesis test, we consider without any loss of generality, the predicted residual covariance matrix \(S_t\) and the residual \(r_t\) at time \(t\). \(S_t\) is symmetric, which means that \(S_t = V_t \Lambda_t V_t^{T}\), where \(V_t\) describes the set of orthonormal eigenvectors and \(\Lambda_t\) is a diagonal matrix representing a set of eigenvalues. As a result, we can obtain the principal component decomposition of \(S_t\). We know that \((S_t)^{-1/2} = (V\Lambda_t^{-1}V_t^{T})^{1/2} = V_t\Lambda_t^{-1/2}\). Given a residual vector \(r_t\sim N(0,S_t)\), we can realize \(\tau_t = (S_t)^{-1/2}(r_t) = S_t\Lambda_t^{-1/2}(r_t)\) such that \(\tau_t \sim N(0,I)\) [12], [58]. Therefore, to detect attacks, the utility stakeholder level \(\chi^2\)-hypothesis test can be formulated as follows: \[\begin{gather} H_0 : S^r_t = S_t \tag{10}\\ H_A : S^r_t \neq S^{true}_t \tag{11} \end{gather}\] The null hypothesis \(H_0\) represents the condition that the covariance of the reported residual vector \(r_t\) denoted by \(S^r_t\) is equal to the expected covariance \(S_t\) predicted by the NLKF framework. In other words, the null hypothesis tests whether the residual vector \(r_t\) indeed follows the parametrized distribution \(N(0,S_t)\). The underlying insight behind \(H_0\) is that perturbations due to abnormal sensor readings or faulty computation of state estimates, will result in residual vectors that adhere to a covariance matrix that is statistically different from the one predicted by the NLKF framework. Further, we can also state that if \(H_0\) holds, then the standardized vector of principal component (PC) scores is given by \(\tau_t\) implies that \(||\tau_t||^2_2 \sim \chi^2_p\), where \(p<d\) is the degrees of freedom corresponding to the number of principal components used [58]. \[\rho = \begin{cases} 1, & \text{if } T_{\chi^2,t} = \|\tau_t\|^2_2 > \chi^2_{m,\alpha}, \\ 0, & \text{otherwise.} \end{cases} \label{eq:alarm95condition}\tag{12}\] Based on Equation 12 , an alarm \(\rho=1\) is triggered when the test statistic \(T_{\chi^2,t} = ||\tau_t||^2_2 > \chi^2_{m,\alpha}\), where \(\alpha\) is the level of significance, while \(\rho=0\) otherwise [13].

4 Privacy Preserving Detection Scheme↩︎

Detecting attacks requires continuous monitoring of the residual vector. However, the alarms might need to be publicly validated by a third party (such as a nodal authority or overseer agency) due to regulatory and compliance reasons. As a result, residual vectors and the covariance matrices need to be disclosed to facilitate public validation of alarm values so as to meet regulatory compliance. However, a critical challenge arises in the inability to provide strong guarantees regarding the privacy loss stemming from the public disclosure of these data.

4.1 Privacy Scope and Objectives↩︎

We focus on a utility based subsystem and its associated state-space model as detailed in Section 3. Our privacy scope particularly focuses on the disclosure of \(S_t\) and whitened residual \(\tau_t\) resulting from the NLKF framework as a means to validate the alarm value \(\rho_t\). In practice, regulatory compliance for CIN utilities is operationalized through ISAC-driven information sharing frameworks [2], [3], wherein utilities are required to disclose detection outcomes and supporting data to regulatory bodies [11]. However, privacy concerns [6] and lack of trust in disclosed information [3], [5] have severely hampered compliance efforts. PRECISE directly addresses these impediments by enabling utilities to satisfy disclosure requirements through differentially private disclosures of \(S_t\) and \(\tau_t\). As a result, we characterize our threat model and related assumptions as discussed below.

4.2 Threat Model↩︎

We consider two classes of adversaries relevant to our framework and discuss the relevant trust and security implications of our underlying privacy-preserving detection framework.

Cyber adversary: The cyber adversary targets the utility-level ICS with the objective of inducing operationally significant damage through manipulation of sensor measurements. Such manipulation can cause the actual residual covariance \(S^r_t\) to deviate statistically from the NLKF predicted covariance \(S_t\) which will statistically trigger the alternate hypothesis \(H_A\). Attack manipulation is assumed to occur only at the sensor level in the ICS hierarchy. We assume that the attacker does not possess visibility into or the capability to manipulate higher levels of the ICS, including the NLKF anomaly detection modules as well as other IT infrastructure itself. We focus on data injection attacks as the modality of the cyber adversary. The treatment of gradual degradation-based attacks [37] requiring persistent long-term control over sensing hardware is currently out of scope but forms a key aspect of our future work.

Privacy adversary: The privacy adversary observes the publicly disclosed DP-perturbed covariance \(\hat{S}_t\) and whitened residual \(\hat{\tau}^{res}_t\) or corresponding functional artifacts of the hypothesis test such as test statistic estimates or levels of significance. The privacy-oriented adversarial goal is to attempt to infer the underlying raw sensor measurements \(y_t\) or system state \(x_t\). The privacy adversary is assumed to be computationally bounded and has knowledge of the DP mechanism and its parameters.

Trust Assumptions: We operate under a semi-honest trust model regarding the disclosure recipients which include regulatory agencies and public bodies. While these entities are assumed to faithfully execute the verification protocol using the disclosed statistics \(\hat{S}_t\) (i.e., they perform valid statistical checks), we assume they are also curious about the underlying system dynamics. Consequently, they may attempt to exploit the disclosure to infer sensitive proprietary information such as raw sensor data \(y_t\) or state space dynamics such as \(x_t\) that exceeds their authorization. Therefore, the privacy guarantees must hold even against such curious recipients. Lastly, we assume that the disclosure channel between utility and regulators itself is trustworthy.

Security Objective: The primary security objective of PRICE is to ensure that public disclosures of the DP-perturbed covariance \(\hat{S}_t\) and residual \(\hat{\tau}^{res}_t\) do not compromise the confidentiality of the underlying raw sensor measurements \(y_t\). We aim to satisfy formal Differential Privacy guarantees such that no curious observer, whether a third-party entity or the regulatory body, can distinguish the true underlying sensor data beyond the pre-agreed privacy budgets \((\epsilon_{cov}, \gamma_{cov})\) and \((\epsilon_r, \gamma_r)\). This ensures that stakeholder ICS dynamics remain hidden from unauthorized inference attempts while maintaining the integrity of the disclosed verification statistics.

4.3 Differential Privacy Primer↩︎

We provide a brief summary of the following useful concepts pertaining to differential privacy.

In other words, Definition 1 ensures that for any two points \(x_1,x_2\) in domain \(\mathbb{R}^n\), the probability that mechanism \(\mathcal{M}(x_1)\) leads to an output in the set \(\mathcal{O} \subseteq \mathbb{R}^k\) is upper bounded by the probability that mechanism \(\mathcal{M}(x_2)\) leads to an output in the set \(\mathcal{O} \subseteq \mathbb{R}^k\) scaled by exponentiation of the privacy loss \(\epsilon>0\) with an addition of \(\delta>0\).

For DP mechanisms, we characterize the concept of adjacency with respect to a multi-variate function \(f:\mathbb{R}^d\rightarrow \mathbb{R}^{d}\). Any two elements \(x_1,x_2\in\mathbb{R}^d\) are deemed to be adjacent, if they differ in at most 1 element. Consequently, the sensitivity of a function depends on the maximum difference in function values caused by adjacent elements. Definition 2 formalizes the concept of sensitivity.

Most common DP formulations utilize \(k=\{1,2\}\) denoting \(l_1\) or \(l_2\) sensitivity assumptions respectively. In this paper, we adopt a two phase approach towards differential privacy wherein each phase is applied sequentially and pertains to protecting the privacy of the covariance matrix and residual vectors respectively. More specifically, we leverage a DP-based method [59] for privacy preserving covariance matrix factorization while employing a Gaussian Differential Privacy (GDP) approach for residual vectors.

The GDP mechanism provides \((\epsilon,\gamma_{r})\)-DP when \(\epsilon \in(0,1)\), \(c^2>2ln(1.25/\gamma_{r})\) and \(\sigma\geq c\Delta_2/\epsilon\). As a result, we have Equations 13 and 14 . Since the \(l_2\) sensitivity \(\Delta_2\) is used in the context of GDP of residuals, we use \(\Delta_2=\Delta_r\) for notational clarity. \[\begin{gather} \sigma > \frac{\Delta_r}{\epsilon}\sqrt{2ln(\frac{1.25}{\gamma_r})} \tag{13}\\ \mathbb{P}\Big(\Big|\sum\limits_{i=1}^{p} e_{i,t} \Big| \geq \theta_r \Big) \leq \gamma_r\text{ , } \theta_r = \frac{\sigma^2\epsilon}{\Delta_r} - \frac{p\Delta_r}{2} \tag{14} \end{gather}\]

4.4 Privacy Preserving Covariance Disclosures↩︎

We utilize the Laplacian noise based DP method [59] that factorizes \(S_t\) to yield a DP driven perturbed matrix \(\hat{S_t}\) as given in Equation 15 . However, instead of directly perturbing \(S_t\), we compute its square root factorization \(\tilde{S}_t = V_t\Lambda^{\frac{-1}{2}}_tV_t^T\). Next we perturb eigenvalues of \(\tilde{S}_t\) as specified in [59] so as to compute \(\hat{S}_t =\hat{V_t}\hat{\Lambda}\hat{V_t}^T\). Consequently, we obtain \(\hat{S}^{\frac{-1}{2}}_t = \hat{V_t}\hat{\Lambda}^{\frac{-1}{2}}\). Using \(\hat{S}^{\frac{-1}{2}}_t\), we can also obtain a perturbed lower dimensional residual \(\hat{\tau}^{cov}_t\) and its corresponding test statistic \(\hat{T}^{cov}_{\chi^2,t}\) denoted by Equations 16 and 17 respectively. Using Laplacian noise to perturb eigenvalues as described in [59] ensures \((\epsilon_{cov},\gamma_{cov})\)-DP for all \(\lambda_i, i \in \{1,d\}\) such that \(\hat{\lambda}_i = \lambda_i + Lap(\Delta_{l}/\epsilon_{cov})\). \[\begin{gather} \hat{S_t} = \hat{V_t}\hat{\Lambda}_t\hat{V_t}^T \tag{15}\\ \hat{\tau}^{cov}_t = (\hat{S_t})^{-1/2}r_t \tag{16}\\ \hat{T}^{cov}_{\chi^2,t} = (\hat{\tau}^{cov})^T_t\hat{\tau}^{cov}_t = ||\hat{\tau}^{cov}_t||^2_2 \tag{17} \end{gather}\] Further, we note that the perturbed test statistic \(\hat{T}^{cov}_{\chi^2,t}\) is computed using a DP-induced covariance matrix \(\hat{S}_t\) and an unperturbed residual vector \(r_t\). Consequently, we can state the relations defined by Equations 18 and 19 as stated in [59]. \[\begin{gather} E_l = ||\Lambda-\hat{\Lambda}||_1 = \underset{i}{max} |\hat{\lambda}_i - \lambda_i|\tag{18}\\ \mathbb{P}\Big(E_l \leq \frac{\Delta_l}{\epsilon_{cov}}log\Big(\frac{d}{\gamma_{cov}}\Big) \Big) \geq 1-\gamma_{cov} \tag{19} \end{gather}\] Equation 19 ensures that the maximum absolute Laplacian noise value given by \(E_l\) as defined in Equation 18 is less than \(\frac{\Delta_l}{\epsilon_{cov}}log\Big(\frac{d}{\gamma_{cov}}\Big)\) with probability at least \(1-\gamma_{cov}\). Analyzing the effect of DP-induced covariance matrices helps us establish Lemma 1.

Lemma 1 enables us to probabilistically bound the observed difference in test statistic value, only when DP is applied on the covariance matrix. However, in the interest of full compliance, it is also important to reveal the reduced residual values, such that the regulator can carry out the entire hypothesis test workflow represented in Equations 10 - 12 . Doing so, would require privacy protections on residuals as well, which we explore in the following section.

4.5 Privacy Preserving Residual Disclosure↩︎

We consider the unperturbed low dimensional residual representation \(\tau_t = (S_t)^{-1/2}(r_t)\) where \(\tau_t \in \mathbb{R}^p\) as defined in Section 3.4. We use the GDP mechanism presented in Definition 3 to perturb \(\tau_t\) according to Equation 20 . \[\begin{gather} \hat{\tau}^{res}_t = \tau_t + e_t, e_t \sim N(0,\sigma^2I) \tag{20}\\ \hat{T}^{res}_{\chi^2,t} = (\hat{\tau}^{res}_t)^T\hat{\tau}^{res}_t = ||\hat{\tau}^{res}_t||^2_2\tag{21} \end{gather}\] \(\hat{T}^{res}_{\chi^2,t}\) in Equation 21 represents the test statistic obtained purely through the GDP perturbation of \(\tau_t\). Generating \(\hat{T}^{res}_{\chi^2,t}\) is especially useful in enabling implementation flexibilities for compliance verification methods as discussed in Section 5. The GDP perturbation leads us to Lemma 2 which provides probabilistic bounds on the GDP noise vector \(e_t\).

Lemma 2 is vital towards deriving the overall privacy implications when the covariance as well as residual perturbations are integrated and presented to the regulator for compliance verification.

We now turn our attention towards characterizing the probability distribution of the GDP induced test statistic \(\hat{T}^{res}_{\chi^2,t}\). As a result, we consider the non-central \(\chi^2\) distribution with non-centrality parameter \(\mu\) and \(k\) degrees of freedom denoted by \(\chi^{2}(k,\mu)\). Further, we denote \(F^{\tau_t}_{\chi^2,p}(x)\) as the CDF of \(\chi^{2}(p,||\tau_t||^2_2)\). Consequently, we derive Lemma 3 which establishes the probability distribution of the variance scaled test statistic \((\hat{T}^{res}_{\chi^2,t}/\sigma^2)\) under GDP provisions.

Lemma 3 provides a distributional characterization of the perturbed residual test statistic under GDP conditions. Therefore, it serves as a precursor to Theorems 1 helping establish probabilistic bounds between perturbed and original low dimensional residual representations. Lemma 3 also proves to be an important enabler for computing DP-informed level of significance for carrying out the \(\chi^2\) hypothesis test at the regulator as represented in Theorem 2.

The main contribution of Theorem 1 lies in its ability to link the perturbed test statistic \(\hat{T}^{res}_{\chi^2,t}\) as a function of the non-central \(\chi^2\) CDF with \(p\) degrees of freedom centered on the original test statistic \(T_{\chi^2,t}\). As a consequence of Theorem 1, we can derive lower and upper bounds on the difference between original and perturbed tests statistics purely as a function of GDP parameters and the unperturbed low dimensional residual representation \(\tau_t\).

4.6 Integrating Covariance and Residual Privacy↩︎

Lemma 1 deals with aspects of privacy when only the covariance undergoes DP, Theorem 1 applies to a scenario wherein GDP is applied only to lower dimensional residual representations. However, integrating both of these individual DP steps is important so as to enable the disclosures of both covariance and residuals separately by the utility. Separate disclosures are vital to recreate the hypothesis test workflow at the regulator level to help satisfy compliance while preserving privacy of utility level operations. In this section, we address two issues which arise as a consequence of separate disclosures. First, introduction of DP noise across the covariance and residual steps might provide different test results at the regulator compared those of the original test at the utility level. Second, minimizing these misclassifications at the regulator level requires an alternate version of the test characterized by a level of significance customized for DP noise injected at the covariance and residual levels. Lastly, regulator level tests that consider separate disclosures covariance and residuals require strong privacy bounds that can help inform the implementation modalities.

We consider \(T_{\chi^2,t}\), \(\hat{T}^{cov}_{\chi^2,t}\) and \(\hat{T}^{res}_{\chi^2,t}\) corresponding to \(\tau_t\), \(\hat{\tau}^{cov}_t\) and \(\hat{\tau}^{res}_t = \hat{\tau}^{cov}_t + e_t\) respectively. Further, we consider \[\begin{gather} \Pi_t = \{r_t, \gamma_{cov}, \gamma_{res}, \epsilon_{cov},\epsilon_{l},\epsilon_{r}, \Delta_l,\Delta_r\} \label{eq:pit} \end{gather}\tag{22}\] In Equation 22 , \(\Pi_t\) denotes the set of observed residuals and DP parameters at time \(t\). We let \(\hat{\rho}_t, \rho_t\) denote the alarms raised with and without differential privacy respectively. We first establish Lemma 3 as a means to determine the upper bound the reduced dimensional DP residual \(\tau\) given the distributional knowledge of the perturbed test statistic.

We now consider the integration of DP driven covariance matrix structure defined in Section 4.4 and the GDP induced residuals discussed in Section 4.5. In order to do so, we consider the sequential application of DP on the covariance matrix followed by the GDP phase on the residual. Adopting a sequential approach enables a seamless framework of theoretical analysis that can be used to derive privacy oriented misclassification rates, alternate levels of significance as well as efficient and customizable implementation strategies. As a result, we have Equations 23 24 that represent the sequential privacy scheme. \[\begin{gather} \hat{\tau}^{cov}_t = (\hat{S_t})^{-1/2}r_t \tag{23}\\ \hat{\tau}^{res}_t = \hat{\tau}^{cov}_t + e_t, e_t \sim N(0,\sigma^2I) \tag{24} \end{gather}\] Next we consider \(\chi^{2,NC}_{\hat{\alpha}}\) which denotes the \(\hat{\alpha}\) level upper quantile of the non-central \(\chi^2\) distribution given by \(\chi^{2}(p,\hat{T}^{cov}_{\chi^2,t})\), where \(\hat{T}^{cov}_{\chi^2,t} = ||\hat{\tau}^{cov}_{t}||^2_2\) forms the covariance privacy induced test statistic defined in Equation 17 . We also note that \(\chi^{2,NC}_{\hat{\alpha}}\) denotes the \(\hat{\alpha}\) level upper quantile representing the level of significance for the hypothesis test carried out with DP measures. Similarly, \(\chi^{2}_{\alpha}\) represents the \(\alpha\) level of significance for the non-DP hypothesis test. We obtain Theorem 2 which considers a scenario wherein a regulator tries to recreate the hypothesis test on the basis of utility stakeholder disclosures that follow the sequential privacy scheme.

Theorem 2 helps characterize the Type-I error rate \(\mathcal{E}_I(\hat{\alpha})\) of the hypothesis test with \(\hat{\alpha}\) level of significance carried out at the regulator. We can see that \(\mathcal{E}_I(\hat{\alpha})\) is dependent on \(\theta_l\) which consists of the covariance DP parameter \(\gamma^{cov}\), covariance DP induced low dimensional residual \(\hat{\tau}^{cov}_t\), its corresponding maxima \(\hat{\tau}^{cov}_{max,t}\) computed according to Lemma 4 as well as their CDFs \(F^{\hat{\tau}^{cov}_t},F^{\hat{\tau}^{cov}_{max,t}}\) respectively. Additionally, we note that \(\mathcal{E}^{max}_I({\hat{\alpha}})\) also incorporates the GDP variance parameter \(\sigma\) as well. Therefore, Theorem 2 lays the foundation for computing the Type 1-error rate of the sequential privacy scheme that is collectively influenced by the privacy measures at the covariance and residual steps. We specifically derive a Gamma function dependent upper bound in Theorem 2 since it is known to provide tighter bounds when attempting to characterize tail-probabilities [60], [61].

A fundamental implication of Theorem 2 is that it helps guide the choice for \(\hat{\alpha}\) depending on the privacy parameters chosen by the stakeholders which also influence their privacy loss. Usually, stakeholders have an established level of significance \(\alpha\) depending on local detection benchmarks and domain expertise. Therefore, estimating the function \((\mathcal{E}^{max}_I)^{-1}(\alpha) = \hat{\alpha}\) provides an equivalent DP level of significance as a function of a pre-existing \(\alpha\). The inverse function estimation ensures that the Type-I error of the DP hypothesis test is upper bounded by the non-DP test.

As a consequence, stakeholders can choose a perturbed level of significance \(\hat{\alpha}\) such that \(\alpha = \mathcal{E}^{\max}_I(\hat{\alpha})\) which can be shared with the regulator. This choice of \(\hat{\alpha}\) ensures that the regulator can reconstruct the hypothesis testing workflow using privacy-preserving disclosures of covariance and residuals, while still maintaining a worst-case Type I error rate that does not exceed that of the unperturbed \(\chi^2\) test at the stakeholder level. In our framework, given a fixed stakeholder level \(\alpha\), we use the Monte-Carlo simulation method to estimate \(\hat{\alpha}\).

We extend Theorem 2 to determine bounds on misclassification rates of the hypothesis tests conducted with and without DP which is presented in Theorem 3.

Theorem 3 formally states the misclassification rates that can occur with respect to the stakeholder and regulator purely on account of privacy preserving disclosures of covariance and residuals as part of the sequential privacy scheme. Using Theorem 3, we can see that when Gamma CDF values decrease and are more sensitive when \(R_t\) is small, resulting in tighter bounds on misclassification rates. On the other hand, with larger \(R_t\) values, Gamma CDF increases culminating in lower misclassification rates.

In addition to Theorem 3, we introduce a GDP noise calibration factor \(\mu_t=||\hat{\tau}^{cov}_{max,t}||^2_2/\chi^{2,NC}_{\hat{\alpha}}\) to adjust the GDP noise variance \(\sigma^2_t=\mu_t.\sigma^{min}\). The calibration factor is designed to incentivize more targeted application of noise for computing residual disclosures. This is particularly useful in high residual cases (such as during an attack window) where the DP threshold \(\chi^{2,NC}_{\hat{\alpha}}\) might fail to deliver a good detection rate. As a result, we leverage \(\mu_t\) to improve the power of the test in a dynamic, DP friendly fashion.

5 DP driven Algorithmic Framework↩︎

We delineate our proposed algorithmic framework into two distinct components pertaining to the regulatory bodies and the utility stakeholders. For the utility level component, we focus on the development of a detection framework as well as relevant data disclosures based on residuals observed from the non-linear Kalman Filter model. On the other hand, the regulatory component purely focuses on the verification aspects based on disclosed data. Specifically, we consider a set of \(j\in \mathcal{J}\) utilities, where \(|\mathcal{J}| = J\). We divide the time horizon into discrete time steps denoted by \(t\) that yield a distinct observation of sensor measurements as well as its associated residual at each utility. Further, we group these time steps into sets of evaluation epochs \(w\), with each epoch consisting of \(W\) consecutive, discrete time steps.

As a consequence of the guarantees derived in Section 3, we can derive two distinct implementation modes of our algorithmic framework pertaining to critical region based compliance and P-value driven verification. It is important to note that both these implementation modalities are mutually exclusive and must be pre-determined with consensus among utilities and regulators. We present the algorithmic framework for each implementation mode.

5.1 Critical Region Based Verification↩︎

For the critical region (CR) compliance verification, we assume that the sole regulatory objective is to verify alarms with respect to differentially private disclosures of residuals and covariance from utilities. Alarm verification can be done by the disclosure of the test statistic by the utility followed by the critical region threshold.

5.1.1 CR based Utility Level Detection↩︎

The CR based utility level detection framework can be described on the basis of Algorithm 3. In Algorithm 3, at each time step the utility observes residual values \(r_t\) and \(C_t\). This is followed by the computation of the aggregated residual \(r_w\) and covariance matrix \(S_w\) and the epoch alarm \(\rho_w\) for the evaluation epoch \(w\). Based on these quantities, the utility can compute DP driven disclosures of \(\hat{\tau}^{res}_w\) and \((\hat{S}_w)^{-1/2}\). Finally, the utility transmits an information tuple \(\Pi_w\) consisting of the perturbed covariance matrix \((\hat{S}_w)\) as well as the transformed DP perturbed residual \(\hat{\tau}^{rg}_w\), the critical region threshold \(\chi^{2,NC}_{\hat{\alpha}}\) and the detected alarm \(\rho_w\) with the regulator. In addition, using the concept of post-processing immunity and composition [42], we can state that the disclosure of \(\hat{\tau}^{rg}_w\) preserves \((\epsilon,\delta)\) privacy as well.

5.1.2 CR based Regulatory Level Verification↩︎

Algorithm 4 captures the sequence of steps taken by a regulator for critical region based verification. At the regulatory level, we consider the set of utilities given by \(\mathcal{J}\). The regulator receives DP-based information tuple \(\Pi^j_w\) for each utility \(j\in \mathcal{J}\) at each evaluation epoch. Using information contained in \(\Pi^j_w\), the regulator can compute the factorization of the DP based covariance matrix \(\hat{C}^{-1/2}\) as well as obtain an estimate of the DP-driven test statistic \(\hat{T}^{j,res}_w\). The regulator can compute an alarm depending on the value of the test statistic \(\hat{T}^{j,res}_w\) and the value of the CR threshold \(\chi^{j,2,NC}_{\hat{\alpha}}\) based on the conditions given by 25 . \[\hat{\rho}^j_w = \begin{cases} 1, & \text{if } \hat{T}^{j,res}_w > \chi^{j,2,NC}_{\hat{\alpha}}, \\ 0, & \text{otherwise.} \end{cases} \label{eq:dp95alarm95condition}\tag{25}\]

As a consequence of Algorithm 4, the regulator independently obtains an estimate of \(\hat{\rho}^j_w\) which can be compared with the reported alarm \(\rho^j_w\). The misclassification rates pertaining to \(\hat{\rho}^j_w\) and \(\rho^j_w\) is provided using Theorem 3. Additionally, we provide a probabilistic bound on the worst case privacy loss incurred as a consequence of the CR based verification mode captured in Algorithms 4 and 3 in Theorem 4.

Theorem 4 tells us that if GDP failure risk is low, as indicated by \(\gamma_r \rightarrow 0\), the likelihood of the overall privacy loss exceeding \(\mathcal{L}(\Delta_r,\sigma^2,\hat{S}_w)\) will be negligible. Additionally, lower values of \(l_2\) sensitivity of residuals (denoted by \(\Delta_r\)) and high GDP noise (denoted by variance \(\sigma^2\)) minimize the lower bound on the worst case privacy loss. Lastly, \(\mathbf{1}^T\hat{S}^{-1}_w\mathbf{1}\) can be viewed as a scaled Rayleigh quotient for \(\hat{S}^{-1}_w\) computed using the vector \(\mathbf{1}\). We can observe generally that increasing covariance privacy noise, characterized by increasing \(\Delta_l/\epsilon_{cov}\), leads to a diminished value of the scaled Rayleigh quotient implying a lower worst case privacy loss.

5.2 P-value Based Compliance↩︎

For the P-value (PV) compliance verification, the regulatory objective is to ensure that alarms have been computed with the correct P-value at the utility level. In this case, the utility discloses the test statistic distribution parameters as well as the DP-equivalent level of significance for verification of alarms at the regulator level.

5.2.1 PV based Utility Level Detection↩︎

To facilitate PV compliance, the utility level detection algorithm is represented by Algorithm 5. Similar to a CR driven setting, the utility computes the values of \(\hat{\tau}^{cov}_w,\hat{\tau}^{res}_w, \hat{\alpha}\) and generates an alarm \(\rho_w\). It discloses the information tuple \(\Pi_w\) consisting of the values \(\hat{T}^{res}_w, \hat{T}^{cov}_w, \hat{\alpha}_w,\rho_w\) to the regulator.

5.2.2 PV based Regulator Level Detection↩︎

The regulator level algorithm for PV compliance verification is given in Algorithm 6. The objective of the regulator in this case is to estimate the non-central chi-square distribution using \(\hat{T}^{j,cov}_w\) as the centrality parameter according to 3 which can be used to estimate \(\chi^{j,2,NC}_{\hat{\alpha}_w}\). On the basis of the alarm condition represented by Equation 25 , the regulator can independently obtain and validate the alarm \(\hat{\rho}^j_w\) with respect to \(\rho^j_w\) for each utility.

In the PV implementation mode the regulator has access to the parametrized probability distribution of the DP test statistic \(\hat{T}^{j,res}_w\) denoted by the non-central chi-squared distribution with centrality parameter \(\hat{T}^{j,cov}_w\). This enables the regulator to obtain P-value of the DP test statistic based on the perturbed level of significance \(\hat{\alpha}\) divulged by the utility. Additionally, we can derive bounds on the privacy loss incurred through the disclosure of \(\hat{T}^{j,cov}_w\) in Theorem 5.

Theorem 5 provides several insights into the privacy implications regarding the disclosure of \(\hat{T}^{cov}_w\). First, we observe that with higher GDP covariance (\(\sigma^2\)) and lower sensitivity (\(\Delta_r\)) individually contribute to \(\epsilon'\) making it closer to \(\epsilon_{cov}\). Additionally, as lower bound on \(\epsilon'\) approaches \(\epsilon_{cov}\), we can also observe that the privacy failure probability \(\delta'\) also tends towards 0. These observations imply that a higher GDP noise covariance and lower sensitivities while disclosing \(\hat{T}^{cov}_w\) increasingly tends towards an \((\epsilon_{cov},\gamma_{cov})\)-DP paradigm.

6 Experimental Results↩︎

We primarily leverage the HAI Security dataset [17], as well as the ORNL power system (ORNL-PS) attack dataset [35], [62] for carrying out all experiments. We provide additional information pertaining to the use of these datasets in Appendix 9.1. Additionally, we provide an overview of the system implementation details in Appendix 9.2. We provide supplementary results of our experimental framework driven by a container based implementation in Appendix 10.1

5pt

5pt

4pt

DP parameter choice: In our experiments \(\frac{\Delta_{l}}{\epsilon_{cov}}\) denotes the scale value for the Laplacian distribution used for covariance disclosures. For all our experiments we utilize \(\sigma = \frac{\Delta_r}{\epsilon_r}\sqrt{2.ln\Big(\frac{1.25}{\gamma_r}\Big)}\). For the HAI and the ORNL-PS datasets, we used \(\Delta_r=50\), and \(\Delta_l=0.1\). These values were chosen based on rigorous empirical analysis of the maximum 2-norm residual values observed as well as the maximum eigenvalue square roots observed in the covariance matrices.

Visualization strategy: We present graphs depicting trends that include a min-max band comprising the minimum and maximum values of the desired quantity observed over five consecutive runs. The min-max encapsulate the median values observed across five independent runs for each experiment. To accommodate the non-linear nature of the model and complexities in the HAI data, we consider a 40 minute window pertaining to the ap_05 attack scenario. In order to account for the nonlinear nature of the detection model, we designate the first 800 secs from the start ap_05 as the attack window for the HAI dataset. Since the attack duration in the ORNL-PS datasets exceeds 800 seconds, we restrict ourselves to results for the first 57 minutes from the beginning of the dataset. Red vertical lines depict false alarms with respect to the median obtained from five independent runs. Green vertical lines reflect the correct detection of an attack.

6.1 Analyzing DP Covariance Disclosures↩︎

We begin by analyzing the effects of differentially private disclosures of covariance matrices. Our analysis focuses on both HAI and ORNL-PS datasets specifically examining impacts of \(\epsilon_{cov}\) and \(\gamma_{cov}\) on the P-value and the test statistic in comparison to the non-DP scenarios.

6.1.1 Effect of Privacy Budget↩︎

We begin by analyzing the P-values of both datasets under varying covariance privacy budget values. Figures 7 (a), 8 (a) reflect the P-value trends pertaining to HAI and ORNL-PS datasets respectively. The corresponding test statistic trends are depicted in Figures 9 (a) and 10 (a) respectively.

We note that increasing values of \(\epsilon_{cov}\) correspond to lower DP noise and a higher privacy budget as well. However, Figures 7 (a), 8 (a), 9 (a) and 10 (a) also indicate that with higher privacy budget, there is a much higher variation observed in both P-value and test statistic trends. In other words, with a higher privacy budget, we can observe much less variation in detection quality. Additionally, the consistency of alarms is higher in median terms during the attack period although this trend also results in some false alarms from the median as well. In Figure 8 (a), we observe that for \(\epsilon_{cov}=500\) the median does not breach the P-value threshold, although the trends make it clear that the attack detection is still robust with far lesser variations observed.

6.1.2 Effect of Failure Probability↩︎

We turn our attention to the effect of failure probability observed when disclosing covariance values. Figures 7 (c), 8 (c) represent trends in the P-value for HAI and ORNL-PS datasets, while Figures 9 (c), 10 (c) represent trends in test statistic. \(\gamma_{cov}\) conventionally represents the DP failure probability for covariance disclosures. Ultimately, a higher \(\gamma_{cov}\) represents a higher likelihood of DP being ineffective at hiding. From Figures 7 (c), 8 (c), 9 (c) and 10 (c), we observe that with increasing values of \(\gamma_{cov}\), we again see more variance and in P-value and test statistic trends. However, we can also observe that the median detection quality results in consistent alarms during the attack window albeit with a higher variance. In summary, PRECISE is capable of providing attack detection with realistic privacy expectations even in cases of high \(\gamma_{cov}\) values, indicating the usefulness of our approach.

6.2 Analysis for DP Residual Disclosures↩︎

We now analyze the impact of differentially private residual disclosures in terms of the privacy budget \(\epsilon_r\) and failure probability \(\gamma_r\).

6.2.1 Effect of Privacy Budget↩︎

We plot the P-value trends for varying values of \(\epsilon_{r}\) for both datasets in Figures 7 (b) and 8 (b) respectively. Similarly, we present test statistic trends for both datasets in 9 (b) and 10 (b) respectively as well. In general, we see that the spread of values as indicated by the min-max spread increases with decreasing values of privacy budget \(\epsilon_{r}\). This results in a slight increase in median alarm consistency for higher values of \(\epsilon_{r}\) as well. Overall, even with a high value of privacy budget, we achieve significant privacy guarantee.

6.2.2 Effect of Failure Probability↩︎

We present the performance of the DP detection framework in terms of the residual disclosure failure probability \(\gamma_r\) in terms of the P-value and test statistics. For both datasets, we can see that the attack detection performance remains robust with steady consistency of the median alarm detection rates as well.

6.3 Analyzing Attack Detection Quality↩︎

In this subsection, we analyze the performance of the combined DP attack detection framework considering the residual based DP disclosures in conjunction with the DP based covariance matrices. The combined analysis is meant to provide vital insights into the latency of attack detection at the regulator level with respect to the local utility.

Therefore, we examine the ability of the DP based framework to detect the attack within three distinct intervals (200s, 400s and 600s) measured from the start of the attack. The results of these experiments are obtained from five independent runs carried out for each DP parameter combination listed in Tables ¿tbl:tab:align95hai? and ¿tbl:tab:align95ornl?. We are primarily interested in tracking the alignment of DP and Non-DP detection at the discrete interval values of 200s, 400s and 600s. The alignment problem can be thought of as run instances wherein both DP and Non-DP frameworks successfully detected an attack. Therefore, we present results in terms of alignment of DP and Non-DP Detection, Non-DP only detection, DP alignment rates as well as the mean and variance of the \(\hat{\alpha}\).

In Tables ¿tbl:tab:align95hai? and ¿tbl:tab:align95ornl? we see a consistent improvement in the alignment rate with increase in duration from the beginning of the attack. The alignment rate measured as a fraction of runs where an attack was detected within the attack window consistently improves from around 0.5 to 0.95. This is powered by the rising number of DP and Non-DP detection instances that is naturally accompanied by falling Non-DP only detection. For both datasets, we see that the mean \(\hat{\alpha}\) stays relatively stable at \(2.56e-4\) and \(1.01e-4\) respectively. The associated variance experiences minor volatility but overall retains stability around \(4.55e-10\) and \(1e-10\) for HAI and ORNL-PS datasets respectively.

In Table 2, we present the false alarm rates of the DP based detection framework for both datasets captured during normal operations. We can see that for all the considered combinations of DP parameters, the false alarm rates stay consistently low. For the HAI dataset, the stability of false alarm rates hover around \(8\%\), while for ORNL-PS dataset, this value exhibits slightly more volatility, ranging from \(4\%\) to around \(6.8\%\). Collectively, Tables ¿tbl:tab:align95hai?, ¿tbl:tab:align95ornl? and 2 demonstrate the robustness of the attack detection quality with respect to the misclassification and alignment rates of the DP mechanism.

7 Conclusion↩︎

In this paper, we present PRECISE, framework geared towards regulatory compliance for detecting data-driven attacks in industrial control systems for critical infrastructure networks. Our proposed method leverages statistical tests on residuals arising out of state space modeling at the utility stakeholder level to raise attack alarms. We focus on cases wherein utilities are interested in convincing regulatory bodies regarding the veracity of their respective alarms by disclosing differential privacy induced covariance matrices and residual values. As a result, PRECISE revolves around a two phase privacy scheme that sequentially perturbs covariance using Laplacian noise followed by a Gaussian differential privacy scheme for residuals. We derive strong privacy guarantees pertaining to the test of residuals in addition to providing tight bounds on the misclassification rates of alarms as well as equivalent levels of significance. We specifically explore two significant modalities of implementation concerning critical region and P-value based compliance schemes. Additionally, we theoretically characterize the privacy implications of each of the modalities. Using real-world ICS datasets, we characterize the performance of our algorithm with respect to varying privacy parameters under diverse attack scenarios. The experimental results demonstrate that PRECISE is capable of matching the performance of the non-DP versions in almost all cases while preserving the privacy of utility stakeholders.

8 Proofs of Lemmas and Theorems↩︎

8.1 Proof of Lemma 1↩︎

8.2 Proof of Lemma 2↩︎

8.3 Proof of Lemma 3↩︎

8.4 Proof of Theorem 1↩︎

8.5 Proof of Lemma 4↩︎

8.6 Proof of Theorem 2↩︎

8.7 Proof of Theorem 3↩︎

8.8 Proof of Theorem 4↩︎

8.9 Proof of Theorem 5↩︎

9 Supplementary Information on Experiments↩︎

9.1 Dataset and Detection Models↩︎

For our experiments, we primarily leverage the HAI Security dataset [17], as well as the ORNL power system (ORNL-PS) attack dataset [35], [62]. For both datasets, we trained an NLKF model as described in Section 3.2 using the corresponding state and sensor variables pertaining to each dataset. In order to yield a well-formed non-linear, extended Kalman Filter model for the HAI dataset, we utilized the multi-level LSTM framework provided in [64]. Our experimental strategy revolves around evaluating the variations pertaining to DP failure probabilities \(\gamma_{cov},\gamma_r\), the privacy budgets \(\epsilon_{cov},\epsilon_r\) respectively.

9.2 System Implementation Details↩︎

All experiments were carried out on a virtual machine (VM) running Ubuntu 24.04 with 100GB of RAM and 16 vCPUs using Python 3.11 with the detection model inference using PyTorch 2.7.1. For evaluating diverse aspects of our proposed framework, we utilized a native as well as a container based execution environment. The native setup was primarily used for evaluating the performance of our framework under various scenarios of differential privacy. The container based execution environment was used to evaluate the distinct implementation modes pertaining to Critical Region Verification (CRV) and the P-Value Compliance (PVC). Specifically, we generated container images representative of the regulator and the utility that replays snippets of the HAI and ORNL-PS datasets under scenarios of attack. In order to evaluate system performance of CRV and PVC mode, we created container images representative of the regulator and the utility that replays attack scenarios of HAI and ORNL-PS datasets. The regulator container service hosts a REST API developed using Flask that receives DP-driven disclosures from the corresponding utility container service for each implementation mode and executes the corresponding compliance steps as provided in Algorithms 4 and 6. Consequently, we measure the system performance in terms of the CPU utilization and memory consumption of both utility and regulator services under both implementation modes. Docker based quick start scripts and the associated code have been provided as part of the accompanying artifacts to our paper.

10 Supplementary Results↩︎

10.1 Analyzing System Performance↩︎

In Table 3, we provide a comparative analysis of system performance of the CRV and PVC implementation modes with respect to HAI and ORNL-PS datasets. Average CPU utilization consistently stays under 11% for all scenarios with standard deviation ranging from 11.15% to 13.6%. Similarly, memory usage for utility service ranges between 1014 MB and 1101 MB, while remaining very close to 128 MB for the regulator service. From Table 3, we also see that the standard deviation values of CRV memory usage is relatively higher for both datasets. This rise can be explained on the basis of the need to factorize the DP-driven covariance matrix by the regulator in the CRV mode as opposed to a simple compliance check needed in the PVC mode. Overall, Table 3 demonstrates that the system performance in terms of both average memory consumption as well as CPU utilization remain consistent and stable across both datasets and implementation modes. The codebase is publicly available at https://github.com/disys-lab/precise.git.

References↩︎

---

1. Under review at ACM Transactions on Privacy and Security.↩︎