Which Hyperparameters Matter?
A Game-Theoretic Framework for Interpretable Hyperparameter Sensitivity Analysis
July 17, 2026
Abstract
This work presents a game-theoretic framework for interpretable hyperparameter–objective interaction analysis rather than proposing a new optimization algorithm. In the proposed framework, Shapley Effects are employed for global sensitivity analysis, while Pareto front sets are utilized to identify effective hyperparameter configurations and support early-stage model evaluation. The resulting analysis reveals which players (hyperparameters) are most influential with respect to different objectives in a given game (application). Consequently, the proposed framework provides interpretable insights into objective-aware hyperparameter interactions, enabling practitioners to guide subsequent optimization, reduce the search space, and perform early-stage model evaluation. The effectiveness of the proposed framework is demonstrated using three distinct neural network architectures across different problem domains under multi-objective settings.
Game Theory, Hyperparameter Sensitivity Analysis, Shapley Effects, Pareto Front Analysis, Multi-objective Learning
1 INTRODUCTION↩︎
Modern machine learning models rely critically on hyperparameters that govern architecture, optimization dynamics, regularization, and inductive bias. These choices are especially consequential for deep learning systems, where training costs are high and performance varies significantly across architectures. Consequently, practitioners invest substantial computational resources into hyperparameter optimization (HPO), commonly using black-box strategies such as random search, Bayesian optimization, evolutionary methods, and bandit-based approaches [1]–[4]. While effective, these methods provide limited insight into which hyperparameters are truly influential for a given network class, often leading to redundant searches and excessive training cost [5], [6].
This opacity is increasingly problematic in safety-critical, resource-constrained, and human-in-the-loop settings, where interpretability, robustness, and efficiency are often as important as peak predictive accuracy [7]–[9]. Recent work has therefore explored explainable HPO by adapting feature-attribution techniques, most notably Shapley values, to analyze hyperparameter influence [10]–[12]. Although informative, these approaches are typically post-hoc, model-specific, and computationally demanding [4], [13].
Shapley values, originating in cooperative game theory [14], provide a principled mechanism for allocating global performance contributions while accounting for interactions. Their impact spans explainable AI, feature attribution, and data valuation [15]–[17]. In global sensitivity analysis (GSA), Shapley Effects overcome key limitations of variance-based Sobol’ indices, particularly under nonlinearities, interactions, and correlated inputs [18]–[22]. These properties are especially relevant for hyperparameter spaces in deep learning, which are high-dimensional and interaction-dominated [23], [24].
In this manuscript, we propose a game-theoretic framework for hyperparameter–objective interaction analysis that leverages Shapley spectra and Pareto front analysis to characterize player influence in a cooperative game formulation applicable to machine learning models across different tasks. These spectra expose architecture-dependent sensitivity patterns, identifying hyperparameters that dominate performance and those with negligible impact that can be fixed or deprioritized [18], [19]. By leveraging input–output mapping-based game-theoretic global sensitivity analysis (GSA) techniques, Shapley spectra can be estimated efficiently without requiring exhaustive retraining [25], [26].
The proposed framework provides practitioners with a general analysis approach that yields reusable and transferable prior knowledge across similar models and tasks while requiring minimal additional computational overhead. The remainder of this paper
is organized as follows: Section 2 introduces the proposed methodological framework, Section 3 evaluates the approach through a set of experimental studies, Section 4 discusses the key takeaways and limitations of the proposed framework, and Section 5 concludes the paper. The codebase and datasets used in this study are publicly available at
https://github.com/NyiNyi-14/GT_MOHA.git.
2 METHODOLOGICAL FRAMEWORK↩︎
In the proposed framework, each tunable component of the learning algorithm is modeled as a player in a cooperative game, where the players correspond to hyperparameters in the context of neural network optimization. The overall workflow of the proposed framework is illustrated in Fig. 1. To construct the input–output mappings between players and objectives, a coarse-grained search is employed in this work. Nevertheless, the proposed framework is not restricted to coarse-grained search and can also be applied to alternative input–output mapping scenarios. The resulting mappings are stored in lookup tables, after which Shapley Effects and non-dominated sets are employed for sensitivity and Pareto front analysis. Finally, interpretable interactions between the players and the objectives are extracted as transferable prior knowledge.
2.1 Game-Theoretic Formulation↩︎
This step defines the game-theoretic formulation of the proposed framework, including the number of players and the values assigned to each player. Regardless of the total number of players, we recommend using a small set of representative values, typically lower, nominal, and upper bounds, to balance computational efficiency and sensitivity characterization. At this stage, fine-grained hyperparameter search is unnecessary, as the objective is to capture global trends rather than precise optima. Let \(\mathcal{P} = \{p_1, p_2, \ldots, p_n\}\) denote the set of players, \[\begin{align} p_k &\in \mathcal{A}_k = \{q_{k,1}, q_{k,2}, \ldots, q_{k,m_k}\}, \\[2pt] \Phi &= \mathcal{A}_1 \times \mathcal{A}_2 \times \cdots \times \mathcal{A}_n \subseteq \mathbb{R}^n, \\[2pt] \lvert \Phi \rvert &= \prod_{k=1}^{n} m_k, \\[2pt] \boldsymbol{\phi}_i &= (p_1^{(i)}, p_2^{(i)}, \ldots, p_n^{(i)}) \in \Phi, \\[2pt] \mathcal{Y}_i &= (y_{i,1}, y_{i,2}, \ldots, y_{i,J}) = f(\boldsymbol{\phi}_i), \\[2pt] \mathcal{D} &= \{ (\boldsymbol{\phi}_i, \mathcal{Y}_i) \mid \boldsymbol{\phi}_i \in \Phi \}_{i = 1}^{\lvert \Phi \rvert}, \end{align} \label{eq:proxy}\tag{1}\]
where \(\mathcal{A}_k\) denotes the discrete value set of the \(k\)-th player, \(q_{k,m_k}\) represents the \(m_k\)-th candidate value (action) assigned to player \(p_k\), \(\Phi\) is the Cartesian product of all such sets defining the player configuration space, \(\phi_i \in \Phi\) represents an individual configuration, and \((y_{i,1}, \ldots, y_{i,J})\) denotes the vector of application-dependent objectives produced by evaluating \(\phi_i\). The lookup table \(\mathcal{D}\), which stores all evaluated player configurations together with their corresponding objective outputs, is constructed at the end of the coarse-grained search.
2.2 Monte Carlo-Based Sobol–Shapley Sensitivity Analysis↩︎
Coalition Construction
First, all possible coalitions are enumerated as defined in 2 , and for each coalition, the corresponding conditional subsets are constructed by fixing the selected players. \[\begin{align} \mathcal{C}_k &= \{ \mathcal{S} \subseteq \mathcal{P} \mid |\mathcal{S}| = k \}, \qquad k = 1,\ldots,n, \\[2pt] |\mathcal{C}_k| &= \binom{n}{k}, \qquad |\mathcal{C}| = \sum_{k=1}^{n} \binom{n}{k} = 2^n - 1, \\[2pt] \mathcal{G}_\mathcal{S} &= \prod_{i \in \mathcal{S}} \mathcal{G}_i = \{ \boldsymbol{g}_\mathcal{S}^{(1)}, \ldots, \boldsymbol{g}_\mathcal{S}^{(|\mathcal{G}_\mathcal{S}|)} \}, \\[2pt] \mathcal{I}(\mathcal{S},\boldsymbol{g}_\mathcal{S}) &= \{ r \in \{1,\ldots,{\lvert \Phi \rvert}\} \mid \phi_{r,\mathcal{S}} = \boldsymbol{g}_\mathcal{S} \}, \\[2pt] \mathcal{D}(\mathcal{S},\boldsymbol{g}_\mathcal{S}) &= \{ (\boldsymbol{\phi}_r, \mathcal{Y}_r) \mid r \in \mathcal{I}(\mathcal{S},\boldsymbol{g}_\mathcal{S}) \}. \end{align} \label{eq:holdings}\tag{2}\]
Here, \(\mathcal{C}_k\) denotes the set of all coalitions of size \(k\), where each coalition \(\mathcal{S}\) specifies a subset of players to be held fixed. \(\mathcal{G}_{\mathcal{S}}\) is the Cartesian product of value sets for all players in \(\mathcal{S}\) and \(\mathcal{G}_i\) denotes the discrete value set assigned to player \(i\). For a given assignment \(\boldsymbol{g}_{\mathcal{S}} \in \mathcal{G}_{\mathcal{S}}\), \(\mathcal{I}(\mathcal{S}, \boldsymbol{g}_{\mathcal{S}})\) denotes the index set of lookup-table entries whose configurations match \(\boldsymbol{g}_{\mathcal{S}}\) on \(\mathcal{S}\), and \(\mathcal{D}(\mathcal{S}, \boldsymbol{g}_{\mathcal{S}})\) is the corresponding conditional subset of the lookup table.
Monte Carlo Estimation
To avoid exhaustive enumeration of the full search space, a coarse-grained search is adopted in place of a fine-grained exploration by bounding player values, yielding a lookup table with on the order of \(\sim10^2\) configurations. Consequently, Monte Carlo (MC) estimation is employed to approximate Sobol indices using a larger number of samples, rather than relying on exact computations from the limited discrete lookup table. Then, conditional variances are estimated by sampling over \(\mathcal{D}(\mathcal{S},\boldsymbol{g}_\mathcal{S})\), as detailed below. \[\begin{align} r^t &\stackrel{\text{i.i.d.}}{\sim} \mathcal{U}(\mathcal{I}(\mathcal{S},\boldsymbol{g}_\mathcal{S})), \quad t=1,\ldots,N, \\[2pt] \boldsymbol{y}_{\mathrm{MC}} &= [f(\boldsymbol{\phi}_{r^1}), f(\boldsymbol{\phi}_{r^2}), \ldots, f(\boldsymbol{\phi}_{r^N})], \\[2pt] \widehat{\mu}(\mathcal{S},\boldsymbol{g}_{\mathcal{S}}) &= \widehat{\mathbb{E}}\!\left[Y \mid \boldsymbol{G}_{\mathcal{S}}=\boldsymbol{g}_{\mathcal{S}}\right] = \frac{1}{N}\sum_{t=1}^{N} \boldsymbol{y}_{\mathrm{MC}}^{(t)}, \end{align} \label{eq:MC95estimator}\tag{3}\]
where \(r^{(t)}\) denotes MC indices drawn uniformly from \(\mathcal{I}(\mathcal{S},\boldsymbol{g}_{\mathcal{S}})\) for \(N\) samples, with \(N = 10^4\) used in this work, \(\boldsymbol{y}_{\mathrm{MC}}\) are the corresponding model outputs and \(\widehat{\mu}(\mathcal{S},\boldsymbol{g}_{\mathcal{S}})\) denotes the MC estimator of the conditional mean.
Sobol Variance Decomposition
Since player sensitivity is normally characterized by strong interactions and nonlinear effects, global sensitivity analysis is particularly appropriate when gradient information is often unavailable. Among available approaches, Sobol indices are of interest because they quantify, in a variance-based and model-agnostic manner, the proportion of output uncertainty explained by fixing a subset of players. In this work, first-order Sobol indices derived from the Sobol variance decomposition in 4 are employed [20]. \[v(\mathcal{S}) = \frac{\widehat{\mathrm{Var}}\big(\mathbb{E}[\mathcal{Y} \mid \boldsymbol{G}_{\mathcal{S}}]\big)}{\mathrm{Var}(\mathcal{Y})}, \label{eq:sobol95ind}\tag{4}\] where \(v(\mathcal{S})\) denotes the coalition value following the Sobol coalition-worth formulation, with \(v(\emptyset)=0\) corresponding to the empty set. \(\mathrm{Var}(\mathcal{Y})\) is the global output variance, and \(\widehat{\mathrm{Var}}\big(\mathbb{E}[\mathcal{Y} \mid \boldsymbol{G}_{\mathcal{S}}]\big)\) is the variance of the conditional expectation which are estimated using the MC procedure described in 5 . \[\begin{align} \mathrm{Var}(\mathcal{Y}) = \frac{1}{|\Phi|}\sum_{r=1}^{|\Phi|}\left(\boldsymbol{y}_r-\bar{y}\right)^2, \quad \bar{y}=\frac{1}{|\Phi|}\sum_{r=1}^{|\Phi|}\boldsymbol{y}_r, \\[2pt] \widehat{\mu}_{\mathcal{S}} = \frac{1}{|\mathcal{G}_{\mathcal{S}}|}\sum_{\boldsymbol{g}_{\mathcal{S}}\in\mathcal{G}_{\mathcal{S}}} \widehat{\mu}(\mathcal{S},\boldsymbol{g}_{\mathcal{S}}), \\[2pt] \widehat{\mathrm{Var}}\big(\mathbb{E}[\mathcal{Y} \mid \boldsymbol{G}_{\mathcal{S}}]\big) = \frac{1}{|\mathcal{G}_{\mathcal{S}}|} \sum_{\boldsymbol{g}_{\mathcal{S}}\in\mathcal{G}_{\mathcal{S}}} \left(\widehat{\mu}(\mathcal{S},\boldsymbol{g}_{\mathcal{S}})-\widehat{\mu}_{\mathcal{S}}\right)^2. \end{align} \label{eq:global95var}\tag{5}\]
Shapley Effect Estimation
While Sobol indices quantify the contribution of subsets of players to the total output uncertainty, they do not yield a unique attribution of importance to individual players in the presence of interactions. To address this limitation, Shapley Effects are employed, as defined in 6 , to fairly redistribute the coalition variance across individual players by averaging marginal contributions over all possible coalition orderings [14]: \[\mathbb{S}_i^{v_j} = \sum_{\mathcal{S}\subseteq \mathcal{P}\setminus\{i\}} \frac{|\mathcal{S}|!\,(n-|\mathcal{S}|-1)!}{n!} \left[ v_j(\mathcal{S}\cup\{i\}) - v_j(\mathcal{S}) \right], \label{eq:shapley}\tag{6}\] where \(\mathbb{S}_i^{v_j}\) denotes the Shapley value of player \(i\), with respect to objective \(j\), \(v_j : 2^{\mathcal{P}} \rightarrow \mathbb{R}\) is the objective-specific coalition value function and \(n = |\mathcal{P}|\) is the total number of players. Here, \(v(\mathcal{S}\cup\{i\})\) denotes the value of the coalition obtained by adding player \(i\) to coalition \(\mathcal{S}\), while \(v(\mathcal{S})\) denotes the value of the coalition excluding player \(i\). This Sobol–Shapley combination yields interaction aware and mode agnostic global sensitivity measures suitable for player analysis. \[\sum_{i\in\mathcal{P}} \mathbb{S}_i^{v_j} = 1. \label{eq:shapley95sum}\tag{7}\] Since the Shapley Effects \(\mathbb{S}_i^{v_j}\) analytically sum to unity for each objective, as shown in 7 , they provide a normalized measure of player importance across objectives. Consequently, the relative influence of each player on the objectives can be quantified and compared directly. Furthermore, as defined in 8 , a player \(i\) is categorized as less sensitive if its Shapley Effect remains below a user-defined threshold \(\tau\) for all objectives. Otherwise, the player is considered highly sensitive with respect to the game objectives. \[\label{eq:fix95vary} p_i = \begin{cases} \text{Non-influential}, & \max\limits_{j \in \{1,\dots,J\}} \mathbb{S}_i^{v_j} < \tau,\\ \text{Influential}, & \text{otherwise}. \end{cases}\tag{8}\] This classification facilitates identifying which players are most influential with respect to each objective and which players exhibit negligible impact across the objectives.
2.3 Pareto-Optimal Player Set Identification↩︎
During the coarse-grained search, each configuration is evaluated only once, and the resulting performance can be noisy due to the inherent stochasticity of neural network training. To mitigate this effect and avoid selecting configurations that benefit from occasional “lucky” runs, Pareto optimality [27] is employed to identify non-dominated player sets from the stored lookup table \(\mathcal{D}\). The resulting Pareto front provides insight into the trade-offs associated with different player configurations before selecting a final candidate set. Furthermore, this analysis enables early-stage model evaluation by revealing whether the model exhibits sufficient performance potential under different hyperparameter combinations prior to fine-grained hyperparameter tuning.
Definition 1 (Pareto dominance). Let \(\{\mathcal{Y}^{(i)} \}_{i=1}^{N}\) be the set of evaluated solutions of \(\mathcal{D}\) from 1 , where \(\mathcal{Y}^{(i)} \in \mathbb{R}^{J}\) is the objective vector for solution \(i\) across \(J\) objectives. Each solution corresponds to the objective vector induced by a distinct player set \(\boldsymbol{\phi}_i\) evaluated during proxy training. Let the \(j\)-th objective be denoted by \(y^{(i)}_j\), with direction \(\delta_j \in \{\text{min},\text{max}\}\). Since objectives can have different optimization directions, a simple sign transformation is applied to convert all objectives into a minimization form. Specifically, for the \(j\)-th objective, we define: \[s_j = \begin{cases} \;\;1, & \delta_j = \text{min},\\ -1, & \delta_j = \text{max}, \end{cases} \qquad j=1,\ldots,J,\] and the transformed objective vector: \[\tilde{\mathcal{Y}}^{(i)} = \big(s_1 y^{(i)}_1,\ldots,s_J y^{(i)}_J\big) \in \mathbb{R}^{J}, \quad i=1,\ldots,N,\] so that all objectives are cast into a minimization form. The solution \(k\) is said to dominate the solution \(l\) if \[\begin{align} \tilde{\mathcal{Y}}^{(k)} &\preceq \tilde{\mathcal{Y}}^{(l)} \;\;\text{and}\;\; \tilde{\mathcal{Y}}^{(k)} \neq \tilde{\mathcal{Y}}^{(l)}, \tag{9} \\ \tilde{y}^{(k)}_j &\le \tilde{y}^{(l)}_j \;\; \forall j\in\{1,\ldots,J\}, \quad \exists \; j \;\text{s.t.}\; \tilde{y}^{(k)}_j < \tilde{y}^{(l)}_j. \tag{10} \end{align}\] Finally, a solution \(k\) is defined as Pareto-optimal (non-dominated) set if there does not exist any \(l\neq k\) that dominates it: \[k \in \mathcal{P}^\star \iff \nexists\, l\in\{1,\ldots,N\}\setminus\{k\} \;\; \text{s.t.} \;\; \tilde{\mathcal{Y}}^{(l)} \prec \tilde{\mathcal{Y}}^{(k)}. \label{eq:pareto95set}\tag{11}\]
The set \(\mathcal{P}^\star\) constitutes the Pareto (non-dominated) front in the transformed objective space and provides information for early-stage model evaluation by characterizing the attainable trade-offs among the objectives.
3 CASE STUDIES & EXPERIMENTAL RESULTS↩︎
To evaluate the proposed framework, we conduct experiments on three distinct machine learning applications spanning different model classes and task types.
3.1 Experimental Setup↩︎
We consider (i) a physics-informed neural network (PINN), (ii) a convolutional neural network (CNN), and (iii) a deep neural network (DNN), each applied to a different task. Across all applications, each hyperparameter is treated as a player in the cooperative game formulation. For each model, the player set \(\mathcal{P} = \{p_1, p_2, \ldots\}\) follows the ordering of hyperparameters listed in the corresponding coarse-grained search tables. In the absence of prior knowledge, these hyperparameter ranges are chosen based on common practice.
3.1.1 Physics-Informed Neural Network: Dual Spring–Mass–Damper System↩︎
The dual spring–mass–damper system is used as a case study for the PINN [28] experiments, where the \(R^2\) metric is employed to evaluate the accuracy of the predicted mass displacements against the ground truth trajectories. The corresponding hyperparameter ranges are summarized in Table ¿tbl:tab:pinn95hyperparams?. The number of training steps is fixed at \(15 \times 10^{3}\), and a linear activation function is employed in the output layer. Since three objectives are considered, namely the prediction accuracies of masses 1 and 2 together with the training duration, the problem naturally constitutes a multi-objective optimization setting. Additional details regarding the system dynamics, including the schematic diagram and physical parameters, are provided in Fig. 2 and Table ¿tbl:tab:SPM95para?.
\begin{table}
| Hyperparameter | Candidate Values |
|---|---|
| Learning rate | \(\{10^{-3},\,10^{-4}\}\) |
| Number of layers | \(\{2,\,4\}\) |
| Neurons per layer | \(\{64,\,128\}\) |
| Physics loss weight (\(\lambda_{\mathrm{phys}}\)) | \(\{10^{-4},\,10^{-2}\}\) |
| Data loss weight (\(\lambda_{\mathrm{data}}\)) | \(\{1,\,10\}\) |
| Physics collocation points | \(\{60,\,120,\,240\}\) |
| Activation function | {Tanh, ReLU, Sine} |
-0.1in \end{table}
| Description | Value | Unit |
|---|---|---|
| Mass of first body (\(m_1\)) | 1 | kg |
| Mass of second body (\(m_2\)) | 2 | kg |
| Spring stiffness (\(k_1\)) | 5 | N/m |
| Spring stiffness (\(k_2\)) | 10 | N/m |
| Damping coefficient (\(c\)) | 0.5 | N s/m |
| External force (\(F(t)\)) | \(\sin(t)\) | N |
The governing equations of the system are given by: \[\begin{align} m_1 \ddot{x}_1 - c(\dot{x}_2 - \dot{x}_1) - k_2 x_2 + (k_1 + k_2)x_1 &= 0, \\[2pt] m_2 \ddot{x}_2 + c(\dot{x}_2 - \dot{x}_1) + k_2(x_2 - x_1) &= F(t). \end{align}\]
3.1.2 Convolutional Neural Network: CIFAR-100 Classification↩︎
The CIFAR-100 dataset is used to evaluate the CNN on a multi-class image classification task. With 100 classes, each containing 600 color images of size \(32 \times 32\), CIFAR-100 represents a relatively large-scale benchmark and is well suited for evaluating the proposed framework. Additional details regarding the dataset are provided in [29].
\begin{table}[ht]
| Hyperparameter | Candidate Values |
|---|---|
| Learning rate | \(\{10^{-3},\,10^{-4}\}\) |
| Batch size | \(\{16,\,64\}\) |
| Convolutional filters 1 | \(\{32,\,64\}\) |
| Convolutional filters 2 | \(\{64,\,128\}\) |
| Convolutional filters 3 | \(\{128,\,256\}\) |
| Kernel size | \(\{3,\,5\}\) |
| Activation function | {ReLU, Tanh, SiLU} |
| Batch Normalization | {True, False} |
-0.1in \end{table}
The CNN architecture is configured as summarized in Table ¿tbl:tab:cnn95hyperparams? with the number of training epochs fixed to 20 for all runs. The activation function is varied only within the convolutional layers, while the activation applied after flattening is fixed to ReLU, and the output layer employs a linear activation across all configurations. For the CNN classification task, a multiclass linear classifier is used at the output layer, and class predictions are obtained by applying an argmax operation to the output logits, without an explicit sigmoid or softmax activation during inference. Prediction accuracy and training duration are treated as the objective outputs in the multi-objective optimization setting.
3.1.3 Deep Neural Network: Adult Income Classification↩︎
The Adult Income dataset is used for the DNN binary classification task, where the network predicts whether an individual’s annual income exceeds $50K based on 14 input features. Dataset details are provided in [30].
\begin{table}[ht]
| Hyperparameter | Candidate Values |
|---|---|
| Learning rate | \(\{10^{-3},\,10^{-4}\}\) |
| Batch size | \(\{32,\,64,\,128\}\) |
| Number of layers | \(\{2,\,4\}\) |
| Neurons per layer | \(\{64,\,128\}\) |
| Activation function | \(\{\text{ReLU},\,\text{Tanh},\,\text{SiLU}\}\) |
| Batch normalization | \(\{\text{True},\,\text{False}\}\) |
| Dropout rate | \(\{0.0,\,0.3\}\) |
-0.1in \end{table}
The coarse-grained search space is summarized in Table ¿tbl:tab:dnn95hyperparams?, with the number of training epochs fixed to 50 for all runs. A linear activation is used at the output layer across all configurations, and a binary logistic classifier is employed, trained using a sigmoid-based binary cross-entropy loss. Prediction accuracy and training duration are considered as the two competing objectives in this multi-objective optimization case study.
3.2 Hyperparameter–Objective Interaction Analysis↩︎
The game-theoretic formulation for each application is summarized in Table ¿tbl:tab:emoho95games?. For each network, a coarse-grained search is conducted over the corresponding player set, and each evaluated configuration, together with its associated objective values, is stored in a lookup table. Sample lookup tables are provided in Tables ¿tbl:tab:pinn95lookup95sample?–¿tbl:tab:dnn95lookup95sample? in Appendix 6.1.
\begin{table}[ht]
| Application | Set of Players, \(\mathcal{P}\) | Objectives, \(\mathcal{Y}\) | Configurations, \(\lvert \Phi \rvert\) | Remarks |
|---|---|---|---|---|
| Pinn | \(\{p_1,\,p_2,\,\ldots,\,p_7\}\) | \(\{y_1,\,y_2,\,t\}\) | 288 | \(7\)-player, 3-objective game |
| Cnn | \(\{p_1,\,p_2,\,\ldots,\,p_8\}\) | \(\{y_1,\,t\}\) | 384 | \(8\)-player, 2-objective game |
| Dnn | \(\{p_1,\,p_2,\,\ldots,\,p_7\}\) | \(\{y_1,\,t\}\) | 288 | \(7\)-player, 2-objective game |
-0.1in \end{table}
3.2.0.1 Global sensitivity analysis
The Shapley Effect is estimated according to 6 , and the resulting global sensitivity of each player in the game with respect to the corresponding objectives is visualized in Fig. 3. The selection of the threshold value \(\tau\) from 8 is user-defined and application-dependent. In general, as the number of players in the game increases, individual sensitivity values tend to be more dispersed. The specific value of \(\tau\) adopted in this work is chosen for experimental purposes to demonstrate the proposed framework, and no claim of optimality is made regarding this selection and alternative threshold values may be selected to reflect different application requirements or user preferences. \[\label{eq:shapley95sum2} \sum_{i\in\mathcal{P}} \mathbb{S}_i^{y_1} \approx \sum_{i\in\mathcal{P}} \mathbb{S}_i^{y_2} \approx \sum_{i\in\mathcal{P}} \mathbb{S}_i^{t} \approx 1.\tag{12}\]
The Shapley effects are normalized and sum to unity for each objective, as shown in 12 . As illustrated in Fig. 3 (a) and (b), several hyperparameters exhibit negligible sensitivity across all objectives and therefore receive less emphasis during subsequent optimization or fine-tuning stages. Consequently, these hyperparameters can be assigned reduced search spaces by either fixing them to a single value or restricting them to a small set of candidate values. In contrast, Fig. 3 (c) shows that the sensitivities of most hyperparameters are relatively evenly distributed, suggesting that the DNN binary classification task considered in this study is comparatively less sensitive to hyperparameter variations. Therefore, extensive hyperparameter fine-tuning is expected to provide only limited improvement in the objectives for the considered model and dataset.
3.2.0.2 Pareto front set identification
The Pareto-optimal set is identified according to 11 , and the results are visualized in Fig. 4 which provides an early-stage evaluation of model performance for the corresponding tasks. In Fig. 4 (a), the PINN accuracies \(y_1\) and \(y_2\) span a wide range, from near zero up to approximately \(99\%\), while the training duration also varies significantly. This indicates substantial optimization potential, but also highlights a pronounced trade-off between accuracy and computational cost. In contrast, Fig. 4 (b) shows that the CNN used for multi-class classification attains a maximum accuracy of approximately \(50\%\), largely independent of training duration suggesting that the model has reached a saturated performance regime. Similarly, Fig. 4 (c) demonstrates that the accuracy of the DNN binary classification task is confined to a relatively narrow range (approximately \(83\%\) to \(86\%\)), indicating weak sensitivity to hyperparameter variations for the selected dataset.
Therefore, the combined global sensitivity analysis and Pareto front evaluation suggest that the PINN is well suited for predicting the dynamics of the dual spring–mass–damper system. In this case, local refinement can be performed by selecting balanced trade-off candidate configurations from the Pareto front and conducting further search in the surrounding region. In contrast, the CNN with a linear classifier applied to the CIFAR-100 multi-class classification task is not expected to achieve substantial improvement in the objectives through additional hyperparameter fine-tuning, since the Pareto front remains concentrated in a relatively low-accuracy region across all candidate configurations. Similarly, the DNN binary classification task on the Adult Income dataset exhibits comparatively low sensitivity to hyperparameter variations. These observations suggest that the proposed framework can provide early-stage model evaluation by revealing the foreseeable performance potential of a model for a given task prior to extensive hyperparameter optimization.
4 KEY TAKEAWAYS & LIMITATIONS OF THE PROPOSED FRAMEWORK↩︎
This work presents a sensitivity-based framework for analyzing hyperparameter—objective interactions in machine learning models rather than a new hyperparameter optimization (HPO) algorithm. Consequently, comparisons with existing HPO baselines are outside the scope of this work. As illustrated in Fig. 3, the influence of a hyperparameter is highly application-dependent, where even the same hyperparameter may exhibit substantially different sensitivities across models and objectives. For example, the learning rate may strongly influence the prediction accuracy of one model while having comparatively lower impact in another. Similar observations are found for the remaining hyperparameters, highlighting that hyperparameter importance cannot be generalized across different learning tasks. The resulting interaction analysis provides interpretable insights into the relative importance of design variables and may also serve as prior knowledge for subsequent hyperparameter optimization.
The proposed framework currently relies on a coarse-grained grid search to construct the input–output mappings required for the Shapley-based sensitivity analysis. Consequently, the resulting interaction analysis depends on the selected search space and the discretization of the hyperparameter values. In this work, the coarse-grained search serves primarily as a proof of concept for demonstrating the proposed analysis framework rather than as an optimal hyperparameter search strategy. Future research will investigate principled approaches for selecting representative sampling points and extend the framework to operate with more general input–output mappings obtained from alternative optimization strategies, such as random search, Bayesian optimization, or surrogate-based optimization methods.
5 CONCLUSION↩︎
This manuscript presents a framework for hyperparameter–objective interaction analysis that enables the interpretation of which hyperparameters are most influential with respect to different objectives and which hyperparameters require less attention. The proposed framework employs sensitivity analysis and Pareto front evaluation on the input–output mappings between players and objectives in a cooperative game formulation, thereby providing transferable prior knowledge regarding the considered model and task. As future work, the proposed framework will be extended to alternative search and sampling algorithms and applied to a broader range of systems.
6 Appendix↩︎
6.1 Coarse-Grained Search Results for Each Model↩︎
The evaluated objectives (\(y_1, y_2, \ldots\)) and their corresponding players (\(p_1, p_2, \ldots\)) from grid search are stored in a lookup table, and sample lookup tables for all models are summarized in Tables ¿tbl:tab:pinn95lookup95sample?–¿tbl:tab:dnn95lookup95sample?.
| Run | \(p_1\) | \(p_2\) | \(p_3\) | \(p_4\) | \(p_5\) | \(p_6\) | \(p_7\) | \(y_1\) | \(y_2\) | \(t\) |
| \((\mathrm{lr})\) | \((n_{\mathrm{layers}})\) | \((n_\mathrm{neurons})\) | \((\lambda_{\mathrm{phys}})\) | \((\lambda_{\mathrm{data}})\) | \((n_{\mathrm{phys}})\) | (Activation) | \((R^2_{x_1})\) | \((R^2_{x_2}\)) | (sec) | |
| 1 | 0.001 | 2 | 64 | \(10^{-4}\) | 1 | 60 | Tanh | \(-0.001\) | \(0.0007\) | 28.61 |
| 2 | 0.001 | 2 | 64 | \(10^{-4}\) | 1 | 60 | ReLU | \(0.0006\) | \(0.007\) | 21.31 |
| 3 | 0.001 | 2 | 64 | \(10^{-4}\) | 1 | 60 | Sine | 0.2253 | 0.2608 | 43.52 |
| ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ |
| 288 | 0.0001 | 4 | 128 | \(10^{-2}\) | 10 | 240 | Sine | 0.7202 | 0.7294 | 140.73 |
| \(\approx\) 4-hr | ||||||||||
| Run | \(p_1\) | \(p_2\) | \(p_3\) | \(p_4\) | \(p_5\) | \(p_6\) | \(p_7\) | \(p_8\) | \(y_1\) | \(t\) |
| (\(\mathrm{lr}\)) | (Batch) | (Conv\(_1\)) | (Conv\(_2\)) | (Conv\(_3\)) | (Kernel) | (Activation) | (Batch Norm) | (Accuracy, % ) | (sec) | |
| 1 | 0.001 | 16 | 32 | 64 | 128 | 3 | ReLU | True | 39.83 | 90.21 |
| 2 | 0.001 | 16 | 32 | 64 | 128 | 3 | Tanh | True | 36.46 | 86.52 |
| 3 | 0.001 | 16 | 32 | 64 | 128 | 3 | SiLU | True | 41.75 | 86.53 |
| ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ |
| 384 | 0.0001 | 64 | 64 | 128 | 256 | 5 | SiLU | False | 37.44 | 84.65 |
| \(\approx\) 7.6-hr | ||||||||||
| Run | \(p_1\) | \(p_2\) | \(p_3\) | \(p_4\) | \(p_5\) | \(p_6\) | \(p_7\) | \(y_1\) | \(t\) |
| (\(\mathrm{lr}\)) | (Batch) | (\(n_\mathrm{layers}\)) | (\(n_\mathrm{neurons}\)) | (Activation) | (Batch Norm) | (Dropout) | (Accuracy, %) | (sec) | |
| 1 | 0.001 | 32 | 2 | 64 | ReLU | True | 0.0 | 84.75 | 200.33 |
| 2 | 0.001 | 32 | 2 | 64 | ReLU | False | 0.0 | 83.09 | 119.01 |
| 3 | 0.001 | 32 | 2 | 128 | ReLU | True | 0.0 | 84.19 | 203.54 |
| ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ |
| 288 | 0.0001 | 128 | 4 | 128 | SiLU | False | 0.3 | 85.46 | 26.43 |
| \(\approx\) 6.3-hr | |||||||||