1 Introduction↩︎

The last decade has witnessed Deep Neural Networks (DNNs) become a cornerstone of AI and machine learning, especially when tasks demand high accuracy and mimic human-like cognitive processes. This proliferation is increasingly facilitated by Machine-Learning-as-a-Service (MLaaS) platforms, which have democratized access to AI capabilities.: image processing [1], [2], speech recognition [3], natural language processing [4]–[6], and automotive driving [7]–[10], and many more[11]–[16]. The economic scale of this transformation is immense; the global MLaaS market is projected to expand from approximately USD 44.2 billion in 2024 to over USD 1.2 trillion by 2034 [17]. Trained DNN models are no longer mere software artifacts but are now considered invaluable intellectual property (IP) and core business assets. With training compute costs estimated to be in millions of dollars, the high barrier to entry and the immense value embodied in the trained model make them a highly attractive target for theft and replication [18].

The architectural paradigm of AI development is undergoing a significant evolution, moving from bespoke, task-specific models toward large-scale, pre-trained “foundation models” [19]. These models are subsequently fine-tuned to perform a multitude of downstream tasks, a strategy that has become dominant across the industry. This centralization of value creates a corresponding centralization of risk. A successful attack that compromises a single proprietary foundation model no longer affects just one application; it can undermine an entire ecosystem of services built upon it. Consequently, the security of foundation models has transcended being a mere operational concern to become a strategic imperative for the entire AI industry.

The concentration of high-value IP within MLaaS platforms has naturally led to the development of model extraction attacks, where an adversary queries a model to reverse-engineer its functionality without direct access to its weights or architecture [20]. Historically, these attacks have relied on privileged access to the host system to monitor fine-grained, low-level resources such as memory, cache, or PCIe traffic [21]. However, such methods are often impractical in real-world public cloud environments for several reasons: first, they require access to system resources that are typically unavailable to a co-located user (such as in a public cloud environment); second, the extraction process is prone to overfitting on limited query data, diminishing the stolen model’s ability to generalize; and third, the reliance on fine-grained observations makes these attacks highly susceptible to data distortion and noise, leading to inaccurate model reconstruction.

The paradigm shift toward fine-tuning pre-trained foundation models fundamentally alters the adversary’s objective. The focus has shifted to leveraging standard pre-trained models, trained on large, publicly available datasets, and then fine-tuning them for specific tasks (e.g., by adding a few layers and training on smaller, task-specific datasets) [22]–[24]. This shift in development practices reduces the need to extract model parameters [25]–[27], as the core features of these pre-trained models are already well-established. Motivated by this 2-phase AI training approach, we propose stealthy and non-intrusive attacks that can easily infer the underlying DNN architecture (e.g., GPT, BERT, or ResNet) used by the target applications. Our primary goals are: (1) to accurately identify the DNN architecture without requiring physical access or root privileges in deployments that expose profiling interfaces, (2) to ensure robust architecture prediction with sparse and noisy data, (3) to achieve high accuracy prediction even when the model is compressed into binary executable file, and (4) to evaluate transferability across hardware/framework settings.

Our empirical analysis demonstrates that aggregate GPU profiles effectively distinguish different DNN architectures, such as convolution-heavy vision models and transformer-based language models. This is primarily due to the specific kernel calls and system-level workloads that vary across DNN architectures, and their variants. From our study, we identify two key insights: first, each combination of DNN layer type and size corresponds to a specific GPU kernel call; second, the type of kernel calls, their execution times, and resource utilization form a unique fingerprint for DNN architectures.

This paper introduces InferNet, an attack method that exploits statistical GPU performance metrics and workload behavior during the execution of a DNN model to identify its underlying architecture. InferNet identifies key GPU kernel calls and memory management events, along with their execution times, to create unique fingerprints for individual DNN layers. These metrics, combined with system-level data (such as memory usage, clock speed, and power consumption), are aggregated to profile the DNN model, and serve as a side-channel. By correlating the GPU profiles with known DNN architectures, InferNet accurately identifies both the DNN family (e.g., Inception, Transformer) and its variant (e.g., InceptionV1, InceptionV3).

InferNet adopts a temporally coarse but semantically rich profiling approach. Unlike prior work that depends on physical or low-level side-channels (e.g., PCIe sniffing or EM traces), our attack directly leverages GPU kernel names and system metrics captured via nvprof aggregate mode. While aggregate mode lacks kernel execution order (compared to time-series), it suppresses profile noises from GPU scheduling and internal resource contention, and therefore, provides precise kernel identities and summary statistics that form a strong architectural fingerprint. This challenges the prevailing assumption that fine-grained traces are essential for DNN architecture inference, and highlights the attack surface exposed by aggregate GPU profiling available to the attacker.

We evaluate the practicality and generalizability of InferNet across 90 diverse DNN architectures, including vision models, large language models (LLMs), and vision transformers. The architecture pool encompasses 21 DNN model families, each with multiple variants (see  1). These models are pre-trained and implemented in two major AI/ML frameworks: TensorFlow [28] and PyTorch [29]. Experiments are conducted on two hardware platforms: the NVIDIA Tesla T4 GPU on Google Colaboratory and the NVIDIA Quadro RTX 8000 GPU on an Ubuntu server.

Our evaluation results show that InferNet is effective across multiple DNN types, AI/ML frameworks, and GPU hardware configurations. Key results includes DNN architecture extraction accuracy of: (i) 100% using complete GPU profile, (ii) nearly 100% using only the top 3 kernel features, (iii) 100% on modified DNN architectures, and (iv) 75% across GPUs. Compared to similar methods, InferNet achieves 100% architecture extraction accuracy over a candidate set that is over four (4) times larger than the next best method (refer to 2 in the Related Work, §2).

Contributions. To summarize, our work makes the following important contributions:

• We identify aggregate GPU profiles—comprising kernel calls and system metrics—as a novel side-channel to fingerprint DNN architectures.

• We develop InferNet, a machine-learning-based attack method that uses GPU profiles to extract a wide range of DNN architectures, e.g., vision, and large language models (LLMs).

• Our method can fully recover the model architecture even when the model has undergone extensive optimizations.

• Empirical evaluation shows that the complete GPU aggregate profile is not required for the prediction; instead, a few key features are sufficient for accurate DNN architecture extraction.

• We characterize the transferability of InferNet across GPU platforms and AI/ML frameworks, and show that heterogeneous training data can substantially improve cross-environment prediction.

Organization.The rest of the paper is organized as follows. Section 2 reviews related work on DNN architecture leakage. Section 3 provides a technical overview of deep neural networks (DNNs) and their acceleration in GPU-enabled systems. In Section 4, we present empirical observations on the mapping of DNN layers to low-level GPU kernel calls, which form the foundation for the design of InferNet. Section 5 outlines the threat model and explains the methodology for extracting DNN architectures using GPU profiles. Sections 6 and 7 evaluate the effectiveness of our attack across different challenging scenarios. In Section 8, we discuss potential countermeasures to mitigate model extraction attacks. Finally, Section 9 summarizes our findings.

2 Review of Related Works↩︎

The high-value, centralized landscape of modern AI development has given rise to a diverse spectrum of security threats against DNNs. These threats include adversarial attacks, which manipulate model inputs to cause misclassification; data poisoning and backdoor attacks, which corrupt the training process to embed malicious behavior; and privacy-violating attacks like membership inference, which aim to determine if a specific data point was used in the model’s training set [30]. Among these threats, a particularly insidious category is the Model Extraction Attack (MEA), where an adversary seeks to steal the model’s architecture and weights [20], [31]. This section provides a comprehensive review of the literature on model extraction, with a particular focus on side-channel attacks that exploit unintentional information leakages from the underlying hardware and software systems to infer a model’s confidential architecture.

2.1 Escalated Attack Privileges↩︎

The landscape of Deep Neural Network (DNN) model extraction is defined by a fundamental trade-off between an attack’s capabilities and its required level of privilege. At one end of the spectrum lie attacks that achieve near-perfect model reconstruction but demand elevated, often impractical, access. For instance, the Hermes [37] attack accomplishes lossless model recovery by snooping unencrypted PCIe traffic between the CPU and GPU. Similarly, DeepSniffer [38] reconstructs architectural layouts by analyzing the volume of memory read/write operations from bus snooping. Other methods have demonstrated success by analyzing physical emanations; the CSI NN attack, for example, uses electromagnetic (EM) traces to reverse engineer the architecture and weights of networks on embedded devices [39]. While formidable, the reliance of these techniques on direct hardware interaction or physical proximity renders them infeasible in typical cloud or Machine-Learning-as-a-Service (MLaaS) environments [40]. Moreover, attacks like DeepPeep [34] and MoSConS [41] require access to the victim DNN during model training. The impracticality of these high-privilege methods in common cloud settings highlights the need for an attack vector that does not rely on physical access or strong adversary capabilities.

2.2 Microarchitectural and Time-Series Side-Channels↩︎

A more practical threat model involves exploiting microarchitectural side-channels in shared computing environments. These attacks typically rely on capturing a fine-grained, time-series trace of events to infer the sequence of a DNN’s operations. Cache-timing attacks [33], [42], [43], using underlying mechanisms like Prime+Probe or Flush+Reload, are a prominent example. Some, like Cache Telepathy [44], monitor cache hits on specific, shared DNN library functions to reconstruct the layer sequence.

A more novel approach, GANRED [45], captures the overall cache timing signature and uses a Generative Adversarial Network (GAN) to train a substitute model that mimics this signature, thereby avoiding a dependency on shared libraries. Other time-series attacks have targeted the GPU, such as Leaky DNN [41], which exploits the timing penalties of GPU context-switching to infer layer composition and hyperparameters during the training phase. The critical vulnerability of these time-series methods is their fragility; they are susceptible to system noise and can be defeated by countermeasures that disrupt the temporal order of operations, such as randomized kernel scheduling and precision limiting [41], [46]. The fragility of the time-series methods and unreliability of microarchitectural approaches create a clear need for a more robust side-channel that is resilient to system noise and temporal countermeasures.

Recent work further broadens the landscape of architecture leakage across side-channel modalities. Arefin and Serwadda [47] show that architecture-level information remains exposed even without direct physical access; Batina et al. [48] synthesize reverse-engineering attacks on neural networks through side channels; and Malan et al. [49] demonstrate DVFS-based fingerprinting in edge inference services. These studies reinforce that architecture information can leak through diverse hardware/software observables. InferNet differs by focusing on aggregate GPU profiling and multiclass prediction over a substantially larger candidate pool.

2.3 Binary-based Attacks↩︎

In scenarios where the adversary has access to the model’s binary or executable file (e.g., compiled CUDA code, ONNX, or LiteRT), several binary-based model extraction techniques become feasible. Prior works have demonstrated architecture recovery through function call and cache tracing [50], control-flow analysis [51], or side-channel observations of compiled code [39], [52]. Tools such as angr, Ghidra, and IDA Pro have also been used to disassemble binaries and recover model components when access to the model artifact is assumed. These approaches, however, typically require stronger adversarial capabilities, such as file system access, reverse engineering expertise, or dynamic instrumentation. In contrast, InferNet targets a weaker and more realistic threat model, where the attacker relies solely on aggregate GPU profiling data to infer the DNN architecture.

Our proposed attach method, InferNet, introduces a paradigm that circumvents the limitations of existing approaches by strategically sacrificing temporal data for statistical robustness. Attacks like Hermes [37], DeepSniffer [38], and DeepPeep [34] assume identical conditions in both training and deployment phases of the DNN lifecycle. Although identical settings maximize performance for all attacks, InferNet retains significant efficacy under varied conditions, demonstrating its practical versatility. It exhibits broader applicability, encompassing a wider range of deep neural network architectures, including vision models, vision transformers, and large language models (LLMs). As shown in 2, InferNet achieves 100% architecture extraction accuracy across a diverse candidate set of 90 models, encompassing vision models, vision transformers, and large language models (LLMs) across both PyTorch and TensorFlow—a pool more than four times larger than the next-best classification-based method.

3 Background↩︎

This section provides the technical background on deep neural networks (DNNs) and their use of multithreading technology on modern GPUs. We begin by describing the structure of DNNs and their key building blocks. Next, we introduce the CUDA toolkit [53] and explain how it accelerates data processing on GPUs. Finally, we discuss how profilers can help monitor various GPU workloads involved in a DNN execution.

3.1 Deep Neural Networks↩︎

Deep neural networks (DNNs) comprise a hierarchical organization of connected layers to model the relationship between the input \(x\) and output \(y\) by approximating an unknown function \(f^\theta(x) = y\). Here, \(\theta\) denotes the learned parameters, i.e., weights, biases, and other model-specific parameters of the layers in the architecture \(f\); and \(f^\theta\) is the resultant model. Computationally, the input \(x\) passes through the DNN in a series of feed-forward matrix operations. Each layer processes the input to extract specific features, progressively transforming it into higher-level representations. The final output \(y\) is typically a probability distribution over multiple categories.

DNNs operate in two main phases: training and inference. In the training phase, the model processes input data to produce an output, which is compared to the ground truth using a loss function to quantify the error in the model’s predictions. The model’s parameters \(\theta\) are adjusted iteratively through backpropagation to minimize this error [54]. Hyper-parameters, such as learning rate, epochs, and batch size that control the training process, are independent of the data [54]. In the inference phase, the trained model makes predictions on new, unseen data. When a DNN is deployed in a real-world setting, it operates in the inference phase.

3.1.1 DNN Architecture↩︎

DNN architecture is defined by its layers, their types or dimensions, and the topology of connections, which form a computational graph. Layers can be connected either sequentially or non-sequentially. In a sequential connection, each layer directly takes the output of the previous layer as input. In contrast, non-sequential connections include more complex structures such as skip connections, branches, or shared layers.

Modern DNN models can be broadly categorized into two types based on their application domains: vision models, designed for processing image data, and large language models (LLMs), specialized for understanding and generating text. Vision models typically employ architectures that process spatial data, utilizing layers such as convolutional and pooling layers to capture local patterns and hierarchies in images. These models rely on grid-like structures to preserve spatial relationships between pixels. In contrast, LLMs are designed to process sequential data, leveraging architectures like transformers [6] with self-attention to capture dependencies across text tokens. While vision models focus on spatial feature extraction, LLMs emphasize understanding contextual relationships in sequential data. Therefore, the architecture and layer configuration of a DNN must be tailored to the specific characteristics of the data it is intended to process.

3.1.2 DNN Optimization↩︎

Modern ‘deep’ neural networks (DNN) tend to have a large number of layers. It is, therefore, essential to reduce the resource requirements for such architectures without affecting their performance. DNN optimization techniques, like pruning [55], reduce the computation overhead by removing the least important parameters \(\in \theta\). This creates a sparser architecture and has been shown to significantly decrease the size of the model with minimal impact on accuracy [55]. Similarly, neural architecture search (NAS) [56] automates discovering optimized architectures tailored to specific constraints, e.g., hardware and cost. Depending on specific constraints, DNN architectures can be adapted into optimized variants, such as MobileNetV2 [57] and MobileNetV3 [58], designed explicitly for resource-constrained hardware platforms.

3.2 GPU-Accelerated DNNs↩︎

Modern AI/ML frameworks leverage the parallel processing capabilities of GPUs [59] for DNN workloads, such as matrix multiplications and convolutions. GPUs come with advanced development toolkits, such as NVIDIA’s CUDA [53], which offer high-level APIs and libraries for general-purpose use.

3.2.1 CUDA↩︎

CUDA is a parallel computing platform equipped with a suite of tools to create and optimize programs to run on NVIDIA GPUs. It acts as an interface between AI/ML frameworks, such as TensorFlow [28] and PyTorch [29], and GPU hardware. The AI/ML frameworks define the DNN architecture and computational workflows, while CUDA translates high-level mathematical operations (e.g., convolutions) into low-level GPU functions, called kernels¹. 1 depicts the key elements involved in the GPU system stack to run a DNN using the CUDA toolkit.

CUDA maps DNN operations, represented as a computation graph, onto the GPU hardware using specialized libraries. In CUDA libraries, each kernel is designed to break down complex operations into smaller, unit-level tasks that can be executed concurrently across multiple GPU cores. For instance, matrix multiplication operations in fully connected layers use the cuBLAS (CUDA Basic Linear Algebra Subprograms) [60] library, while convolutional operations are optimized using the cuDNN (CUDA Deep Neural Network) [61] library. Activation functions like ReLU and sigmoid are executed as element-wise operations using highly parallelized CUDA kernel calls.

3.2.2 NVIDIA Profiler↩︎

nvprof [62] is a command-line tool for monitoring CUDA activities during the execution of GPU-accelerated applications. nvprof captures details like the type of kernel executed and time spent on execution. It also tracks memory-related activities, such as host-to-device and device-to-host data transfers, as well as memory allocation and deallocation events. Additionally. It monitors system-level performance metrics, including GPU clock speed, power consumption, and thermal data, to provide a comprehensive view of the application’s performance. All the collected data is compiled into a detailed report, referred to as a profile.

nvprof can be configured to generate profiles in either time series or aggregate mode. In time-series mode, it records detailed temporal data, including the execution sequence of GPU kernels, memory operations, and performance metrics associated with each kernel invocation. It is, therefore, useful for analyzing fine-grained sequences of operations.

Unlike time-series, the aggregate provides a high-level view of resource usage and performance distribution. Here, nvprof collects metrics including total time spent, number of calls, and average, minimum, and maximum execution times for each type of kernel. 2 illustrates the process of collecting and summarizing various metrics to create an aggregate GPU profile. 3 shows partial aggregated GPU profiles generated by nvprof for 10 inferences on GoogleNet [63] and RoBERTa [64]. Similarly, it collects average, minimum, and maximum values for various system-level metrics (see 4). In this paper, we will only focus on the profiles collected in the aggregate mode of nvprof.

4 Aggregate GPU Profiling↩︎

We demonstrate that adversaries can infer DNN architectures by leveraging aggregate GPU profiles as a side-channel. While these profiles are simple, non-intrusive, and coarse-grained (in contrast to time-series), they are powerful enough to reveal DNN architectures. Our work stands apart from previous works [34], [37], [38] that rely on a complex analysis of fine-grained time-series data to reconstruct DNN architectures layer-by-layer with a certain degree of uncertainty. This section highlights key insights into the aggregate CUDA kernel calls that reveal unique characteristics of DNN architectures.

4.1 Mapping CUDA Kernels to DNN Layers↩︎

By mapping CUDA kernel functions to DNN layers, it is possible to identify key characteristics of DNN architectures. For example, matrix multiplication in fully connected layers is executed by dividing the input matrix into smaller blocks and assigning them to GPU threads for parallel computation. Similarly, convolutional layers use kernels that distribute the workload across threads to compute multiple filters simultaneously. NVIDIA’s cuDNN library provides tailored functions to optimize operations like convolutions, where the computation strategy may vary based on filter size, e.g., \(3 \times 3\) vs. \(5 \times 5\) convolutions. As a result, convolution operations with different filter sizes invoke different kernels.

Specifically, kernels like volta_sgemm_AxB_tn, which handle matrix multiplications, can indicate the presence of fully connected layers and their relative sizes. Similarly, kernels such as void splitKreduce_kernel<int=A, int=B, ...>² provide insights into convolutional operations, including the size and the number of filters. Analyzing low-level kernel calls across multiple iterations can reveal and fingerprint architectural nuances specific to different DNN families. 3 highlights the most common kernels for various DNN families. These kernels, as we ranked them by relevance, reveal the architectural characteristics that distinguish vision models from LLMs, as well as their types.

4.2 Profiling CUDA Activities with nvprof↩︎

Aggregate profiling offers high-level summary metrics for kernel activities and resource utilization that can fingerprint DNN architectures. 4 compares aggregated kernel and API metrics for two popular DNN models: GoogLeNet [63] and RoBERTa [64]. A closer examination of the metrics shows that GoogLeNet exhibits higher usage of convolutional kernels like cudnn_infer_volta_scudnn_winograd_128...³, accumulating over 46.3% of all operations. The higher usage of convolution kernels aligns with the architectural characteristics of the vision models. In contrast, RoBERTa relies more heavily on specialized fully connected operations with kernels like volta_sgemm_XXXXX_tn, reflecting the matrix-heavy computations needed for large transformer blocks in language models [6].

Additionally, system-level signals from 4 further emphasize the architectural differences between LLMs and vision models. GoogLeNet’s lower average SM clock speeds and power consumption reflect its smaller size and fewer trainable parameters, i.e., \(\sim\)​6M [63]. Whereas, RoBERTa exhibits significantly higher power usage and clock rates, corresponding to its larger model size and a much higher number of trainable parameters, i.e., \(\sim\)​355M [66].

Within the same DNN family, variations in activity patterns emerge due to differences in layer depth and applied optimization strategies. For example, GoogLeNet [63] (InceptionV1) and InceptionV3 [67] are both vision models that employ branched convolutions but differ in kernel configurations and architectural complexity. In GoogLeNet, a large portion of GPU time is spent on Winograd convolution [68] kernels, with cudnn_infer_volta_scudnn_winograd_128x...⁴ accounting for 31.87% of total execution time. Other prominent operations include matrix multiplications using volta_sgemm_32x32_sliced1x4_nn (14.43%) and host-to-device memory transfers (8.87%). In contrast, the InceptionV3 profile reveals a significant portion of execution time spent in FFT-based operations like fft2d_r2c_32x32, which contributes 39.36% of execution time, and gemv2T_kernel_val, a matrix-vector multiplication kernel, accounting for 22.24%. 4 shows a scatter plot comparing kernel runtime ranks for both models. The ranking differences indicate that InceptionV3 employs more specialized and optimized kernels in place of the more generic kernels used in GoogLeNet. These distinctions in kernel usage and time distribution yield unique execution profiles, enabling fingerprinting of model variants even within the same architecture family.

4.3 Key Takeaways↩︎

Our analysis demonstrates that aggregate profiling effectively captures distinct execution characteristics of DNN families, such as convolution-heavy vision models or transformer-based language models. For instance, the divergence in kernel usage and system-level metrics between GoogLeNet and RoBERTa underscores the utility of aggregate GPU profiles in identifying architectures across diverse families. Similarly, the higher runtime contribution of FFT-based kernels in InceptionV3 compared to GoogLeNet highlights differences across variants of the same DNN family captured by GPU profiles. We argue that aggregate GPU profiles can be used as a side-channel for extracting DNN architectures. Our argument is based on two key observations:

1. Each unique combination of DNN layer type and size maps to a specific CUDA kernel call.

2. Patterns of kernel calls, along with their execution times and resource utilization, provide unique fingerprinting for DNN architectures.

5 InferNet Design↩︎

Using the aforementioned insights, we present InferNet: an attack paradigm to extract the DNN architectures using aggregate GPU profiles as a side-channel (illustrated in  5).

5.1 Threat Model↩︎

The adversary aims to conduct an accurate and cost-effective model extraction attack using a coarse-grained and non-intrusive side-channel. We assume that the adversary has black-box access to standard DNN architectures, including state-of-the-art vision and large language models, through an executable file that they can run on a GPU-enabled system. They have no prior knowledge of the target DNN architecture and its parameters, and may only query the model by providing an input \(x\) and observing the output \(y = f^\theta(x)\).

The adversary is assumed to have non-root access to the GPU-enabled system running the target model and access to profiling tools such as nvprof. While NVIDIA introduced an administrator-only access flag in later versions of nvprof to address security concerns [46], this restriction is not uniformly enforced across cloud platforms. Cloud environments may deploy different versions of the CUDA Toolkit—e.g., CUDA 10.2 uses nvprof 10.2, CUDA 11.0 uses nvprof 11.0, and CUDA 11.1 onward migrates to Nsight Systems—with the specific profiler version depending on the provider’s deployment choices. In some deployments, these environments allow profiling by default for non-root users. Additionally, even when newer profilers enforce access restrictions, legacy CUDA toolchains may remain usable in containerized user-space environments [41]. Accordingly, InferNet applies to deployments in which aggregate GPU profiling remains available to the attacker due to platform configuration, legacy software stacks, or equivalent profiling exposure; we do not claim that this access model is universal across all MLaaS platforms.

We assume that the adversary does not tamper with system assets: software or hardware. The attack does not require access to the PCIe bus, operating system (OS), or any internal system components. This makes our attack non-intrusive, which relies solely on profiling data generated during the execution of the target model.

Why aggregate profiles? Aggregate profiles are less sensitive to instrumentation differences across profiler versions. Aggregation suppresses noise arising from non-deterministic GPU scheduling across multiple processes and kernels—GPU’s internal scheduling policies may vary between runs, affecting timing and execution order. Aggregate profiles also mitigate variability introduced by thermal throttling and power management, where dynamic changes in performance states (P-states) alter runtime characteristics across inferences. Additionally, aggregate metrics average out fluctuations caused by internal resource contention, such as shared memory and SM-level warp scheduling. These effects introduce instability in fine-grained (time series) profiles, whereas aggregate data provide a more stable and representative fingerprint. Moreover, our feature selection process further reduces the influence of residual noise by retaining only the most predictive components of the profile, enabling high inference accuracy (see 6) with minimal complexity.

5.2 Architecture Extraction Attack↩︎

We formulate a model extraction attack where an adversary infers the architecture of a victim model \(f_v^\theta\) with architecture \(f_v\) using aggregate GPU profiles \(\mathcal{H}(f_v^\theta)\) as a side-channel. Our attack methodology includes an offline preprocessing phase that includes collecting GPU profiles, cleaning the dataset, and training the architecture prediction model, followed by an online attack phase (see 5).

Experiment Setup. We take a diverse set of 90 candidate DNN architectures, including widely used LLMs (e.g., GPT [4], BERT [5]), vision models (e.g., ResNet [69], GoogLeNet [63]), and vision transformer models (e.g., MaxViT [70], ViT [71]). The candidate models are pretrained and sourced from two major AI/ML frameworks: TensorFlow (v2.17.1) [28], and PyTorch (v2.3.0) [29], where 24 models are implemented in both of the frameworks. We import TensorFlow vision models from tf.keras.applications, and LLMs from HuggingFace’s AutoModels [66]. Similarly, for PyTorch, we import models from TorchVision (v0.18.0) and TorchText (v0.18.0). The complete list of the models used in this paper is provided in 10.

To further enhance the robustness of our approach, we conduct our experiments on two different NVIDIA GPUs: Tesla T4 [72] on Google Colaboratory [73], and Quadro RTX 8000 [74] on a local server (running Ubuntu 24.04). By using two separate experiment setups, we demonstrate that our approach is generalizable to different hardware and software platforms.

A complete summary of the experimental settings used for architecture prediction, including profile collection, feature engineering, classifier configuration, and evaluation metrics, is provided in 11.

Collecting Aggregate Profiles. We define a candidate architectures set \(C = \{f_{c_1}, f_{c_2}, ... f_{c_n}\}\) comprising popular large language models (LLMs) and vision models. For each candidate \(f_{c_i} \in C\), we collect \(N = 20\) aggregate profiles \(\mathcal{H}(f_{c_i}^{\theta_i})\) using nvprof in aggregate mode to construct a data sample \(S_i = \{\mathcal{H}(f_{c_i}^{\theta_i}), f_{c_i}\}^N\). Every profile contains combined metrics, such as kernel runtimes, memory operations, and system-level resource utilization, for ten (10) inferences with zero input. We avoid batch execution to reflect realistic attack conditions where the batch sizes are unknown to the adversary.

We consolidate all the profiles from the candidate architectures into a comprehensive training dataset \(D = \bigcup_i S_i\). With 90 candidates and 2 GPU platforms, our dataset accumulates \(|D| = 90 \times 2 \times 20 = 3600\) GPU profiles representing 36,000 inference runs. This dataset represents the computational characteristics of the architectures in the candidate set \(C\), where each kernel name serves as a distinct feature for the architecture’s prediction model.

Data Cleaning. In the combined dataset \(D\), some GPU kernels and memory operations appear only for specific architectures; therefore, creating a sparse dataset. We address this by adding binary indicator columns for each kernel that denote the kernel’s presence or absence in a given profile. Then, we fill any missing values with the mean of the corresponding feature across the dataset. Finally, we account for the different magnitudes of metrics by normalizing and scaling all the features in the dataset.

We perform feature selection to improve the signal-to-noise ratio and reduce the risk of overfitting. In this regard, we use Recursive Feature Elimination (RFE) to identify and retain the most relevant features. The resulting trained prediction model creates a more reliable mapping between aggregate profiles and the target architecture \(f_{v}\).

Training an Architecture Prediction Model. We train an architecture prediction model \(\mathcal{A}\) on the clean dataset \(D\) to learn the mapping between aggregate GPU profiles \(\mathcal{H}\) and their corresponding architectures \(f_{c_i}\). Formally, \(\mathcal{A}\) attempts to solve the problem: \[\label{eq:a} \mathcal{A}( \mathcal{H}(f_{v}^{\theta})) = \mathbb{E} [ f_{v} | \mathcal{H}(f_{v}^{\theta}) ] = \hat{f}_{v}\tag{1}\] where \(\hat{f}_v\) is the predicted architecture of the victim model, and \(\mathcal{A}\) is typically a machine learning model. The training process uses simple supervised classifier models, i.e., Logistic Regression, Neural Network, KNN, Nearest Centroid, Naive Bayes, Random Forest, and AdaBoost. Using multiple models allows us to optimize the attack for different architectures.

Online Attack. The online attack phase focuses on extracting the architecture of an unknown victim model. We use a binary executable to run the victim model \(f_v^\theta\) on a GPU-enabled system, and use nvprof to generate an aggregate profile \(\mathcal{H}(f_v^\theta)\). This profile is fed into the trained prediction model \(\mathcal{A}\), which outputs a predicted architecture \(\hat{f}_{v} = \mathcal{A}( \mathcal{H}(f_{v}^{\theta}))\).

6 InferNet Attack Evaluation↩︎

In this section, we assess the accuracy of InferNet across various attack scenarios. We begin by evaluating a practical attack scenario using vanilla settings. Next, we optimize the attack, enhancing its resilience across diverse use cases. Finally, we demonstrate the scalability of the attack with respect to future DNN architectures. For model training, unless otherwise stated, we used the default hyperparameters of the corresponding scikit-learn implementation, with the exceptions listed in 11.

6.1 Practical Attack↩︎

We begin by evaluating whether the aggregate GPU profiles can reveal DNN architectures in a practical scenario with standard settings. In this setting, the victim model \(f_v^\theta\) and candidate model \(f_{c_i}^{\theta_i}\) have the same number of output classes. The profiles are split into two datasets: \(D_1\) for training (75%) and \(D_2\) for testing (25%). Dataset \(D_1\) represents profiles collected during the offline preprocessing phase of the attack, while \(D_2\) simulates victim profiles collected for the online attack. We stratify the data by the DNN architecture names to ensure that each DNN architecture is represented in both the training and test datasets. We train seven (7) architecture prediction models \(\mathcal{A}\) on \(D_1\) and evaluate their performance on \(D_2\). Each model is trained using different subsets of profile features (e.g., GPU kernel features, API call features) as shown in 5.

Key Findings. Our key findings are (refer to 5):

Effectiveness of architecture prediction models: The results confirm that aggregated profiles can effectively reveal victim architectures in a common scenario. Most models, including Neural Network, K Nearest Neighbors, Naive Bayes, and Random Forest, achieve near-perfect accuracy (\(\ge 96\%\)) on \(D_2\) when trained on all 849 features. Models like Random Forest, Naive Bayes, and K Nearest Neighbors exhibit robustness even when trained on reduced feature sets.

Impact of feature selection: GPU kernel features are the most critical for architecture prediction. Models trained only on GPU kernel features achieve accuracy comparable to those trained on all features. API call features and system-level metrics contribute less, as training exclusively on these features results in significant accuracy drops.

Overfitting on system features: We observe spurious correlations in system-level metrics that do not generalize across datasets. Models trained only on the system features overfit on the training data \(D_1\). For example, K Nearest Neighbors achieves 100% accuracy on \(D_1\) but drops to 47.3% on \(D_2\). Similarly, the accuracy for Random Forest, the best model overall, drops from 100% to 61.3%. These results suggest that system-level metrics, may reflect specific runtime conditions, hardware configurations, or workload distributions that are unique to the input data and not the DNN architecture.

Impact of memory management kernels: Memory-related GPU kernels, such as CUDA_memcpy_HtoD, are less informative for architecture prediction. Removing memory management features from the GPU kernel subset causes minimal (\(\le 1\%\)) accuracy loss for all models except AdaBoost. This highlights that memory operations contribute little to distinguishing architectures.

6.2 Attack Optimization for Sparse Data↩︎

Since not all features strongly indicate the DNN architecture, we investigate how small a subset of features is sufficient for high prediction accuracy. We focus primarily on the 450 non-memory GPU kernel features (see Finding 2 in § 6.1). Using Recursive Feature Elimination (RFE) [75], we rank the kernel features based on their importance and train the architecture prediction models \(\mathcal{A}\) on \(D_1\) from the baseline attack (6.1) using only the top features.

Key Findings. Our results, illustrated in 6, validate the hypothesis that a small number of features can achieve comparable accuracy to models trained on the full feature set. These findings highlight the importance of feature selection in choosing a minimal feature subset, mitigating overfitting, and reducing computational overhead.

The top features are highly effective: Models like K Nearest Neighbors, Nearest Centroid, Naive Bayes, and Random Forest achieve the same high accuracy (\(\sim\)​100%) using only the top 3 features as they do with over 450 features. The top three features, shown in 3, include Time (ms) volta_sgemm_128x64_tn, Avg (us) volta_sgemm_128x64_tn, and Time (ms) void splitKreduce_kernel<int=A, int=B, ...>⁵.

Feature requirements vary by prediction model \(\mathcal{A}\): Neural Networks and Logistic Regression require more features, i.e., 17 and 22, respectively, to achieve peak accuracy. This suggests that not all models capture the same activity patterns and therefore prioritize different feature sets for predictions.

The top-kernel features are informative because the paper’s mapping analysis ties kernels to layer structure and variant-specific execution behavior; therefore, feature selection suppresses environment-specific noise rather than discovering arbitrary correlations.

6.3 Attack Resilience Against Modified Architectures↩︎

We extend our attack to a more challenging scenario where the victim model \(f_v^\theta\) is modified by additional final layer(s) with a different number of output classes than the original base model. We simulate this scenario by taking a subset of \(D_1\prime\) from  6.1, containing only the 56 TorchVision models. The \(D_1\prime\) dataset represents candidate models pretrained on ImageNet [76] with 1000 output classes. However, we construct a new dataset \(D_3\) with vision models trained in CIFAR10 [77] with only 10 output classes.

Key Findings. For evaluation, we train the prediction models \(\mathcal{A}\) on \(D_1\prime\), and test them on \(D_3\). The features used for training include the top 3, top 25, and all 450 non-memory GPU kernel features identified in § 6.2. Our findings, from 6, are as follows:

High correlation with top features: The top 3 features: (i) Time (ms) volta_sgemm_128x64_tn, (ii) Avg (us) volta_sgemm _128x64_tn, and (iii) Time (ms) void splitKreduce_kernel<int=A, int=B, ...>⁶, continue to show strong predictive power. This suggests that these kernel features are highly correlated with the architectural characteristics of DNNs and are robust to variations in the number of output classes.

Performance gain with top 25 features: Expanding the feature set to the top 25 features improves the performance of Logistic Regression and Neural Network models, with accuracy on the test set \(D_3\) of 100%, and 95.2%, respectively. Therefore, with additional features available, a broader range of models can be used for architecture prediction with high confidence.

Overfitting with all features: Using all 450 non-memory kernel features introduces overfitting for models that prioritize memory as well as non-memory kernels. In particular, the accuracies for Logistic Regression and Neural Networks drop to 58.7% and 61.4%, respectively, on \(D_3\). This performance is far lower than their predictions using all GPU kernels (see 5), with accuracies of 99.4%, and 99.9%, respectively. Random Forest, however, remains robust and consistently achieves 100% accuracy.

The results confirm that our attack method remains effective even when the DNN is fine-tuned or modified. This demonstrates that GPU kernel features serve as a reliable side channel for extracting DNN architectures, regardless of changes to the model architecture.

The high predictive value of the top-ranked features is consistent with the kernel-to-layer mapping discussed in Section 4: these kernels are not arbitrary correlates, but compact summaries of architecture-dependent operations. At the same time, our results show that indiscriminate inclusion of many low-value features can reduce transfer performance, which is why we treat full-feature results as an ablation rather than as a primary attack configuration.

6.4 Scaling Attack to Future Architectures↩︎

Adding new candidate DNN architectures to the attack’s scope requires additional profiles and significant computation for retraining the prediction model \(\mathcal{A}\). For reference, generating a single profile for DNN execution takes approximately 45-120 seconds, and generating the complete dataset \(D_1\) requires 20+ hours. In contrast, the online attack phase is computationally inexpensive, with less than one minute needed to profile the victim model and predict its architecture. Reducing the time spent in the offline preprocessing phase is critical for enhancing the practicality of the attack.

We analyze the computational efficiency of the InferNet attack by reducing the training dataset size – and therefore, the time for collecting profiles – without compromising the accuracy. We hypothesize that fewer profiles per architecture are sufficient for training accurate architecture prediction models. To test this, we train the prediction models \(\mathcal{A}\) on subsets of \(D_1\prime\) with a varying number of profiles per candidate architecture. We limit the training to the top 25 GPU kernel features identified in 6.3. Finally, we test \(\mathcal{A}\) on \(D_3\) (profiles of victim models trained on CIFAR10) to evaluate the impact of dataset size reduction.

Key Findings. 7 confirms that architecture prediction models can generalize effectively with minimal training data. For all models except Logistic Regression and AdaBoost, a single profile per candidate architecture is sufficient to achieve the same accuracy as training on the full dataset of 20 profiles. Similarly, these models show very low variance for the top 25 features which indicates that repeated executions of DNN models often result in identical GPU profiles.

Reducing the training data requirements offers significant practical advantages. Collecting just one profile per architecture reduces the offline preprocessing phase to under 30 minutes (i.e., 97.5% time reduction). Similarly, adding a new architecture to the candidate set requires only one additional profile or just over one minute to fine-tune the attack. These findings highlight the scalability of InferNet and its adaptability to expanding candidate model pools.

7 Generalizability Evaluation of InferNet↩︎

While our evaluation results in Section 6 indicate that InferNet is reliably architecture-agnostic; it is strongest when the attacker can match or approximate the victim’s software and hardware stack. This section evaluates the generalizability of InferNet across various deployment scenarios, including pruned DNNs, different GPU hardware, and multiple AI/ML frameworks.

7.1 Evaluation Over Pruned Models↩︎

DNNs are often pruned to optimize inference performance and reduce computational overhead. To evaluate whether InferNet is effective against such models, we create a dataset \(D_p\) using profiles of 56 TorchVision models trained on CIFAR10 [77] and pruned to 50% capacity. Architecture prediction models \(\mathcal{A}\) are trained on the top 25 non-memory GPU kernel features from the unpruned dataset \(D_1\) (from § 6.1) and tested on \(D_p\). This setup also simulates the scenario where the DNN architecture is modified and fine-tuned with a different number of output classes (see §6.3).

Key Findings. The results, as shown in 7, demonstrate that InferNet generalizes well to pruned models with minimal performance degradation. Random Forest and Naive Bayes models achieve 100% architecture prediction accuracy. However, the rest of the models show a slight performance decline from the non-pruned test set (see 6). Performance for some of the models, like Logistic Regression and Neural Network, improves significantly in Top5 evaluation, indicating that the true architecture is often among the top five predictions. We also include DNN Family prediction to analyze if the prediction models \(\mathcal{A}\) recognize the DNN family, if they fail to recognize the architecture itself. Most models, except AdaBoost, deliver reliable performance, with many achieving 100% accuracy in predicting the DNN family.

7.2 Evaluation Across GPUs↩︎

In practice, a diverse range of GPUs is available for training and running DNN models. Therefore, it is crucial to establish the effectiveness of the attack when the adversary does not have easy access to the same GPU for both offline and online phases.

We collect two datasets \(D_2\prime\) and \(D_3\prime\) using profiles NVIDIA Quadro RTX 8000 GPU; whereby, \(D_2\prime\) contains profiles of candidate models trained on ImageNet [76] and \(D_3\prime\) contains profiles of models trained on CIFAR10 [77]. Additionally, we re-use the datasets \(D_1\prime\), \(D_2\), and \(D_3\) collected from NVIDIA Tesla T4 GPU in § 6.1 and § 6.3. Both datasets use the top 25 non-memory GPU kernel features for training and testing. For cross evaluation, training data consists of profiles from one GPU, while the test data comes from the other GPU, and vice versa.

Key Findings. The results from 8 show that InferNet achieves varying levels of accuracy across GPUs. For example, Logistic Regression achieves only 34.1% Top1 accuracy when trained on Quadro and tested on Tesla, and even lower accuracy when reversed. Neural Network accuracy is similarly low, at 10.7% and 12% Top1 accuracy in both cross-GPU scenarios, respectively. However, models like Naive Bayes and Random Forest demonstrate stronger cross-GPU transfer across the two evaluated NVIDIA GPUs, with Naive Bayes achieving 71.4% Top1 accuracy when trained on Quadro and tested on Tesla, and 74.6% when trained on Tesla and tested on Quadro. Interestingly, even when architecture prediction accuracy drops, family prediction accuracy remains relatively high for most models.

Our hardware selection of Tesla T4 and Quadro RTX 8000 represents two widely used deployment tiers within NVIDIA’s Turing generation: a data-center accelerator and a high-end workstation GPU. The evaluation reinforces that InferNet exploits fundamental architectural behaviors rather than specific hardware instance noise. Successfully transferring the attack between two vastly different operational tiers of the Turing architecture, i.e., the T4 and the RTX 8000, indicates that the aggregate profile acts as a robust, hardware-agnostic side-channel within an architectural generation.

Analysis of the combined data shows that the GPU kernels used by DNN architectures are largely consistent across GPUs. This consistency enables some generalizability. However, the observed performance drop highlights the need for further tuning or additional training data to improve the cross-GPU transferability of InferNet.

7.3 Evaluation Across ML Frameworks↩︎

Given the effectiveness of InferNet in various scenarios, the next important question is whether the adversary must know the framework used to implement the victim model or if architecture prediction models can be generalized across frameworks. To evaluate this, we create a dataset of GPU aggregate profiles for 24 vision DNNs implemented in TensorFlow v2.17.1 (see 10). We choose those DNNs that are also available in PyTorch, and they are part of our \(D_1\prime\) profiles dataset. To collect 20 profiles each for the TensorFlow models to construct a dataset \(D_t\). For evaluation, we train using profiles from one framework and test with profiles from the other framework, and vice versa. The results indicate that TensorFlow DNNs utilize a largely disjoint set of GPU kernels compared to PyTorch. This results in a significant accuracy drop when using PyTorch-trained models to predict TensorFlow architectures (see 9).

In cross-framework evaluations (PyTorch to TensorFlow), while raw Top-1 accuracy drops, the security implication remains severe. The ability to identify the model family (e.g., VGG) reduces the architectural search space for an adversary from thousands of possibilities to a handful of known variants. This facilitates targeted model-stealing, as the adversary can focus their resources on a narrow set of probable architectures once the family is confirmed.

Optimization. To address the poor performance, we create a heterogeneous dataset by appending PyTorch profiles with the TensorFlow profiles. Since these two frameworks are the most prevalent, this dataset generation step is typically required only once, as it is unlikely that a new framework will significantly challenge their importance. With heterogeneous training data, Random Forest achieves 100% Top1 accuracy, while Naive Bayes and K Nearest Neighbors exceed 95% (see 10). This confirms that adding profiles from multiple frameworks can effectively remove the dependency on a specific framework and, therefore, ensure generalization across frameworks.

8 Proposed Countermeasures↩︎

To defend against side-channel architecture extraction attacks like InferNet, our recommendations to disrupt or mitigate the attack surface include:

Dummy Computations: Adding dummy GPU workload—such as no-op kernels or randomized matrix operations—can disrupt the signal-to-noise ratio in the side-channel data [25], [33], [34], [37], [38], [78]. Such computations should vary across inference runs to prevent adversaries from filtering them out via differential analysis. This defense impacts various GPU kernel features, such as average runtime, total runtime, and kernel invocation counts. Therefore, it requires careful design, as perturbing some features (e.g., average time spent per kernel invocation) can incur significant performance overhead.

Randomized Kernel Scheduling: Introducing randomized scheduling between dependent or independent GPU kernels can distort execution time distributions without altering model behavior. Such noise makes it harder for adversaries to associate specific runtime patterns with architectural components. Timing jitter and minor delay injection, implemented at the framework or driver level, can increase feature variance and reduce prediction accuracy for attacks relying on stable kernel profiles.

Limited Precision in GPU Profiling Tools: Restricting the precision of profiling information (e.g., truncating runtime metrics or kernel counts) can reduce the value of GPU profiles to adversaries [41], [46]. This approach is especially effective against statistical learning methods, which rely on feature importance derived from fine-grained runtime behavior. However, tuning the defense requires careful assessment of the impact on the legitimate use of profiling tools for debugging and performance optimization.

Monitor and Restrict Access to Profiling Tools: Access to tools like nvprof should be restricted to trusted users with administrative privileges [41], [46]. Enabling administrator-only flags in profiling tools can prevent adversaries from leveraging these tools for side-channel attacks. Similarly, inference services should audit profiler invocations and implement rate-limiting or sandboxing to detect repeated or automated profiling attempts.

9 Conclusion↩︎

This paper presents InferNet, a novel attack method that uses simple, non-intrusive GPU performance metrics to predict the underlying DNN architecture of a victim application. Unlike traditional model extraction techniques, which rely on fine-grained data or full access to the model’s parameters, InferNet only requires high-level system metrics such as GPU kernel calls, memory usage, and clock speed, making it both practical and stealthy. Our empirical evaluation demonstrates that InferNet achieves high accuracy across a diverse set of DNN architectures, including vision models, large language models, and transformers, with minimal data and even under model compression or optimization. The method is robust across multiple AI/ML frameworks and GPU platforms, providing a powerful tool for identifying DNN architectures without requiring root access or detailed model parameters.

10 Model List↩︎

11 Implementation Details and Hyperparameters↩︎

We used a common preprocessing pipeline across all prediction models: feature-wise mean imputation, binary indicator augmentation for sparse kernels/events, z-score normalization, and stratified train/test splitting by architecture label. For sparse-feature experiments, features were ranked with RFE and truncated to the top-3, top-25, or full 450 non-memory GPU-kernel features, depending on the experiment. Tables 13–18 summarize the experimental settings used for the architecture-prediction component of InferNet.

Note. We report only the settings relevant to the architecture-prediction results in the main paper. Broader end-to-end extraction settings not directly used in the reported architecture-inference evaluation are omitted for clarity.

References↩︎

---

1. A CUDA kernel, not to be confused with an OS kernel, refers to a function executed concurrently by multiple GPU threads.↩︎

2. Truncated long kernel name.↩︎

3. Truncated long kernel name.↩︎

4. Truncated long kernel name.↩︎

5. Truncated long kernel name.↩︎

6. Truncated long kernel name.↩︎