1 Introduction↩︎

The last decade has witnessed an unprecedented proliferation of intelligent embedded devices, from wearables monitoring vital signs to industrial sensors detecting early signs of mechanical failure. At the heart of this trend lies Tiny Machine Learning (TinyML): the discipline of deploying machine learning algorithms directly on highly constrained hardware, such as Microcontroller Units (MCUs), operating under tight budgets of memory, computation, and energy [1]. By pushing ML inference to the extreme edge of the network (i.e., into the devices themselves, rather than delegating computation to remote servers) TinyML enables applications that are responsive, privacy-preserving, and functional even in the absence of network connectivity [2]. Historically, the dominant paradigm in TinyML has been to separate the learning and inference phases entirely: models are trained on external hardware and subsequently deployed to the device, which is then responsible solely for running predictions. This separation keeps the computational burden on the device to a minimum, but it comes at a cost. Models trained offline on curated datasets frequently underperform once exposed to real-world conditions, as the data encountered after deployment rarely matches the distribution seen during training [3], [4]: in other words, the data distribution changes after deployment. These shortcomings have motivated a growing body of work under the umbrella of On-Device Learning (ODL) [5], which seeks to execute not just inference but the learning process itself on the device, with the objective of adapting models to the distribution changes that occur after deployment.

Realizing this vision on TinyML hardware, however, requires confronting three constraints that together define ODL as a fundamentally distinct research challenge: memory and compute budgets must accommodate the learning process alongside inference, labeled data is scarce and costly to obtain on deployed devices, and solutions cannot be validated before they begin operating in the field. Despite these constraints, ODL substantially widens the scope of what TinyML systems can do: it enables fine-tuning to specific deployment conditions [6], personalization to individual users or contexts [7], and continuous adaptation to non-stationary data streams [8], [9]. The development of adaptive TinyML solutions is therefore an increasingly active research area, and one in need of careful analysis. However, despite the centrality of distribution change as the fundamental motivation for ODL, the existing literature has devoted surprisingly little effort to characterizing the different types of change that can occur after deployment, and prior surveys do not make explicit that different solutions operate under fundamentally different distribution change regimes. Motivated by this observation, we present a comprehensive survey of the Tiny ODL literature that, for the first time, places the nature of the distribution change at the center of its structure and analysis, using it as the primary lens through which solutions are characterized, compared, and evaluated.

Concretely, we distinguish three types of distribution change regimes. In the single-change regime, the distribution shifts exactly once at deployment and then remains stable: the challenge is to adapt quickly and efficiently to the new conditions, so that effective inference can be performed for the rest of the device’s life. In the concept drift regime, the distribution shifts continuously and unpredictably throughout the operational life of the device, requiring the solution to adapt quickly to each new condition while discarding knowledge that is no longer relevant. In the continual learning regime, new concepts are introduced incrementally over time, and the solution must accumulate new knowledge without forgetting what it has previously learned, a challenge known as catastrophic forgetting [8]. Together, these three regimes cover all the scenarios addressed in the ODL literature, and they serve as the primary organizing axis of this survey.

These three settings place fundamentally different demands on the applications that can be addressed and on the learning algorithm and the hardware we use to address them, and important patterns only emerge when works are compared within the same regime. For each regime, we analyze the literature across three complementary lenses: the application contexts each regime naturally accommodates, the hardware platforms suited to its computational demands, and the technical composition of the solutions it requires. Beyond characterizing each regime individually, this structure exposes patterns that cut across all three regimes: first, which application domains tend to drive the use of particular hardware platforms, and second, which hardware constraints rule out entire families of learning algorithms, leaving only a narrow set of viable approaches for a given device.

The rest of the paper is organized as follows. Section 2 positions the survey with respect to prior work. Section 3 formalizes the ODL problem, defining the application requirements, device constraints, and solution components that constitute the three analytical lenses through which the literature is surveyed in Sections 4, 5, and 6 respectively. Section 7 discusses open challenges and future directions, and Section 8 concludes the paper.

2 Scope and Positioning↩︎

Several surveys have recently addressed adjacent topics, but none covers the same combination of scope and perspective: prior works either target a broader hardware range than MCU-class devices or organize solutions along a technical axis that cuts across the distribution change regimes introduced here. Table 1 summarizes the key differences between related surveys and the present work.

Rajapakse et al. [5] is the closest in spirit, surveying both Over-The-Air (OTA) updates and ODL approaches for TinyML. Their taxonomy organizes ODL solutions along a technical axis, distinguishing offline learning from online learning approaches. While useful for characterizing algorithmic families, this axis cuts across the distribution change regime a solution operates under: a solution in which the distribution changes only once at the moment of deployment and one in which it can continuously change are grouped together, despite solving fundamentally different problems under fundamentally different constraints. Furthermore, the survey was published in 2023 and does not capture the substantial body of work published since. As visible in Figure 1, most of the works present in the literature were published after 2023, highlighting the growing relevance of the field. Other more recent works are present in the literature, but they are less close in scope with respect to [5]. Abozaid et al. [10] focus narrowly on the model-related aspects, offering detailed accounts of individual papers without a unifying taxonomic framework. Lourenço et al. [11] restrict their scope to data stream learning for IoT, covering neural networks and decision trees as relevant model families, but without a focus on truly constrained hardware and without considering the hardware and application aspects.

The present survey differs from prior work in three respects. First, it is strictly scoped to MCU-class hardware and below, keeping hardware constraints central to the analysis. Second, with approximately 70 works surveyed, it covers a substantially larger portion of the literature than any prior effort. Third, and most distinctively, it organizes the entire literature around the distribution change regime, distinguishing between single-change, concept drift, and continual learning regimes, and analyzes each through three complementary lenses: application, hardware, and technical composition of the solutions.

2.1 Literature Search and Inclusion Criteria↩︎

The literature search was conducted using Google Scholar as the primary database, complemented by IEEE Xplore, ACM Digital Library, and arXiv. The search combined several keyword groups, including “On-Device Learning",”On-Device Training", “Online Learning",”Continual Learning", “Transfer Learning", and”Learning in the presence of concept drift", each paired with the terms “tiny",”MCU", or “IoT" to restrict results to the TinyML domain. The references of selected works were iteratively reviewed to identify additional relevant contributions not captured by the initial keyword search.

Works were included in the survey if they satisfied the following criteria:

• The work targets hardware within the TinyML scope. Concretely, this corresponds to MCU-class devices (e.g., ARM Cortex-M series) or below, operating under tight RAM memory (in the order of the MB), frequency (<500 MHz), and power (<500 mW) budgets. Application-class processors (e.g., ARM Cortex-A series, as found in devices such as the Raspberry Pi) were considered outside this scope, unless the work explicitly frames its solution around the constraints of MCU-class deployment.

• The work addresses the learning phase of a machine learning algorithm, not inference only.

• The learning phase is executed or designed to be executed on the target device, not exclusively on external hardware.

3 Problem Formulation for On-device Learning under distribution changes↩︎

This section formalizes the on-device learning problem under distribution change, establishes the vocabulary and conceptual framework used throughout the survey, and defines the three distribution change regimes that categorize the surveyed works across the application, hardware, and solution lenses. A table summarizing the notation used in this section is provided in the appendix 9.

The goal of the ODL problem is to obtain a solution \(A{}\)¹ addressing an application, which comprises and enables the on-device inference and learning of a machine learning model over data drawn from a data-generating process \(P{}\) directly on a target device \(D{}\). The nature of \(P{}\), the requirements of the application, and the constraints of \(D{}\) together determine the space of viable solutions, and are formally characterized in the following subsections.

Section 3.1 characterizes \(P{}\) and introduces the distribution change regime taxonomy that serves as the primary organizing principle of this survey: the nature of the change in \(P{}\) after deployment determines which regime a solution operates under. Section 3.2 formalizes the requirements that an application may impose on the solution and on the choice of \(D{}\). Section 3.3 characterizes \(D{}\) and the constraint it imposes on \(A{}\). Section 3.4 introduces the component decomposition used throughout the technical analysis of the solutions, together with the criteria for evaluating \(A{}\) in each regime.

3.1 The data generating process \(P{}\) and the distribution change regimes↩︎

Let \(P{}\) be a data generating process that, at each time instant \(t\), provides a pair \((x^t, y^t)\) sampled from an unknown probability distribution \(p^t(x, y)\), where \(x\) is the data collected from an on-device sensor (e.g., an image or an audio clip) and \(y\) its classification label. Without loss of generality, the supervised information \(y^t\) might not be available at every time instant, reflecting the practical reality of TinyML deployments where labeled data is scarce and learning episodes are intermittent rather than continuous.

We assume that there exists a time instant \(T{}\) when the proposed solution is deployed on an embedded device \(D{}\). We assume that before \(T{}\), the unknown distribution is stationary: \(\forall\, t < T{} \colon p^t(x, y) = p^{t+1}(x, y)\). This property might not hold for \(t \geq T{}\), since the unknown distribution might change after deployment. It is worth noting that the change in \(p^t(x, y)\) might affect the input \(x\) (e.g., by the introduction of noise), the set of classes (e.g., class change), or the relationship between \(x\) and \(y\). The set of realizations \(S_0= ((x^0, y^0), \ldots, (x^{T{}-1}, y^{T{}-1}))\) obtained from \(P{}\) before deployment constitutes a dataset that can be used for pre-training or initializing the various components of the solution \(A{}\), while the set of realizations \(S{} = ((x^{T{}}, y^{T{}}), \ldots)\) represents a stream of data arriving at the device after deployment, which can be used for the adaptation of the solution.

3.1.1 Distribution change regimes↩︎

Depending on whether and when a change in the distribution of \(P{}\) occurs during \(S{}\), we can distinguish among three distribution change regimes, which together cover all the works present in the ODL literature:

• Single-change regime: The distribution of the data-generating process changes exactly once, at the moment of deployment \(T{}\), i.e., \(p^{T{}-1}(x, y) \neq p^{T{}}(x, y)\) and \(\forall\, t > T{} \colon p^t(x, y) = p^{t+1}(x, y)\). Note that this regime also encompasses solutions trained entirely on-device without any pre-deployment data, i.e., solutions with \(S_0{} = \varnothing\). A typical example is a keyword spotting system that enrolls a user-defined wake word at deployment time and requires no further adaptation thereafter [12].

• Concept drift regime: The distribution of the data-generating process changes multiple times after deployment, i.e., \(\exists\, t > T{} \colon p^{t-1}(x, y) \neq p^{t}(x, y)\). Changes in this regime are typically frequent and the focus is on designing solutions that adapt quickly to the new distribution, discarding or downweighting knowledge acquired under previous distributions. A typical example is a pressure sensor calibration model that continuously tracks measurement drift caused by environmental changes [13].

• Continual learning regime: This regime shares the same formal definition as the concept drift regime: \(\exists\, t > T{} \colon p^{t-1}(x, y) \neq p^{t}(x, y)\). The distinction is not in the properties of \(P{}\), but in the assumptions made about the relevance of past knowledge and in the criteria used to evaluate solutions. Changes in this regime are typically less frequent and correspond to the introduction of genuinely new concepts or tasks. The focus is therefore on designing solutions that acquire new knowledge without forgetting previously learned information, a challenge commonly referred to as catastrophic forgetting [14]. A typical example is an EEG-based brain-machine interface that incrementally learns to recognize new motor commands while retaining the ones learned previously [15].

While the concept drift and continual learning regimes share the same formal characterization of \(P{}\), treating them as a single category would group solutions designed under incompatible assumptions about the relevance of past knowledge, making direct comparison between them meaningless. The distinction is therefore maintained as a primary organizing principle throughout this survey.

The stationary case, in which no change occurs at any point in time, i.e., \(\forall\, t > 0 \colon p^{t-1}(x, y) = p^{t}(x, y)\), is explicitly excluded from this survey. In such a setting, ODL provides no advantage over the train-then-deploy paradigm: any performance gain could be achieved more reliably by collecting additional labeled data and retraining offline. ODL is therefore only meaningful, and only surveyed here, in the presence of a change in \(P{}\) after \(T{}\).

3.2 The Application requirements↩︎

An application is characterized by a data-generating process \(P{}\), by the task it addresses (e.g., classification, regression ...), and optionally by two technical requirements on the deployment: a power consumption requirement, i.e., the power drawn by \(D{}\) must remain within a budget imposed by the energy source and form factor of the deployment², and an execution time requirement, i.e., each prediction must be produced within \(\overline{E{}}{}\) time units of the corresponding observation \(x^t{}\) arriving at \(D{}\): \[\forall t> T{} \quad E{}\!\left(A{}^{t{}}, D{}\right) \leq \overline{E{}}{},\]

Where \(E{}\!\left(A{}^{t{}}, D{}\right)\) denotes the execution time of \(A{}\) on \(D{}\) at time \(t{}\). Satisfying these requirements involves two degrees of freedom: the choice of \(D{}\) and the design of \(A{}\). The large majority of works in the literature fix \(D{}\) first and optimize the predictive performance of \(A{}\) subject to the resulting constraints³. The different applications addressed in the literature are analyzed in detail in Section 4.

3.3 The Tiny Device \(D{}\)↩︎

Let \(D{}\) be the target device on which \(A{}\) is executed, chosen or designed to meet the application requirements introduced above. \(D{}\) is characterized by its RAM capacity \(M{}_{D{}}{}\), which bounds the amount of data, intermediate activations, and learnable parameters that can reside in memory simultaneously during execution, and by its clock frequency \(F_{D{}}{}\), which jointly determines \(E{}\) alongside the algorithmic complexity of \(A{}\). Once \(D{}\) is chosen, it imposes its own constraint on \(A{}\) through \(M{}_{D{}}{}\): \[M{}_{A{}}^{t{}} \leq M{}_{D{}}{},\] which must be satisfied at every time instant \(t{} \geq T{}\). Note that \(M{}_{D{}}{}\) denotes the total on-chip RAM of the device; the memory actually available to \(A{}\) is strictly smaller, as firmware, the system stack, and runtime overhead consume a non-negligible share of it. Nevertheless, most surveyed works treat \(M{}_{D{}}{}\) as the effective memory budget for the solution, and we follow this convention throughout the survey. \(D{}\) is further characterized by a flash memory capacity, which determines the amount of static data, such as frozen model parameters and program code, that can be stored on the device. While flash capacity constrains the overall size of \(A{}\), peak RAM is the binding feasibility constraint for ODL: unlike flash, which is written once at deployment, RAM must simultaneously accommodate the model parameters, the input data, the intermediate activations of the learning mechanism, and the data management buffer during the learning phase. Peak RAM is therefore the primary focus of the memory analysis throughout this survey. The different devices employed in the literature are analyzed in detail in Section 5.

3.4 The Components of an ODL Solution and its evaluation↩︎

Any ODL solution in the surveyed literature can be characterized in terms of four components that communicate with each other as illustrated in Figure 2, regardless of the distribution change regime it is designed to operate in:

• A dimensionality reduction block \(R{}(\bullet)\), which takes as input the raw data \(x^t{}\) and outputs a lower-dimensional representation \(\delta{}^t\), where \(|\delta{}^t| \leq |x^t{}|\);

• A data management block \(B{}\), which selects the instances to be passed to the learning mechanism \(L{}\). It is characterized by: (i) an admission condition, determining whether a new instance \(\delta{}^t\) is stored in an internal buffer \(\Delta{}^t\) of maximum capacity \(|\Delta{}|\); (ii) the buffer \(\Delta{}^t\) itself, which holds the selected instances and evolves over time as new instances are admitted; and (iii) a trigger condition, determining when the learning mechanism \(L{}\) is activated on the entirety of \(\Delta{}^t\);

• An ML algorithm \(f{}^t(\bullet)\), which takes as input \(\delta{}^t\) and outputs the prediction \(\overline{y^t}{}\);

• A learning mechanism \(L{}\), which uses the buffered instances \(\Delta{}^t\) to update \(f{}^t\), with the objective of improving its performance on future instances of \(S{}\).

3.4.1 Resource Requirements↩︎

At each time instant \(t{}\), each component has an associated memory requirement and an associated execution time, which together constitute the total memory requirement \(M{}_{A{}}^{t{}}\) and total execution time \(E{}\!\left(A{}^{t{}}, D{}\right)\)⁴ of \(A{}\) at time \(t{}\), the quantities that must satisfy the constraints introduced in Sections 3.2 and 3.3:

\[\label{eq:memory} M{}_{A{}}^{t{}} = M{}_{R{}}^{t{}} + M{}_{B{}}^{t{}} + M{}_{L{}}^{t{}} + M{}_{f{}}^{t{}},\tag{1}\] \[\label{eq:exectime} E{}\!\left(A{}^{t{}}, D{}\right) = E{}_{R{}}^{t{}} + E{}_{B{}}^{t{}} + E{}_{L{}}^{t{}} + E{}_{f{}}^{t{}}.\tag{2}\] Without loss of generality, at each time instant \(t{}\), any component of \(A{}\) can reduce to the identity function, making both its memory requirement and its execution time equal to zero. For instance, a solution with no dimensionality reduction block sets \(R{}(\bullet) = \text{id}(\bullet)\), yielding \(M{}_{R{}}^{t{}} = E{}_{R{}}^{t{}} = 0\). Similarly, any block can be inactive at a given \(t{}\), reducing its execution time to zero for that instant, while still maintaining a non-zero memory footprint. Finally, both \(M{}_{A{}}^{t{}}\) and \(E{}\!\left(A{}^{t{}}, D{}\right)\) may vary over time, depending on the architecture, parameters, and amount of data involved in the operations of each block.

With these edge cases (i.e., \(R{}(\bullet) = \text{id}(\bullet)\) and \(|\Delta{}| = 1\)) this decomposition covers all works surveyed in this paper. The boundary between \(R{}\) and \(f{}^t\) is not always sharp in end-to-end trained networks, but the separation is present and meaningful in the large majority of surveyed works.

The different solutions components employed in the literature are analyzed in detail in Section 6.

3.4.2 Evaluation of Predictive Performance of ODL Solutions↩︎

Evaluating an ODL solution is fundamentally more complex than evaluating a static ML model. In the train-then-deploy paradigm, performance is measured once on a held-out test set after training is complete. In ODL, by contrast, the learning phase occurs during the operational life of \(D{}\), and performance evolves over time accordingly.

Since the model that will actually run on \(D{}\) does not yet exist at evaluation time, ODL works typically perform evaluation offline and off-device, using a held-out data stream that simulates \(S{}\), constructed from real data withheld for evaluation purposes. Let \(m{}(A{}, t{})\) be a performance metric of \(A{}\) at time \(t{}\), such as classification accuracy or F1 score. The appropriate evaluation protocol depends on the distribution change regime.

In the single-change regime, \(A{}\) is run on the held-out stream for a given number of steps, and final performance is measured on a separate held-out test set \(S_{test}{}\) drawn from the same post-deployment distribution [16]–[18]. This procedure can be repeated across multiple stream lengths to assess data efficiency.

In the concept drift regime, the held-out stream serves as both the source of learning updates and the basis for evaluation, following the prequential protocol [19], in which each sample is first used for inference and then for a learning update [20], [21]. Performance is summarized as a running average of \(m{}(A{}, t{})\) over the stream.

In the continual learning regime, a separate \(S_{test}{}\) is maintained for each distribution encountered in \(S{}\). Each \(S_{test}{}\) is evaluated both immediately after \(A{}\) finishes learning from the corresponding distribution, and again after exposure to all subsequent distributions [22]–[24]. This longitudinal evaluation tracks knowledge acquisition and retention independently: a drop in performance on a previously evaluated \(S_{test}{}\) indicates catastrophic forgetting [14], while an improvement on a not-yet-seen \(S_{test}{}\) indicates positive forward transfer.

4 The Application Lens↩︎

This section analyzes the application contexts in which ODL solutions have been deployed and evaluated. The literature is organized along two complementary axes. The primary distinction separates method-oriented works, which propose algorithmic contributions validated on standard benchmark datasets without a specific application framing, from application-oriented works, which are motivated by and evaluated on a specific real-world deployment scenario. Within each of these two categories, works are further organized by the distribution change regime in which they operate, following the same single-change, concept drift, and continual learning taxonomy established in Section 3.1.

4.1 Method-Oriented Works↩︎

Method-oriented works use standard benchmark datasets as a proxy for real-world scenarios; for this reason, Table [tab:benchmarks] lists those present in the surveyed literature, organized by regime and task.

4.1.1 Single-Change Regime↩︎

The majority of works evaluate on image classification benchmarks, including CIFAR-10, CIFAR-100, MNIST and their variants, and domain-specific collections such as Flowers, Food, Cars, CUB, and Pets [18], [26]–[32], [34]–[37], [46], [52], [58], [72]. A number of these works additionally include audio benchmarks, in particular the Google Speech Commands dataset [29]–[32], [49], [52] and DCASE2020 [29], [30], as well as Inertial Measurement Units (IMUs) sensor benchmarks for Human Activity Recognition (HAR) such as PAMAP2, SHL, and CWRU [58] and UCI-HAR [30], [31], reflecting the relevance of these tasks in the TinyML community. Finally, a smaller group of works evaluates on tabular datasets such as Iris, heart disease, and cancer [35]–[37]. While these datasets are simpler than image or audio benchmarks, they are representative of the sensor data commonly encountered in IoT applications, making them a relevant, even if underused, evaluation domain for TinyML.

4.1.2 Concept Drift Regime↩︎

The method-oriented works in the concept drift regime are fewer than in the single-change setting. [20], [66], and [63] evaluate on image and audio benchmarks from the broader machine learning community, including Google Speech Commands (GSC), MNIST, CIFAR-10 and ImageNet, where concept drift is simulated through synthetic corruptions of standard datasets (e.g., CIFAR-10-C, ImageNet-C, SHIFT). In contrast, [21] evaluates exclusively on classic stream learning benchmarks from the data stream mining community, including NOAA, ELEC, RIALTO, POSTURE, COVER, and POKER, which feature naturally occurring distribution shifts rather than synthetic ones, at the cost of addressing tasks that are less representative of typical TinyML deployment scenarios.

4.1.3 Continual Learning Regime↩︎

The method-oriented works in the continual learning regime evaluate primarily on image classification benchmarks, with CORe50 [23], [68], [69], CIFAR-10 and CIFAR-100 [22], [68]–[70], and MiniImageNet [22] being the most commonly used. [68] also includes MNIST as an additional benchmark. CORe50 is particularly well suited to continual learning evaluation as it was specifically designed for object recognition under continuous learning conditions, making it more representative of real deployment scenarios than generic image classification benchmarks. Beyond image classification, [22] and [68] additionally evaluate on appropriately modified versions of GSC and UCI-HAR respectively, providing some evidence of generalization beyond the image domain.

4.2 Application-Oriented Works↩︎

Application-oriented works are motivated by and evaluated on specific real-world deployment scenarios, making the application domain the primary axis of comparison; Table [tab:applications] lists those present in the surveyed literature, organized by regime and task.

4.2.1 Single-Change Regime↩︎

4.2.1.1 Keyword Spotting.

Keyword Spotting (KWS) is the most represented application in the single-change regime, addressed from three complementary angles. Personalization to a specific user’s speech characteristics is addressed by [73], which adapts a pre-trained model using a small number of labeled utterances from the target user. Noise adaptation to the acoustic conditions of the deployment context is addressed by [80], [81], which fine-tune a subset of model parameters using recordings collected on-site. The enrollment of new, user-defined keywords is addressed by [12], [17], [74], [76], where the challenge is to recognize previously unseen words from a handful of recordings: some works rely on few-shot supervised learning [17], [74], [76], while others eliminate the labeling requirement through self-supervised pseudo-labeling of the incoming audio stream [12]. [16] extend the KWS personalization setting to speaker verification, combining keyword detection with user identity authentication in a two-stage on-device pipeline.

4.2.1.2 Human Pose Estimation.

[82], [83] demonstrate on-device self-supervised fine-tuning for human pose estimation aboard nano-drones, where the model adapts to the visual characteristics of a new deployment context. The supervision signal is derived from a secondary on-board sensor available only during the adaptation phase, making the approach applicable without any human annotation.

4.2.1.3 Human Activity Recognition.

[84] address personalization of IMU-based HAR on STM32 microcontrollers, demonstrating that fine-tuning with a small user-specific labeled dataset significantly reduces inter-subject variability. Both a publicly available benchmark and a proprietary collected dataset are used for evaluation, providing evidence of real-world applicability beyond standard benchmarks.

4.2.1.4 Biosignal Processing.

Physiological signals are inherently person-specific, making them natural candidates for on-device personalization. For ECG anomaly detection, [91] train a personalized model from scratch on the MIT-BIH arrhythmia dataset, a clinically validated benchmark that closely reflects real deployment conditions. For sEMG-based applications, [86] and [87] address gesture recognition personalization, where cross-session variability due to electrode shift is the primary challenge, while [89] extend this to hand dynamics estimation on the HYSER dataset, demonstrating that incremental online training achieves cross-day accuracy comparable to offline training.

4.2.1.5 Anomaly Detection.

Anomaly detection appears across multiple deployment contexts in the single-change regime, sharing the common characteristic that normality must be learned on-device since it cannot be defined a priori. [93] address industrial time series anomaly detection by fitting an analytical model directly from unlabeled post-deployment data. [18] demonstrate on-device anomaly detection for a fan monitoring application, adapting an autoencoder to the normal operating conditions of the deployment platform from collected data. [94] address driver behavior anomaly detection, where normality is user-dependent and learned through unsupervised on-device clustering of vehicle sensor data.

4.2.1.6 Presence and Occupancy Detection.

[95] address occupancy detection from ambient environmental sensors in smart buildings, providing a systematic comparison of online learning algorithms on resource-constrained devices for this task.

4.2.1.7 Gesture Classification.

[96] tackle on-sensor gesture classification in an extremely constrained setting, learning a personalized model from scratch within 8 kB of memory from proprietary collected data.

4.2.2 Concept Drift Regime↩︎

4.2.2.1 Sensor Calibration.

Sensor calibration is the only real-world application addressed in the concept drift regime. Physical sensors exhibit continuous drift over time due to aging, temperature variations, and mechanical stress, making calibration a genuinely non-stationary problem that requires the learning mechanism to remain active throughout the operational life of the device. [97] address IMU calibration under thermal stress, learning a compensation model on the sensor’s integrated processor from data collected under controlled temperature variations, with additional validation on the EuRoC MAV dataset under dynamic conditions. [13] address pressure sensor calibration, learning a compensation model from sporadic reference measurements available during deployment, without requiring any offline recalibration procedure.

4.2.3 Continual Learning Regime↩︎

4.2.3.1 Brain-Machine Interfaces.

EEG-based Brain-Machine Interfaces (BMI) are a natural application for continual learning personalization, as neural signals exhibit strong inter-session variability that causes performance degradation over time, and the set of mental states or motor commands the system must recognize may expand as the user’s needs evolve. [15] and [24] address this by enabling continual on-device adaptation of EEG classification models to the evolving signal characteristics of a specific user, demonstrating that catastrophic forgetting can be mitigated while maintaining the ultra-low power requirements of wearable BMI devices. Both works evaluate on a dataset collected from wearable EEG hardware, with [24] additionally validating on a novel collected dataset.

4.2.3.2 Human Activity Recognition.

Class-incremental learning for HAR addresses the scenario where new activities must be added to the model after deployment without retraining from scratch. [98] address class-incremental HAR from IMU data, evaluating on four publicly available benchmarks including Opportunity, PAMAP2, Skoda, and MHEALTH, and demonstrating resource-efficient expansion of the model as new activity classes are introduced.

4.3 Findings↩︎

4.3.1 Method-Oriented vs Application-Oriented Works↩︎

A striking observation that emerges from the comparison between method-oriented and application-oriented works is the limited overlap in the tasks and datasets they address. KWS and, to a lesser extent, IMU-based HAR are the only tasks that appear consistently in both categories, with GSC and UCI-HAR serving as their respective shared benchmarks.

This misalignment is more pronounced in TinyML than in conventional machine learning, and reflects two concurrent structural factors. On one side, the application domains that are most studied in the methodological literature, primarily image classification, are too memory and computationally demanding to build relevant real-world TinyML applications with current hardware. On the other side, many of the real-world application domains that do fit within TinyML constraints, such as biosignal processing, sensor calibration, and anomaly detection, lack standardized benchmarks that methodological works can adopt for fair comparison. The result is a field where algorithmic progress and application development are advancing largely in parallel, with limited cross-fertilization.

4.3.2 Personalization, Environmental Adaptation, and Sensor Calibration↩︎

The application-oriented works in this survey address three broad categories of real-world challenges. Personalization tasks, the most represented, aim to adapt a model to a specific individual, spanning domains such as KWS, HAR, biosignal processing, and driver behavior analysis. Environmental adaptation tasks address distribution shifts originating from the deployment environment rather than from the user, as in the case of acoustic noise adaptation and visual perception aboard drones. Sensor calibration emerges as a distinct category, particularly in the concept drift regime, where the continuously drifting output of physical sensors must be compensated on-device throughout the operational lifetime of the device.

4.3.3 The Absence of Explicit Application Requirements↩︎

Despite the centrality of execution time and power consumption as application-level constraints, as established in Section 3.2, the application-oriented works surveyed here rarely state the requirements they are targeting explicitly. In industrial deployments, power budgets and latency constraints (\(\overline{E{}}{}\)) are strict and non-negotiable. In research works, they are typically left implicit, with the choice of hardware class serving as a proxy. This implicit convention has a structural consequence: without benchmarks that fix an explicit power budget and latency constraint \(\overline{E{}}{}\), the co-optimization of \(D{}\) and \(A{}\) becomes practically infeasible, and hardware and algorithmic choices are made sequentially rather than jointly. Establishing benchmarks that specify explicit deployment requirements alongside task and dataset would be a concrete step toward enabling this co-optimization, and is discussed further in Section 7.

5 The Hardware Lens↩︎

This section analyzes the hardware platforms on which the surveyed ODL solutions have been deployed and evaluated. As established in Section 3.3, the feasibility of any ODL solution is fundamentally constrained by the on-device memory budget \(M{}_{D{}}{}\) and the clock frequency \(F_{D{}}{}\). The hardware choices made across the literature therefore reveal both the current boundaries of what ODL can achieve on real devices and the gap between algorithmic contributions evaluated on capable platforms and solutions deployable on the most constrained hardware.

The hardware platforms reported across the surveyed works can be grouped into three classes based on their computational architecture: ultra-constrained processors (Section 5.1), standard MCUs (Section 5.2), and PULP platforms (Section 5.3). These classes are not merely points on a performance spectrum: each represents a qualitatively different architectural design philosophy, with distinct implications for which ODL solutions are feasible. A non-negligible fraction of the surveyed works do not evaluate on a specific hardware platform, and a smaller number target platforms outside the TinyML scope defined in Section 3.3, such as Cortex-A processors, Graphics Processing Units (GPUs), and smartphones. These works are excluded from the analysis carried out in this section but are included in the application and solution analyses, where their algorithmic contributions remain relevant.

5.1 Ultra-Constrained Processors↩︎

Ultra-constrained processors include platforms lacking a hardware floating-point unit, such as Cortex-M0 [100] based MCUs, with \(F_{D{}}{} \leq 100\) MHz, and the In-Sensor Processing Unit (ISPU - STMicroelectronics LSM6DSO16IS [101]). The ISPU integrates a hardware Floating-Point Unit (FPU) but operates at only \(F_{D{}}{} = 5\)–\(10\) MHz, placing it in the same practical regime as FPU-less devices. It is also architecturally unique in that it is embedded directly inside the sensor package, eliminating data movement between sensing and processing entirely. The constraints we just described, combined with a very small \(M{}_{D{}}{}\), imposes the tightest memory and computational budgets of the three classes and constrains the algorithmic choices available, as discussed in Section 6.

Devices in this class span \(F_{D{}}{} \in [5~\text{MHz},\, 133~\text{MHz}]\) (ISPU to RP2040), with \(M{}_{D{}}{}\) ranging from 8 kB to 264 kB, as summarized in Table [tab:hardware95ultra95agg]. Active power consumption is rarely reported in this class. Based on datasheet figures, the ISPU draws approximately 1 mW in active mode, while Cortex-M0 based boards typically fall in the 10–50 mW range, depending on the specific board and supply voltage.

Cortex-M0 based boards appear in both method-oriented and application-oriented works. Their widespread availability and low cost make them a natural evaluation target for method contributions seeking to demonstrate feasibility on constrained hardware, as well as for application-driven deployments where the algorithmic and power budget is tight. The ISPU, by contrast, appears exclusively in application-oriented works, consistent with its nature as a purpose-built in-sensor processor: its architectural constraints make it unsuitable as a benchmark device for algorithmic contributions, which typically require more flexibility in terms of model architecture and evaluation tooling.

The surveyed works on ultra-constrained hardware are almost entirely concentrated in the single-change regime, with a single exception in concept drift [97] and no works in the continual learning regime. This concentration reflects the constraints of the class: the algorithmic approaches feasible on these platforms can handle a single bounded adaptation phase, but sustaining a continuously active learning mechanism throughout the operational life of \(D{}\) would require memory and computational resources beyond what they offer.

5.2 Standard MCUs↩︎

\(^{a}\)The upper end (600 mW) marginally exceeds the 500 mW criterion in Section 2.1

. The device (STM32H7 at 480 MHz) is MCU-class in all other respects; 600 mW is a worst-case datasheet estimate across supply-voltage configurations, not a measured workload power. This work is included on that basis.

Standard MCUs include general-purpose single-core processors equipped with a hardware FPU, such as Cortex-M4 [105] and Cortex-M7 [106] based platforms, with \(F_{D{}}{} \in [80,\, 480]\) MHz. These devices follow a conventional scalar pipeline architecture with no hardware-level optimization for neural network workloads, and represent the workhorse class of TinyML, supported by a mature software ecosystem and available across a wide range of off-the-shelf boards.

The presence of an FPU and a larger \(M{}_{D{}}{}\), typically between 256 kB and a few MB, makes backpropagation-based learning feasible on this class, provided that appropriate memory optimizations are applied as discussed in Section 6. Devices in this class span \(F_{D{}}{} \in [64,\, 480]\) MHz, with \(M{}_{D{}}{} \in [60~\text{kB},\, 512~\text{kB}]\) across the surveyed works. Active power figures are not directly reported in any of the surveyed works in this class; all values in Table [tab:hardware95mcu] are estimated from datasheet active-mode figures. Based on these estimates, Cortex-M4 boards typically draw 20–60 mW, while Cortex-M7 boards fall in the 100–600 mW range, reflecting genuine differences in \(F_{D{}}{}\) and supply voltage across device families rather than workload variation, as active power is a property of the silicon.

Standard MCUs are the most balanced class in terms of regime coverage, with works present in all three regimes. The single-change regime is by far the most represented, covering both application-oriented works and methodological contributions exploring backpropagation optimization [27], [102], [103] and forward-only learning [34], [49], [52], [58]. Within this regime, contributions are distributed fairly evenly across Cortex-M4 and Cortex-M7 cores, and between method-oriented and application-oriented works, with no clear concentration in either dimension. In the concept drift regime, standard MCUs are the most represented hardware class [20], [63]: concept drift solutions rely on lightweight incremental algorithms that neither require replay buffers nor incur the backpropagation overhead of continual learning, making standard MCUs a natural fit. In the continual learning regime, by contrast, only one work targets a standard MCU [22], with most solutions pushed toward more powerful devices by the memory demands of maintaining and replaying past distributions.

5.3 PULP Platforms↩︎

PULP platforms include multi-core parallel ultra-low-power architectures such as the commercial platforms GAP8 and GAP9, and the research prototypes Mr. Wolf and VEGA. What distinguishes this class is its parallel multi-core architecture: instead of a single general-purpose core, PULP platforms feature a cluster of up to nine small cores operating in parallel over a shared memory, allowing them to process the same workload faster and at a lower energy cost per operation than a single high-performance core.

Individual cores span \(F_{D{}}{} \in [100, 400]\) MHz and \(M{}_{D{}}{} \in [15~\text{kB},\, 64~\text{MB}]\) across the surveyed works, reflecting the wide range of deployment scenarios this class supports, as summarized in Table [tab:hardware95pulp]. Active power is more consistently reported in this class than in the others. GAP9 figures range from 19 to 66 mW during single-change workloads [17], [81]–[83] and from 21 to 50 mW for continual learning applications [15], [24], with [70] reporting 45 mW. Mr. Wolf draws approximately 10.4 mW [87]. VEGA and GAP8 works do not report active power figures.

In the single-change regime, PULP is heavily used for audio and biosignal applications, with GAP9 being the dominant platform [12], [17], [75], [80]–[83], [89], alongside GAP8 for an earlier sEMG work [86], VEGA for KWS personalization [73], and Mr. Wolf for EMG gesture recognition [87]. In the concept drift regime, no PULP works were identified. Unlike the absence of ultra-constrained devices in continual learning, which can be explained by hardware limitations, the absence of PULP in concept drift is more likely a reflection of the different research communities that have historically addressed these two problems, rather than a fundamental feasibility constraint. The continual learning regime is where PULP platforms are most distinctive: they account for the large majority of continual learning works with reported hardware [15], [23], [24], [70], reflecting the substantially higher memory and computational requirements of continual learning algorithms compared to single-change and concept drift solutions.

5.4 Findings↩︎

5.4.1 Hardware Capability and Regime Complexity↩︎

A clear correlation emerges between hardware capability and regime complexity. Ultra-constrained processors appear almost exclusively in the single-change regime, where the learning phase is bounded and lightweight algorithms suffice. Standard MCUs span all three regimes but are thinly represented in continual learning, where their \(M{}_{D{}}{}\) is often insufficient to sustain an active learning mechanism over the device lifetime. PULP platforms dominate the continual learning regime, being the only class capable of supporting its most demanding solutions. This monotonic relationship suggests that the choice of hardware is not independent of the target regime: more complex regimes impose stricter requirements, and pushing ODL further into non-stationary regimes will likely require hardware capabilities beyond what standard MCUs can currently offer.

A complementary pattern emerges along the method-application axis. General-purpose ARM-based boards, whether Cortex-M0, M4, or M7, are the preferred platform for methodological contributions, favored for their wide availability, low cost, and reproducibility. More specialized platforms, namely the ISPU and PULP devices, appear predominantly in application-oriented works, where the hardware properties are themselves part of the contribution. This divide between algorithmic research, which uses commodity hardware as a neutral substrate, and system-level research, which develops the solution in tight coupling with the target platform, is a structural feature of the current ODL literature.

5.4.2 Hardware Reporting in the Literature↩︎

Comparing hardware platforms in a solution-agnostic way requires metrics that depend solely on \(D{}\), independently of the algorithmic choices made on top of it. The metrics most commonly reported in the surveyed works, namely peak RAM usage \(M{}_{D{}}{}\), energy per learning event, and latency \(E{}\!\left(A{}^{t{}}, D{}\right)\), all depend jointly on \(D{}\) and \(A{}\), as established in Section 3.4, and are therefore unsuitable for this purpose. These metrics are instead examined in depth in Section 6.3.

In this chapter, we instead characterize each platform by three solution-agnostic metrics: RAM capacity \(M{}_{D{}}{}\), clock frequency \(F_{D{}}{}\), and peak instantaneous active power consumption, all of which are properties of the silicon rather than of the workload. These metrics describe what a device offers, but not how efficiently it executes a given ODL solution.

This points toward a broader gap: the absence of fixed-solution benchmarks that, by holding \(A{}\) constant and varying \(D{}\), would make energy consumption, latency, and peak RAM usage comparable across platforms and enable principled hardware selection. This is discussed further as an open challenge in Section 7.

5.4.3 PULP Platforms and the Hardware Landscape for ODL↩︎

The dominance of PULP platforms in the continual learning regime deserves closer examination, particularly in light of a class of devices that sits between standard MCUs and PULP but has not appeared in the surveyed literature: MCUs equipped with a dedicated neural network accelerator, such as the ARM Cortex-M55 paired with an Ethos-U55 [107] or Ethos-U65 [108] Neural Processing Unit (NPU). These platforms retain the single-core scalar architecture of a standard MCU but add a fixed-function hardware block accelerating specific neural network operations, and could in principle offer a middle ground between standard MCUs and PULP platforms. Their absence from the ODL literature is likely explained by the fact that their software ecosystems have been developed almost exclusively for inference, with no support for gradient computation or on-device training, and that they were not widely available at the time of the earliest works in this survey.

PULP platforms have instead accumulated concrete advantages explaining their prominence. A dedicated ODL research community has formed around GAP9 and VEGA, producing shared methodology and open-source tools such as DORY [109] and PULP-TrainLib [110] that directly expose training primitives. PULP also offers a contained power envelope with substantially higher computational throughput than standard MCUs, and its flexible architecture, where active cores, memory banks, and off-chip memory can be configured independently, makes it adaptable across a wide range of deployment scenarios.

These advantages are accompanied by significant practical limitations. PULP platforms are considerably more complex to program than standard MCUs, requiring familiarity with cluster execution models, direct memory access management, and multi-bank memory hierarchies. More fundamentally, no commercial off-the-shelf development kit is currently available, restricting access to research groups with direct connections to the platforms’ developers, and their unit cost remains substantially higher than commodity MCU solutions.

6 The Solution Lens↩︎

This section analyzes the technical composition of the ODL solutions surveyed in this work, focusing on how the components described in Section 3.4 are implemented in practice. As argued in Section 3.1, solutions operating under different distribution change regimes are not directly comparable, and the analysis is organized accordingly. The primary axis is the learning paradigm: Section 6.1 covers batch learning solutions, in which \(L{}\) executes exactly once on a collected dataset before \(A{}\) transitions permanently to inference, and which by construction are limited to the single-change regime. Section 6.2 covers incremental learning solutions, in which \(L{}\) remains active throughout the operational life of \(D{}\). Within Section 6.2, the secondary axis is the distribution change regime, following the single-change, concept drift, and continual learning taxonomy of Section 3.1.

6.1 Batch Learning Solutions↩︎

In batch learning solutions, as illustrated in Figure 3, \(B{}\) first collects a set of \(N{}\) instances from \(S{}\) into the buffer \(\Delta{}^t\). Once the trigger condition fires, \(L{}\) is executed exactly once on \(\Delta{}^t\), producing a definitive version of \(f{}^t\). The solution then transitions permanently to inference mode, with no further learning taking place. Formally, \(A{}\) goes through three configurations:

• For \(t{} \in [T{}, T{} + N{} - 1]\), only \(R{}\) and \(B{}\) are executed, collecting incoming instances into \(\Delta{}^t\);

• at \(t{} = T{} + N{}\), the trigger condition fires and \(L{}\) is executed on \(\Delta{}^t\), producing a trained \(f{}^t\);

• for \(t{} > T{} + N{}\), only \(R{}\) and \(f{}^t\) are executed, performing inference on \(S{}\).

By construction, batch learning solutions have no mechanism for managing further changes in the distribution of \(P{}\) after the initial adaptation phase, and are therefore only applicable in the single-change regime. The surveyed batch learning solutions are summarized in Table [tab:batch95learning], and the implementation of each component across these works is analyzed below.

6.1.1 Dimensionality Reduction Block \(R{}\)↩︎

The large majority of works implement \(R{}\) as a pre-trained Convolutional Neural Network (CNN) whose parameters remain frozen throughout the on-device phase [12], [16], [17], [73], [74], [80]–[84]. In audio applications, the CNN is typically preceded by a Mel-frequency cepstral coefficients (MFCC) front-end that converts the raw audio signal into a compact spectral representation before feature extraction [12], [16], [17], [73], [74], [80], [81]. A smaller group of works foregoes a learned feature extractor entirely: [91] relies on a fixed reservoir followed by Principal Component Analysis (PCA), while [35], [36], [93] pass raw or minimally processed data directly to \(f{}^t\). The works focusing on backpropagation optimization [26]–[31], [46], [102] set \(R{}\) to the identity function, incorporating the full network into \(f{}^t\) instead. This last pattern is exclusive to method-oriented works: no application-oriented work in the batch learning literature trains end-to-end on-device, reflecting the current practical infeasibility of full network training on constrained hardware for real deployment scenarios.

6.1.2 Data Management Block \(B{}\)↩︎

Across all batch learning solutions, \(B{}\) reduces to a buffer of fixed capacity \(|\Delta{}|\) with a trivial admission condition: every incoming instance is stored until the buffer is full, at which point the trigger condition fires and \(L{}\) is activated. The primary source of variation is in how \(|\Delta{}|\) is determined. The large majority of works assume the complete adaptation dataset fits in memory and set \(|\Delta{}|\) accordingly [26]–[31], [35], [36], [73], [80]–[84], [91], [93], [102]. A smaller group targets few-shot settings, fixing \(|\Delta{}|\) to a small number of labeled examples per class, typically between one and five [16], [17], [74]. [12] occupies an intermediate position, maintaining a balanced buffer of positive and negative examples whose size is determined by the self-labeling procedure rather than fixed in advance. Similarly, [73], [80], [81] retain a portion of \(S_0{}\) in the buffer alongside new examples, anticipating the replay strategies seen in the continual learning regime.

The memory footprint of \(B{}\), i.e., \(M{}_{B{}}^{t{}} = |\Delta{}| \cdot |\delta{}^{t{}}|\), is frequently the largest term in Equation 1 and is the primary source of underestimation in works that do not target a real deployment device or implement a simplified version of the solution on-device. It is usually not reported in methodological works focused solely on backpropagation optimization [26], [27], [29]–[31], [46], whose reported memory figures usually compare just the cost of \(f^t\) and \(L\) with and without the optimization.

6.1.3 ML Algorithm \(f{}^t\) and Learning Mechanism \(L{}\)↩︎

The joint choice of \(f{}^t\) and \(L{}\) can be organized into three families, which broadly correspond to the choice of \(R{}\) described above.

The first and most common family pairs a small MLP as \(f{}^t\) with standard backpropagation as \(L{}\). This combination is natural when \(R{}\) is a frozen pre-trained CNN: the learning phase is restricted to the classification head, keeping both the memory requirement of \(L{}\) and the execution time \(E{}_{L{}}^{t{}}\) minimal [73], [80]–[84].

The second family uses a complete neural network as \(f{}^t\), with \(R{}\) reducing to the identity function. \(f{}^t\) is initialized from a network pre-trained on \(S_0{}\) and then partially updated on-device, with the associated increase in memory and execution time addressed through an optimized \(L{}\). Approaches include bias-only updates [26], sparse parameter updates applied selectively to the most important parameters or channels [27], [29], [30], [46], [102], and forward-only learning rules that avoid backpropagation entirely [28], [31].

The third family foregoes neural networks entirely, pairing non-deep classifiers as \(f{}^t\) with learning mechanisms tailored to their structure. Prototype-based classifiers are learned by constructing class representatives from averaged feature vectors [12], [17], [74], instance-based classifiers store feature vectors directly for similarity-based classification [16], SVMs are trained through their standard optimization procedure [35], [91], linear classifiers are fitted through SGD with a one-vs-one scheme [36], and parametric models are fitted through expectation-maximization [93]. As noted above, many of these solutions still rely on a frozen pre-trained CNN as \(R{}\), so that the non-deep nature applies to \(f{}^t\) and \(L{}\) only.

6.2 Incremental Learning Solutions↩︎

In incremental learning solutions, \(L{}\) is executed multiple times as \(S{}\) progresses, updating \(f{}^t\) every time the trigger condition is satisfied, as illustrated in Figure 4. All four components of \(A{}\) remain active throughout the operational life of \(D{}\), though not all are required to execute at every time instant \(t{}\). Unlike batch learning solutions, incremental learning solutions are applicable in all three regimes, as they retain the ability to adapt to further changes in the distribution of \(P{}\) throughout the operational life of \(D{}\). The surveyed incremental learning solutions are summarized in Table [tab:incremental95learning], and their technical composition is analyzed below, organized by component.

6.2.1 Dimensionality Reduction Block \(R{}\)↩︎

In Incremental Learning, a frozen pre-trained CNN is also the dominant choice for \(R{}\) across all three regimes [15], [18], [20], [22]–[24], [32], [34], [49], [52], [58], [63], [69], [70], [72], [86]. This pattern is driven by the constraints of \(D{}\) rather than algorithmic preference: by keeping \(R{}\) frozen, both \(M{}_{L{}}^{t{}}\) and \(E{}_{L{}}^{t{}}\) are kept minimal regardless of the regime. In the continual learning regime, this choice is nearly universal, with only [68] and [98] setting \(R{}\) to the identity and training end-to-end on-device. In [22] and [70], \(R{}\) is additionally meta-learned offline to produce representations particularly amenable to few-shot on-device adaptation.

The concept drift regime is the exception to this pattern. The majority of concept drift works set \(R{}\) to the identity function, operating directly on raw or minimally processed sensor data [13], [21], [66], [97], with only [20] and [63] relying on a frozen pre-trained CNN. This reflects the nature of the tasks addressed in that regime: sensor calibration and data stream classification operate on low-dimensional tabular or scalar signals that do not require a learned feature extractor, unlike the audio and image inputs that dominate the single-change and continual learning regimes.

6.2.2 Data Management Block \(B{}\)↩︎

The configuration of \(B{}\) reveals the sharpest cross-regime divergence in the incremental learning literature. In the single-change and concept drift regimes, fully online operation with \(|\Delta{}| = 1\) is the dominant strategy, meaning \(L{}\) is executed on each incoming instance individually [13], [18], [21], [34], [37], [49], [89], [95]–[97]. This choice minimizes the memory footprint of \(B{}\), keeping \(M{}_{B{}}^{t{}} \approx 0\), and is consistent with the lightweight algorithmic families that dominate these regimes. A smaller group of single-change works uses explicit buffering: [72] and [32] adopt an active learning strategy that selects only the most informative instances to add to the buffer; [52], [58] empty the buffer after each update; while [86] maintains a fixed-capacity buffer with uniform and exponential sampling strategies. In the concept drift regime, [63] maintains a First-in-First-out (FIFO) buffer of fixed capacity, and [20] and [66] activate \(L{}\) only when an explicit change detection mechanism signals a distribution shift.

The continual learning regime stands in sharp contrast: all solutions use explicit buffering, with \(B{}\) collecting instances corresponding to a new distribution before triggering \(L{}\). This is a structural necessity rather than a design preference, as preventing catastrophic forgetting requires retaining some representation of past distributions alongside new data, in the most common continual learning solutions. The primary sources of variation are in how the buffer content is managed and when \(L{}\) is triggered. To minimize the memory footprint of the replay buffer, [23] and [69] store compact latent representations of past distributions using quantized and binary encodings respectively, while [68] populates \(\Delta{}^t\) with synthetic samples obtained through dataset distillation rather than real instances. [22] reduces the buffer requirements from the other direction, using a meta-learned \(R{}\) that produces representations amenable to few-shot adaptation, so that fewer past samples are needed to anchor previous knowledge. Regarding the trigger condition, [15] activates \(L{}\) only when classification accuracy drops below an acceptability threshold, while all other works trigger when the buffer is full.

6.2.3 ML Algorithm \(f{}^t\) and Learning Mechanism \(L{}\)↩︎

The joint choice of \(f{}^t\) and \(L{}\) follows a clear cross-regime pattern. In the continual learning regime, backpropagation applied to a DNN is the dominant and nearly universal combination [15], [22]–[24], [68], [69], [98], with [70] as the only exception, using a prototype-based classifier whose \(L{}\) reduces to constructing a new class representative from averaged feature vectors. The structural reason is straightforward: since all continual learning solutions maintain a replay buffer of sufficient size to prevent catastrophic forgetting, the memory cost of backpropagation is unlikely to constitute a bottleneck relative to the buffer itself.

In the single-change and concept drift regimes, the picture is more diverse. Backpropagation-based DNNs remain present in both: [18], [32], [49], [72], [86], [89] in single-change, and [63] and [66] in concept drift. However, lighter alternatives are well represented. Forward-only rules appear in both regimes: [52], [58] pair a hyperspherical Restricted Coulomb Energy (RCE) classifier with a forward-only update in single-change, while [97] and [13] use Radial Basis Function (RBF) networks with a forward-only mechanism in concept drift. Non-neural approaches are also present: K Nearest Neighbors (KNN)-based classifiers in [34] and [20], prototype construction in [96], a binary linear classifier in [37], a Hoeffding-inspired decision tree in [21], and a multi-classifier evaluation in [95].

The concentration of backpropagation in continual learning and its diversification in the other two regimes reflects a structural property discussed further in Section 6.3: instance-based and prototype-based solutions tend to grow monotonically in memory over time (at least in their basic version without optimizations), making them poorly suited to regimes where \(A{}\) must remain active and adapt indefinitely.

6.3 Findings↩︎

6.3.1 Solution Components, Application Environments, and Hardware classes↩︎

The patterns observed across the surveyed solutions can be understood as consequences of the trade-offs inherent in each component choice, with hardware capability acting as the primary constraint shaping which combinations are feasible. Table [tab:components] summarizes these trade-offs with qualitative estimates of \(M{}_{A{}}^{t{}}\) and \(E{}(A{}^{t{}}, D{})\) for each component choice. The estimates are necessarily qualitative rather than quantitative, for a reason that goes beyond measurement difficulty: no component can be evaluated in isolation from the others. A frozen pre-trained CNN as \(R{}\) produces compact \(\delta{}^t\) that allow \(f{}\) to be a simple classifier, whereas identity forces \(f{}\) to absorb the full input complexity. Similarly, full backpropagation may be viable on an ultra-constrained device if \(f{}\) is a single linear layer, yet infeasible if it is a deep CNN. The table should therefore be read as a characterization of the trade-offs each choice introduces, not as an independent ranking of alternatives.

For \(R{}\), the choice ranges from identity (no overhead, but full input dimensionality exposed to \(L{}\)) through statistical feature extractors (lightweight dimensionality reduction without pre-training) to frozen pre-trained CNNs (highest cost, but compact discriminative representations that directly reduce \(|\delta{}^t|\), the memory footprint of \(\Delta{}^t\), and the labeled data required for adaptation). The frozen CNN is essentially mandatory for image and audio inputs, and is the dominant choice on standard MCUs and PULP platforms. Identity and statistical extractors are more common on ultra-constrained devices, where the cost of running a CNN may be prohibitive, and where simpler signal types are typically used.

For \(B{}\), fully online operation with \(|\Delta{}| = 1\) minimizes \(M{}_{B{}}^{t{}}\) and per-update execution time, but limits \(L{}\) to a single pass per instance, making a larger number of labeled instances necessary to reach convergence. Explicit buffering with \(|\Delta{}| > 1\) recovers data efficiency at the cost of a larger \(M{}_{B{}}^{t{}}\). Batch collection is the extreme case: peak RAM is largest, as \(M{}_{B{}}^{t{}}\) and the requirements of \(L{}\) and \(f{}^t\) overlap during the single learning episode, after which the solution runs inference only. This cost can nonetheless be kept within budget when the learning episode is designed for few-shot settings, where \(|\Delta{}|\) remains small by construction. Online solutions dominate in single-change and concept drift regimes. Explicit buffering is structurally necessary in continual learning, where past data must be retained to prevent catastrophic forgetting.

For \(f{}^t\) and \(L{}\), non-deep classifiers are cheaper to update than backpropagation-based networks, but instance-based and prototype-based methods grow in memory as new data are incorporated. Neural networks have a fixed memory footprint regardless of data seen, making them better suited to non-stationary regimes. Within neural network solutions, structured architectures such as RBF and RCE networks reduce learning to a forward pass, while sparsification, quantization, and binarization lower the cost of backpropagation. These optimizations concentrate in the single-change regime, where per-update cost is the primary concern. In continual learning, full backpropagation on capable hardware is the dominant choice, driven by the explicit replay buffers that all solutions in that regime require.

6.3.2 The Gap Between Methodological and Application-Oriented Solutions↩︎

Two structural gaps emerge when comparing methodological and application-oriented works.

The first concerns scope. Backpropagation optimization, the largest single cluster of methodological contributions in this survey, has produced a rich body of work on sparsification, quantization, and forward-only alternatives, yet most of these contributions are either validated only on standard benchmarks without any on-device implementation, or deployed on standard MCUs under controlled conditions that do not reflect real application constraints. Either way, this work has not translated into application deployments: no application-oriented work across any of the three regimes adopts these optimizations. The reason is not technical incompatibility but a narrowness of focus: methodological works optimize a single component of the pipeline in isolation, typically \(L{}\), and measure only its cost, without considering that the bottleneck in a real deployment may lie elsewhere, for instance in \(B{}\) or in the labeling strategy. The ODL problem is not solved by optimizing one component: it requires the entire pipeline to be viable simultaneously.

The second gap concerns label availability. Methodological works uniformly assume that labeled data is available in the quantities needed to validate their approach, an assumption that is rarely satisfied in practice, and which is particularly acute in the concept drift and continual learning regimes where application-oriented works remain almost entirely absent. Label availability is often the binding constraint that prevents methodological solutions from being ported to real deployments, and backpropagation optimization and label-efficient learning are rarely developed together despite being complementary concerns. The application-oriented works that have succeeded in real deployments have done so precisely because they treated label acquisition as a first-class design requirement alongside the algorithmic and hardware choices. Four distinct strategies emerge from these works. Few-shot learning provides adaptation from a small number of labeled examples supplied by the user, and is adopted by [12], [16], [17], [81] for personalization tasks such as keyword enrollment and speaker verification. Self-supervised learning with an auxiliary sensor derives labels automatically from a secondary on-board sensor, eliminating human annotation entirely, as in [82], [83] where supervision comes from the drone’s flight controller during adaptation. Unsupervised learning requires no labels at all, and applies naturally to anomaly detection and sensor calibration, where normality is learned from unlabeled post-deployment data [18], [93]. Implicit labeling through structured stimulus sequences derives labels from a predefined enrollment protocol rather than explicit annotation, as adopted by [24], [96] for biosignal applications. Each strategy directly constrains the viable solution architecture: few-shot works pair a frozen CNN with a prototype or instance-based classifier, unsupervised works avoid backpropagation entirely, and implicit labeling is specific to settings where an enrollment phase can be designed into the deployment protocol.

Taken together, both gaps point toward the same underlying need: solutions and benchmarks that treat the full pipeline as the unit of design and evaluation, rather than optimizing and evaluating individual components in isolation. This means end-to-end frameworks that expose \(R{}\), \(B{}\), \(f{}\), and \(L{}\) as jointly optimizable objects, and application-oriented benchmarks that reflect realistic labeling conditions and deployment constraints, as discussed further in Section 7.

6.3.3 Solution Reporting↩︎

Evaluating and comparing ODL solutions is substantially harder than evaluating static ML models. The difficulty manifests across two classes of metrics: joint metrics that depend on both \(A{}\) and \(D{}\), and solution-only metrics that characterize \(A{}\) independently of hardware.

6.3.3.1 Joint metrics: latency and energy.

Table [tab:reporting] summarizes the normalization conventions used for latency and energy across the surveyed works. The fragmentation is striking: figures are reported per instance, per batch, per epoch, per learning event, and per complete training run, with no consistency across works or even within the same regime. This makes direct comparison impossible even between solutions targeting the same hardware and task. The problem is not merely cosmetic: a latency reported per complete training run and one reported per instance differ by orders of magnitude for the same solution, depending on dataset size.

Regime-level patterns are visible in Table [tab:reporting]. In the single-change regime, reporting per complete training run dominates for both latency and energy: the learning phase is a one-off event, so total cost is the most operationally relevant figure. In the concept drift regime, energy is entirely absent from the reported metrics, a critical gap given that the cumulative cost of continuously repeating small updates is precisely what matters in this regime; per-instance or per-update figures would be the natural unit yet are rarely used. In the continual learning regime, “per learning event" is the most principled unit, reflecting the episodic structure of task-incremental updates, and the works that do report energy adopt it consistently [22], [23].

In summary, joint metrics are present across the literature but reported with inconsistent normalization conventions, making direct cross-work comparison unreliable.

6.3.3.2 Solution-only metrics: memory, computational cost, and predictive performance.

Peak RAM is the most consistently reported metric, yet many works report only the memory footprint of \(L{}\) and \(f{}^t\), ignoring \(M{}^t_{B{}}\), which as argued above can be the dominant term in Equation 1 . While most surveyed works report hardware-dependent metrics such as \(E{}\!\left(A{}^{t{}}, D{}\right)\) or the memory budget \(M{}_{D{}}{}\), a minority resort to hardware-independent proxies such as FLOPs, multiply-accumulate operations (MACs), parameter counts, or activation sizes [13], [26], [31], [46], [68], [69], [95], typically when no deployment target \(D{}\) is available. Although these metrics do not directly capture the peak RAM memory consumption or the execution time on a real \(D{}\), their hardware independence is a genuine advantage: a worst-case analytical estimate can, for certain classes of solutions, enable fairer cross-platform algorithmic comparison than wall-clock measurements tied to a specific \(D{}\).

On the predictive performance side, the time axis introduced by ODL makes comparison fundamentally harder than in static ML. In non-stationary regimes, a single accuracy figure is insufficient: a solution that adapts quickly but forgets rapidly is not comparable to one that adapts slowly but retains past knowledge. Even in the single-change regime, the amount of adaptation data is a critical variable that is rarely controlled for: works report results at vastly different numbers of labeled examples per class, making cross-paper comparison unreliable. This is discussed further in Section 7.

7 Future Directions↩︎

The analyses carried out in this survey point toward a coherent set of open challenges that, taken together, suggest a broader research agenda for the field.

7.0.0.1 Standardized evaluation frameworks.

A fundamental obstacle to progress in ODL is the absence of standardized evaluation frameworks. As highlighted in Sections 4.3, 5.4, and 6.3, the literature currently lacks consistent evaluation conventions at every level of the analysis. Three complementary frameworks are needed, each addressing a different aspect of the problem.

A solution-oriented benchmark, aimed at evaluating \(A{}\) independently of hardware, should fix a set of worst-case RAM and computational budgets and report predictive performance for solutions designed within each budget. Results should span multiple stream lengths to capture the adaptation curve rather than final accuracy alone, and the full memory footprint \(M{}_{A{}}^{t{}}\), including \(M{}_{B{}}^{t{}}\), must be verified against each budget. Where possible, deploying all candidate solutions on a common reference platform would allow joint hardware-solution metrics to be collected and worst-case estimates to be confirmed empirically. This benchmark is most directly applicable to method-oriented works, where the goal is to demonstrate algorithmic effectiveness across a range of resource constraints.

A hardware-oriented benchmark, aimed at evaluating the ODL capabilities of a given \(D{}\), should fix a set of ODL solutions and report the joint hardware-solution metrics: latency \(E{}\!\left(A{}^{t{}}, D{}\right)\), energy consumption for a clearly defined learning event, and peak instantaneous active power. The relevant hardware specifications, including \(M{}_{D{}}{}\) and \(F_{D{}}{}\), should be reported alongside to enable fair cross-platform comparison. This enables direct, reproducible comparison of hardware platforms independently of algorithmic choices, as argued in Section 5.4. This benchmark is most directly applicable to the evaluation of novel hardware platforms, whether research prototypes or commercial devices, across a range of ODL tasks and solutions.

A co-design benchmark, aimed at characterizing the tradeoff between \(A{}\) and \(D{}\), should specify explicit budgets for latency \(\overline{E{}}{}\) and power, and report joint hardware-solution metrics (latency \(E{}\!\left(A{}^{t{}}, D{}\right)\) and energy per learning event) together with predictive performance over time, after an optimization process that may involve changes to both \(A{}\) and \(D{}\). As argued in Section 4.3, the absence of such benchmarks currently prevents co-optimization in application-oriented works, forcing hardware and algorithmic choices to be made sequentially rather than jointly. This benchmark is most directly applicable to application-oriented works, where constraints are dictated by application requirements and the priority is to find the best tradeoff between solution complexity and hardware cost.

All three benchmarks should be organized by distribution change regime, since solutions operating under different regimes are not directly comparable, and should be accompanied by clear instructions for dataset construction to enable the community to extend them to new application domains. Real-world application-oriented works, such as those surveyed in Section 4.2, provide the most concrete inspiration for what these benchmarks should look like in practice.

Addressing label efficiency should be an explicit design requirement of the solution-oriented benchmark rather than a separate concern. Concretely, this means constructing data streams that mix labeled and unlabeled instances in realistic proportions, and evaluating solutions on their ability to exploit unlabeled data alongside the scarce labeled examples they receive. Such benchmarks, particularly if designed for the concept drift and continual learning regimes where application-oriented works are currently almost absent, would reflect the actual conditions under which ODL solutions must operate in the real world. This in turn would create the conditions for developing solutions in these regimes that can be transferred to real deployments with greater confidence, rather than remaining validated only on synthetic or laboratory data streams.

7.0.0.2 End-to-end design of solution components.

As argued in Section 6.3, the surveyed literature predominantly optimizes components of \(A{}\) in isolation, yet the overall efficiency of \(A{}\) depends on all four components jointly. Closing this gap requires not just better benchmarks, but dedicated software tools: libraries and design frameworks that expose all components of \(A{}\) as jointly optimizable objects, support end-to-end evaluation under realistic deployment constraints, and make the cost of each component visible during the design process. No such tool currently exists in the ODL literature, and its development would create the infrastructure needed for principled end-to-end design of \(R{}\), \(B{}\), \(f{}\), and \(L{}\) jointly.

7.0.0.3 Hardware for ODL

As argued in Section 5.4, the dominant architectural pattern in the surveyed literature keeps the heaviest computation in the frozen \(R{}\) inference pass. Hardware that accelerates inference efficiently is therefore currently more impactful for ODL than hardware that accelerates gradient computation.

Looking further ahead, the constraints of ODL differ fundamentally from those of cloud or edge training: labeled data is scarce, validation cannot happen on-device, power budgets are tight, and the solution must remain operational throughout the learning process. A hardware platform designed with these constraints in mind from the outset, rather than adapted from cloud computing systems architecture, could offer qualitatively different trade-offs. Whether such platforms emerge from extensions of current MCU architectures or from entirely new design paradigms remains an open question.

8 Conclusions↩︎

This survey has presented a comprehensive analysis of On-Device Learning for TinyML, organized around a single unifying principle: the distribution change regime that a solution is designed to operate under. By distinguishing between single-change, concept drift, and continual learning regimes, and analyzing each through application, hardware, and solution lenses, the survey provides a framework for comparing ODL solutions in a principled way that prior surveys have not adopted.

From the application lens, a significant structural gap emerges between methodological and application-oriented works. Methodological works cluster around image classification benchmarks too demanding for most real-world TinyML deployments, while the domains that do fit within TinyML constraints lack standardized benchmarks for fair comparison. The works that have succeeded in real deployments share a common characteristic: they treat label acquisition as a first-class design requirement, through few-shot learning, self-supervision, unsupervised approaches, or implicit labeling. Application-oriented works in the concept drift and continual learning regimes remain almost entirely absent, reflecting both the difficulty of obtaining labeled data in non-stationary settings and the absence of realistic benchmarks for these regimes.

From the hardware lens, a clear monotonic correlation emerges between hardware capability and regime complexity: ultra-constrained processors appear almost exclusively in the single-change regime, standard MCUs span all three but are thinly represented in continual learning, and PULP platforms dominate where memory demands are largest. Across the literature, the heaviest computation consistently resides in the frozen feature extractor inference pass, meaning hardware that accelerates inference efficiently is currently more impactful for ODL than hardware designed to accelerate gradient computation.

From the solution lens, the combination of a frozen pre-trained feature extractor and a lightweight learnable classifier is the dominant pattern across technical compositions in all three regimes. Backpropagation optimization concentrates in the single-change regime, while non-stationary regimes rely more heavily on prototype-based classifiers, forward-only learning, and latent replay. A recurring limitation is that components are optimized in isolation, yet the overall efficiency of a solution depends on all four components jointly, and them in an end-to-end manner under realistic deployment constraints remains underexplored.

Taken together, these findings point toward a coherent research agenda with three open challenges. First, the field lacks standardized evaluation frameworks: a solution-oriented benchmark evaluating \(A{}\) across RAM and computational budgets, a hardware-oriented benchmark comparing \(D{}\) on a fixed solution, and a co-design benchmark jointly optimizing \(A{}\) and \(D{}\) under explicit deployment constraints. Second, dedicated software tools are needed that expose \(R{}\), \(B{}\), \(f{}\), and \(L{}\) as jointly optimizable objects, making end-to-end co-design of ODL solutions tractable. Third, hardware designed specifically for ODL constraints (scarce labels, no on-device validation, tight power budgets), rather than adapted from cloud training architectures, could offer qualitatively different tradeoffs than current MCU-class platforms. The framework introduced in this survey, by making the distributional assumptions of each solution explicit and enabling meaningful comparison across the growing body of ODL literature, provides a foundation for addressing these challenges.

9 Mathematical Notation↩︎

Table 2 lists all mathematical symbols used throughout the paper.

References↩︎

---

1. We adopt the term ODL solution rather than ODL model, as used in some prior work, to emphasize that a complete ODL solution must encompass not only the components for model learning and inference, but also those responsible for extracting and managing relevant features from the incoming data stream.↩︎

2. Depending on the application, this may apply to average power over a defined operational window, to instantaneous peak power, or both.↩︎

3. The reverse is also possible: design \(A{}\) first and identify the most efficient or cost-effective \(D{}\) capable of executing it. Co-optimization of both simultaneously is conceivable, but requires explicit weighting of cost, power consumption, and predictive performance against each other.↩︎

4. The additive decomposition assumes sequential execution of the components, which holds for the large majority of the surveyed works. On parallel architectures, components may execute concurrently, in which case \(E{}\!\left(A{}^{t{}}, D{}\right)\) may need to be computed differently.↩︎