1 Introduction↩︎

The Tsetlin Machine (TM), first introduced in 2018 [@Granmo2018TheTM], represents a logic-driven machine learning paradigm that serves as an alternative to Artificial Neural Networks (ANNs). Since its introduction, an extensive family of TM variants has emerged, including [@granmo2019ctm; @abeyrathna2019regression; @jiao2020mctm; @abeyrathna2020weighted; @abeyrathna2021iwtm; @abeyrathna2021cmtm; @yadav2021continuous; @abeyrathna2021recurrent; @yadav2022fuzzy; @yazidi2022reltm; @tarasyuk2024mltm; @Kuruge_2024; @10.3389/frai.2025.1377944; @granmo2026tsetlinmachinegoesdeep], each extending the capabilities of the core framework.

The theoretical foundation of TM can be traced back to the work of Michael Tsetlin, who pioneered the concept of the learning automaton and formulated the theory of automata collective behavior [@1974automation; @PuppetsWithoutStrings; @M.L.Tsetlin_1963]. According to the theory, intricate, system-level behavior can emerge from local interactions of agents following simple rules and performing a set of simple actions. Through local interactions alone, without any centralized controller, the group can pursue and attain complex common goals that exceed the capabilities of any individual agent [@swarm_intelligence]. This emergent perspective underpins the design philosophy of the Tsetlin Machine.

Concretely, TM builds on the Tsetlin Automaton (TA) and propositional logic; it uses collections of logical clauses constructed from Boolean input features to encode and recognize patterns in data. These clauses, shaped through reinforcement-driven updates of the underlying automata, form interpretable logical expressions that capture relevant input–output relationships.

Because its operations are grounded in discrete, logic-based computations rather than continuous-valued matrix operations, TM is inherently simple from a computational standpoint and can be implemented efficiently in both software and hardware. This simplicity translates into reduced energy consumption [@Lei2020FromAT]. Consequently, TM is particularly attractive for resource-constrained and energy-limited environments, such as embedded systems and edge computing platforms [@Wheeldon2020LearningAB; @10.1145/3560905.3568512]. Grounded in the framework of propositional logic, the TM’s underlying decision-making process can be rationalized and explained by human-readable conjunctive clauses.

Data-generating edge devices typically operate under strict constraints on both computation and energy. In these settings, it is also important to safeguard the privacy of locally collected data. Most machine learning methods rely on complex arithmetic operations and therefore consume substantial energy, posing barriers for edge devices, particularly during model training. One possible approach to circumvent this limitation is to delegate the computationally intensive tasks to cloud-based resources. However, this strategy introduces additional drawbacks, notably increased latency, higher energy expenditure associated with data transmission and network communication, and a compromise of privacy by uploading raw data to the cloud.

Instead, it is desirable to perform learning and inference in proximity to the data sources, exploiting and coordinating the local computational resources available at all participating agents.

Federated Learning (FL) [@AbhishekV2022FederatedLC; @mcmahan2023communicationefficientlearningdeepnetworks; @akhtarshenas2024federatedlearningcuttingedgesurvey] is used when the data is physically distributed, e.g., generated from a network of sensors spatially spread, and when there is a need to preserve data privacy. FL offers a paradigm in which a central server orchestrates model training across a collection of devices. FL is typically classified into Horizontal Federated Learning (HFL) and Vertical Federated Learning (VFL) based on the data distribution between the participating agents [@yang2019federatedmachinelearningconcept].

In HFL, agents share the same feature set but have different samples, whereas in VFL, agents possess different feature sets but the same sample set. In HFL, a shared model is trained across several agents. A central server coordinates the training by sending an initial model to the clients. Each client trains the model locally on its private data. The model updates (weight deltas, gradients) are sent back to the server, which aggregates them to improve the global model. The cycle repeats until the model reaches the desired performance. In the standard workflow, each agent uses the globally trained model independently for inference.

In VFL settings, different agents typically hold different feature subsets for the same samples. Each agent has its own local sub‑model that consumes its own features only and outputs an embedding or partial prediction. A central aggregation model (on a server or central agent) combines these embeddings to produce the final prediction. Knowledge Distillation-based approaches [@Li_2024] have emerged as the most prevalent in this category, enabling knowledge transfer between models with differing structures. Whereas conventional VFL paradigms restrict training to shared samples across participating parties, the approach proposed in [@Huang_2023] generalizes this framework by enabling each party to additionally exploit its own private, non-overlapping local samples via a dedicated knowledge transfer mechanism.

VFL is designed for cross-silo environments where multiple parties possess data on the same set of users or entities but operate in distinct domains, are willing to cooperate, and do not have conflicting interests. It is particularly advantageous when data cannot be consolidated into a single repository because of privacy constraints, regulatory requirements, or organizational barriers. Prominent VFL applications include collaborative healthcare analytics, banking, credit risk assessment, and e-commerce [@wu2025verticalfederatedlearningpractice]. It is also employed to jointly train large-scale machine learning models in a distributed fashion [@JMLR:v22:20-815].

Decentralized Federated Learning (DFL) [@kairouz2021advancesopenproblemsfederated] eliminates the central server; agents only communicate with their neighbors to reach a consensus model. DFL offers a communication advantage and further preserves data privacy.

FedMD [@li2019fedmdheterogenousfederatedlearning] is a heterogeneous federated learning method based on knowledge distillation. Every agent maintains its own private dataset and an individually tailored model. The information learned by each agent is converted into a common representation, which is then aggregated by a central server to derive a consensus that is shared back with the agents.

Federated learning was proposed for the Tsetlin Machine in [@Qi2023FedTMMA; @qi2025fedtmos; @gohari2024tpfltsetlinpersonalizedfederatedlearning], all of which use a central server for model aggregation. In this work, we investigate a TM-based decentralized learning framework under a vertical federated learning data partitioning scheme, wherein agents hold disjoint feature subsets of the same data instances and rely on peer-to-peer communication to achieve consensus-based prediction (inference). The agents may be diverse, such that each agent may have its own machine learning model, which is not shared with the other agents, thereby further increasing the heterogeneity of the learning process.

Our novel approach is grounded in a two-layer hierarchical TM architecture, to the best of our knowledge, the first of its kind, which is paired with a gossip-based communication protocol. This configuration enables agents to exchange information efficiently with their neighboring peers, while simultaneously precluding the direct transmission of raw data and explicit model parameters, to safeguard the confidentiality of both (formal leakage analysis is left for future work). At the first layer of the hierarchy, which we name the “input layer", the TM operates on locally available features and performs classification tasks using only this subset of features. At the second layer, which we name”the neighborhood aggregation layer", the framework consolidates the rule-based outputs produced by each agent with those obtained from its neighboring agents. Through this aggregation over a cluster of interconnected nodes and an expanded, more heterogeneous feature space, the second tier can recognize and characterize more intricate input patterns that cannot be captured by isolated local models alone. The proposed architecture is presented in Fig. 1.

Main contributions↩︎

The main contributions of this work are as follows:

• We introduce a hierarchical TM framework that enables distributed, decentralized learning under aligned samples and a vertical feature-partitioning setting with consensus-driven inference. This framework is designed to preserve the confidentiality of both the underlying model parameters and the local data of each participant.

• The proposed paradigm accommodates heterogeneous TM-based agents with differing data acquisition means, local data distributions, or computational resources, thereby facilitating the integration and fusion of information in settings such as large-scale sensor networks and multi-modal sensing environments.

• We propose an inter-agent asynchronous gossip-based communication protocol for coordination and data exchange.

• We analyze the model’s performance, using software implementation, on the MNIST and Fashion-MNIST datasets, partitioned among several agents.

• We analyze the model’s performance on a synthetic heterogeneous sensor network setup, in comparison to a centralized ANN model.

• We assess the sensor network model under two-dimensional grid and general connected graph network topologies.

• We evaluate performance as a function of system configuration variants such as the number of agents (dataset partitions) and the TMs’ hyperparameters in each hierarchy.

Organization↩︎

The remainder of this paper is structured as follows. Section 2 provides an introduction to TM and reviews previous work related to FL using TM, Section 3 describes the proposed algorithm for collaborative decentralized TM learning, and Section 4 presents performance results over the MNIST handwritten digit recognition benchmark, the Fashion-MNIST benchmark, and a heterogeneous sensor network model. Section 5 summarizes this work.

2 Technical Background↩︎

This section outlines the architecture of the TM model, explains the core principles underlying its operation, and specifies its hyperparameter definitions. In addition, it presents techniques used in distributed machine learning settings.

2.1 TM Overview↩︎

A block diagram of TM is presented in Fig. 2 (a).

The Tsetlin Machine (TM) performs classification of input features by employing conjunctive clauses to detect discriminative patterns. The input space is represented by Boolean features together with their complements, collectively referred to as literals. Half of the clauses (positive clauses) are dedicated to recognizing patterns associated with the target class, whereas the remaining clauses (negative clauses) are responsible for identifying patterns indicative of non-membership in the target class.

The final classification decision is obtained by summing the outputs of the positive and negative clauses, taking into account their respective signs, and comparing the resulting aggregate score to a predefined threshold. This procedure effectively implements a voting mechanism over the set of clauses.

Each clause is associated with a team of Tsetlin Automata (TAs), with one automaton assigned to each literal. A TA is a finite-state machine (FSM) employed to construct logical clauses through the inclusion or exclusion of literals. Depending on its current state, a TA chooses between two possible actions: Include or Exclude. For every action executed, the Learning Feedback module produces either a Reward, Penalty, or Inaction, following a probability distribution that depends on the specific literal assigned to the automaton as well as the relationship between the clause output and the expected output. The TA updates its internal state in response to this feedback to rapidly converge to the action associated with the highest reward probability. Figure 2 (b) depicts an example of a TA comprising six states.

A Tsetlin Machine is characterized by two principal hyperparameters, denoted by \(T\) and \(s\). The threshold parameter \(T\) regulates the target magnitude of the aggregate clause vote. Consequently, a higher threshold value entails that a greater number of clauses contribute to the voting process and thereby affect the feedback dynamics governing the TA states. During the training phase, the probability of issuing feedback increases as the deviation of the clause vote from the threshold \(T\) grows. In contrast, when the clause vote reaches or surpasses the threshold \(T\), the feedback mechanism is effectively suppressed and no further feedback is provided.

The hyperparameter \(s\) regulates the probability with which TAs transition between internal states. It controls the sensitivity (Specificity) of the learning process. A high value of \(s\) promotes the learning of specific patterns by forming specialized clauses, while a low value of \(s\) generates more general clauses.

Careful tuning of these parameters can be used to determine learning stability and robustness [@Granmo2018TheTM].

As illustrated in Fig. 2 (b), a TA updates its state in response to one of three possible forms of feedback: Reward, Penalty, or Inaction. Two distinct feedback mechanisms are defined within the TM learning framework: Type I feedback and Type II feedback, which are specified in Table 1 and Table 2, respectively.

Type I feedback reinforces true positive output (Type Ia feedback) and reduces false negative output (Type Ib feedback). When the clause outputs "1" for the target class, if a TA action is to include a literal valued ‘1’, it receives reward feedback with probability \(\frac{s-1}{s}\), otherwise “inaction" occurs with a probability of \(\frac{1}{s}\). If the TA action, in this case, is to exclude a literal valued ‘0’, it receives a reward feedback with probability \(\frac{1}{s}\), otherwise,”inaction" occurs with a probability of \(\frac{s-1}{s}\). In case the clause outputs ‘0’ for the target class, the feedback is directed to combat the false negative output. Given the need for clause reconstruction, the feedback penalizes all inclusions while rewarding all exclusions.

Type II feedback is activated when the clause outputs ‘1’ for a pattern that does not belong to the target class. In this case, the exclusion of a literal that is ‘0’ is penalized.

TM has been shown to be effective in image classification [@granmo2019ctm], IoT applications [@10.1145/3560905.3568512; @Wheeldon2020LearningAB; @10455063], low-power speech recognition [@lei2021lowpower], and Natural Language Processing (NLP) [@https://doi.org/10.1111/exsy.12873; @bhattarai2023tsetlin; @bhattarai2020measuring; @Yadav_Jiao_Granmo_Goodwin_2021].

For reference, Table 3, reproduced from [@granmo2019convolutionaltsetlinmachine], shows the MNIST and Fashion-MNIST test accuracy results, comparing classic TM with various machine learning algorithms. A comparative analysis of TM to Neural Networks regarding learning convergence and energy efficiency is presented in [@Lei2020FromAT].

2.2 Federated Learning with Tsetlin Machine↩︎

Inspired by conventional Federated Learning with Convolutional Neural Networks (CNNs), FedTM [@Qi2023FedTMMA] is a framework that uses Tsetlin Machine in FL. As in HFL, all FedTM clients use a global model. The server performs aggregation in two steps: aggregation of clause weights (AverageCW) and aggregation of clause states (TopK). AverageCW calculates the weighted average of integer clause weights for each class across the participating \(J\) clients, based on their dataset sizes. TopK aggregates the states from a subset of \(K\) clients for a specific class. The states are selected from clients with the highest number of samples per class, indicating greater confidence, and are combined using the bitwise OR operator. Compared to conventional Federated Averaging (FedAvg) with CNNs, on average, FedTM provides a substantial reduction in communication costs by 30.5× and 36.6× reduction in memory footprint.

Tsetlin-Personalized Federated Learning with Confidence-Based Clustering (TPFL) [@gohari2024tpfltsetlinpersonalizedfederatedlearning] was introduced to tackle a key challenge in federated learning (FL): the heterogeneity of data distributions across clients. In this approach, a client’s model parameters are aggregated only when the client is sufficiently confident in what it has learned, meaning it has seen enough training samples for a given class. This notion of confidence is derived from the voting mechanism of the Tsetlin Machine (TM) algorithm: a larger number of votes in favor of a class indicates higher confidence in the corresponding prediction. Each client transmits to the server its weight vector along with the label of the class for which it has the highest confidence. This substantially lowers the communication overhead. The server then partitions the clients into \(K\) clusters, where cluster \(k\) consists of those clients whose maximum confidence score is associated with class \(k\). Within each cluster \(k\), the clients’ weight vectors are aggregated.

One-Shot Federated Learning With Tsetlin Machine, FedTMOS [@qi2025fedtmos], restricts communication with the server to a single round, thus minimizing communication errors and reducing the risk of interference caused by iterative updates. Given \(J\) clients, each having local datasets \(D_1,D_2, ...,D_J\) . The objective is to aggregate local TM models, \(T = \{T_1, T_2, ..., T_J\}\), into \(\phi\) server models \((\phi < J)\) that generalize well over all datasets in one communication round. The aggregation of the models is done by applying the principles behind TM Composites [@granmo2023tmcompositesplugandplaycollaborationspecialized]. In the initial step, clients upload their scaled clause weights, which are adjusted based on the proportion of samples per class relative to the client’s total sample size. Alongside these weights, clients also upload their individual normalized Gini index, which quantifies the inequality in their local data distributions. The server then rescales the weights using the mean normalized Gini Index and performs k-means clustering on the weights. This inter-class weight separation technique is applied to create models that enhance class distinction. Finally, the \(\phi\) server-side models are initialized, and class weights from each cluster are reassigned to maximize inter-class separation within each model.

2.3 Motivation↩︎

The TM has been introduced as an alternative learning architecture grounded in logic-based, bit-level operations that naturally map to hardware implementations. Due to its reliance on simple, low-complexity computations rather than continuous-valued arithmetic, the TM is particularly well suited to severely resource-constrained and energy-limited environments, including dense sensor networks and other edge-computing scenarios. Several works on TM-based federated learning have recently been published [@Qi2023FedTMMA; @qi2025fedtmos; @gohari2024tpfltsetlinpersonalizedfederatedlearning], however, the domain of distributed and decentralized TM learning has, to a large extent, remained unexplored and has received minimal attention in the current literature. In this work, we consider a model-heterogeneous decentralized learning setting, where agents may employ different local model architectures and collaborate without exposing the local data or the underlying model. We assume that training samples are vertically aligned and that the class labels are available to all agents.

3 Algorithm Design↩︎

The proposed distributed and decentralized learning algorithm is defined over an underlying undirected, fixed-in-time, and connected communication graph. The experimental evaluation in this study primarily considers a two-dimensional cellular lattice of size \(n \times n\), in which each cell contains a single agent and is connected to its immediate eight neighboring cells. To eliminate boundary artifacts arising from agents having fewer than eight immediate neighbors, we consider a toroidal wrap-around grid. In addition, we assess the algorithm’s performance on a more general graph network topology.

Each agent (for instance, a sensor node in a distributed sensor network) acquires and processes its own local dataset, which remains strictly private and is never directly shared. During both the training (learning) phase and the prediction (inference) phase, agents cooperate only with their immediate neighbors in the grid topology, exchanging information in a way that does not reveal their underlying private models. The collective objective during inference is for the distributed ensemble of agents to arrive at a shared, globally consistent decision or estimate, i.e., to achieve consensus across the entire grid.

Our proposed architecture is organized as a two-layer hierarchy, where each layer contains a TM, as depicted in Fig. 1. The TM in the first layer (the Input Layer) operates directly on the Boolean input features. During the learning phase, it forms clauses that specialize in recognizing particular patterns present in these input features.

The TM in the second layer (Neighborhood Aggregation Layer) receives, as its input, the outputs of the clauses from the first-layer TM associated with its own cell, together with the clause outputs originating from the eight nearest neighboring cells. Based on this combined input, the Neighborhood Aggregation Layer TM learns to detect higher-level, more global response patterns emerging from the local cluster comprising the cell itself and its nearest neighbors. In a multi-class classification scenario, each agent maintains a pair of TMs for every class.

As empirically demonstrated in Section 4, the proposed aggregation layer yields superior performance relative to simpler, non–learned aggregation strategies, such as voting-sum aggregation. In particular, the clause-output states exhibit a stronger and more precise correlation with the clusters’ input features than those produced by the aggregated voting sum. The aggregation-layer TM effectively learns the activity patterns of the clauses within the cluster.

Crucially, only the binary clause outputs are exchanged between neighboring cells, while the internal structure and composition of the clauses remain private and are not revealed. Because each clause output can be encoded as a single bit, the communication overhead between neighbors is kept very low, substantially reducing the volume of data that needs to be transmitted.

3.1 Distributed Learning↩︎

The complete training workflow is detailed in Algorithm [alg:training32algorithm]. The training of an agent is carried out in two distinct stages. In the first stage, the Input Layer TM is trained exclusively from the Boolean feature representation provided as input (lines 6, 7). During the second stage, the Neighborhood Aggregation Layer TM is trained (line 10) while the Input Layer TM, as well as the Input Layer TMs of neighboring agents, are kept in inference mode, thereby supplying their outputs as input signals. The Input Layer clause outputs of the nearest neighbors are gathered (lines 8, 9) via a dedicated communication protocol, the details of which will be presented and analyzed in Section 3.2.

In a multi-class classification scenario, each agent maintains a pair of TMs for every class. For a given training instance, the TM pair (consisting of the Input Layer and the Neighborhood Aggregation TMs) associated with the instance’s ground-truth label is updated first (lines 6-10). After this label-specific update, the training step is repeated for another pair of TMs corresponding to a different class (lines 11-15), which is selected at random but known to all agents. This sequence of operations defines a single training iteration for the agent.

3.2 Inter-Agent Communication↩︎

Inter-agent communication is employed both to coordinate agents’ activities and to transfer necessary data for processing. In the context of distributed edge computing, the design of the underlying communication protocol becomes a key architectural concern, because the energy overhead associated with transmitting and receiving data typically exceeds the energy cost of performing the computations themselves [@Zhao02112018]. Our model assumes a wireless (or other shared media) communication setting where a single transmission can be overheard by multiple nodes. We employ a gossip random broadcast scheme, akin to that presented in [@4787122] (nodes asynchronously broadcast to their neighbors when information is ready). The broadcast mechanism enables an agent to disseminate information to all of its neighboring nodes in a single transmission, thereby accelerating convergence and reducing communication overhead. Furthermore, the use of an asynchronous protocol ensures that each agent operates autonomously, with less dependency on other nodes. This obviates the necessity for a globally synchronized master clock and mitigates delays or bottlenecks associated with synchronization, allowing the system to function in a decentralized and temporally flexible manner.

3.2.1 Clause Outputs Update↩︎

After training its Input Layer TM for the given example, the agent calculates the output of the corresponding Input Layer TM clauses. Then it broadcasts these output values to its immediate neighboring agents located within its connectivity radius. Messages are tagged with sample identifiers to ensure that clause outputs correspond to the same training instance. Each clause output can be encoded as a single bit, substantially reducing the volume of data that needs to be transmitted. All agents that successfully receive this communication accordingly update their internal state. Once the originating agent has collected the updated information from each of its nearest neighbors, it initiates the training procedure for its Neighborhood Aggregation Layer TM. However, if any communication packet is lost during this exchange, the agent may omit the Neighborhood Aggregation Layer TM training phase for that cycle.

3.2.2 Class Vote Sum Update↩︎

Algorithm 4 presents the procedure executed by each agent. For class prediction (inference), each agent’s local class vote sums need to be disseminated to all other agents, so that a network-wide consensus on the predicted class can be achieved. In this context, consensus denotes a state in which every agent possesses complete information regarding the vote sums of every other agent, such that each node can compute an identical, globally consistent class decision. To accomplish this, we apply a multi-source, all-to-all dissemination protocol.

Each node maintains a vote-sum table that contains one entry for every node in the grid. At the beginning of the procedure, each node transmits its own current vote sum to its eight immediately adjacent neighbors (line 11). When a node receives a message, it updates the corresponding entries in its local vote-sum table with the newly obtained values (lines 12-21). Furthermore, whenever a node acquires previously unknown vote-sum information, it propagates this new information by broadcasting it to its nearest neighbors (gossip with aggregation) (lines 22-24). A given node can obtain identical data from several independent upstream sources. This redundancy may improve the resilience of the protocol to communication impairments, for example, in the presence of packet loss, intermittent connectivity, or transient link failures, the likelihood that the node still receives the required information remains high. The impact of communication impairments was not evaluated in this study and is deferred for future work. Through repeated local exchanges of this kind, the complete set of vote sums is gradually diffused throughout the grid, enabling all agents to converge on a shared global decision. Assuming reliable delivery and a fair scheduler, all agents become fully informed of all values in \(\Theta(n)\) time. The total number of broadcast events is \(\Theta(n^2)\) (every agent has to transmit at least once) [@2821576; @355459; @1238221].

4 Experimentation↩︎

In the initial assessment of our distributed ML model, we operate under the assumption of ideal communication among all agents; consequently, practical network imperfections such as packet loss, delays, or message corruption are not yet incorporated into the analysis. For empirical evaluation, we employ the MNIST dataset of handwritten digits as well as the Fashion-MNIST dataset.

The distributed computational architecture is organized as an \(n \times n\) two-dimensional lattice, yielding a total of \(n^2\) agents. Each image from the datasets is spatially partitioned into \(n^2\) equally sized tiles, with each tile uniquely assigned to one agent in the grid. All agents share the same TM hyperparameters (number of clauses, threshold \(T\), and specificity parameter \(s\)), ensuring a homogeneous configuration at the architectural level. Nevertheless, each agent’s internal model is adapted to the local data it receives and thus learns to detect and classify patterns that are characteristic of the particular image tile for which it is responsible.

4.1 MNIST Benchmark↩︎

The MNIST dataset [@lecun2010mnist] is composed of images with a resolution of \(28 \times 28\) pixels. Classification performance on this dataset is reported for two distinct experimental setups. In the first setup, the system is composed of 49 agents, where each agent is responsible for processing a \(4 \times 4\) pixel subregion (tile) of the original image. In this configuration, each tile covers approximately \(2.05\%\) of the entire image area. The collective decision of the ensemble of 49 participating agents is then used to assign a class label to the complete image. Fig. 5 presents the vote patterns of the 49 agents associated with the class representing the digit “7" and of those associated with the class representing the digit”2" for two images. The number in each tile is the agent’s vote for that tile. In the first experiment, an image of a handwritten 7 is provided as input, whereas in the second experiment, the input is an image of a handwritten 2. As the chart shows, the aggregated class-wise vote sum serves as a robust discriminator, reliably separating the true class from competing classes.

Table [tab:MNIST324932agents] reports the test accuracies obtained after 50 training epochs by an ensemble comprising 49 agents, where each agent is responsible for learning a \(4 \times 4\) image patch. The table reports test accuracy results obtained from single-layer and two-layer agent architectures. The performance of the two-layer architecture is substantially better than that of a single layer. The results are shown for multiple architectural settings of both the Input Layer TM and the Neighborhood Aggregation TM, enabling a comparison of performance across different configuration choices.

Table [tab:MNIST321632agents] presents the test accuracies obtained after 50 training epochs by an ensemble comprising 16 agents, where each agent is responsible for learning a \(7 \times 7\) image tile. In this setup, each agent learns a tile that contains 6.25% of the whole image. The table reports test accuracy results obtained from single-layer and two-layer agent architectures, showing the superior performance of the two-layer architecture.

4.2 Fashion-MNIST Benchmark↩︎

The Fashion-MNIST dataset [@Xiao2017FashionMNISTAN] consists of images with a resolution of \(28 \times 28\) pixels, depicting items from 10 distinct categories of apparel (such as T-shirts, trousers, and bags). We begin by analyzing the behavior of an ensemble model composed of 49 agents, where each agent processes a dedicated \(4 \times 4\) pixel patch of the input image, corresponding to 2.05% of the entire image area.

For the 49-agent Fashion-MNIST setup, Fig. 6 illustrates the voting behavior of the agents for three specific classes (pullover, shirt, and ankle boot), along with the resulting aggregated class scores, for two example input images: a Pullover and an ankle boot. The number in each tile is the agent’s vote for that tile. In both examples, the correct class label is clearly identifiable from the overall class sum, which stands out relative to the competing classes.

For the pullover input, the Shirt class also attains a positive cumulative vote, reflecting the visual similarity between shirts and pullovers in the dataset (e.g., similar silhouettes or overlapping features). Nevertheless, the pullover class still exhibits a substantially higher class sum than the shirt class, indicating that the ensemble is able to correctly disambiguate between these two visually related categories.

Table ¿tbl:F-MNIST324932agents? summarizes the test accuracies achieved for the Fashion-MNIST dataset after 50 training epochs by an ensemble of 49 agents, with each agent dedicated to modeling a distinct \(4 \times 4\) image patch. The table presents results for several architectural parameters of both the Input Layer TM and the Neighborhood Aggregation TM.

Table ¿tbl:F-MNIST321632agents? summarizes the test accuracies achieved for the Fashion-MNIST dataset after 50 training epochs by an ensemble of 16 agents, with each agent dedicated to modeling a distinct \(7 \times 7\) image patch. The table presents results for several architectural parameters of both the Input Layer TM and the Neighborhood Aggregation TM.

4.3 Sensor Network↩︎

In this experiment, we model distributed learning across a sensor network with heterogeneity in sensor data.

The setup employs 25 agents arranged in a \(5 \times 5\) two-dimensional lattice. The class semantics (3 classes) are common to all agents; however, the mapping of sensor data to those classes differs across clients. Feature distribution heterogeneity is applied by:

• Agents having one or two sensors (structural heterogeneity).

• Sensors having different scaling depending on the agent id (sensor 0: \(\times1.8/1.0\), sensor 1: \(\times0.6/1\))

• Sensor 0 having different offset depending on agent id (\(1.2/none\))

• Different noise levels on sensor 0 and 1 (zero mean, \(std\;0.3/0.4/0.8\))

For each class, the dataset is generated by sampling a base mean uniformly from \([-1,1]\). The variance parameters are independently sampled from \([2,3]\), and the resulting values are used to generate samples from a multivariate Gaussian distribution. A sensor-heterogeneity model is then applied to introduce agent-specific adaptations and capture inter-agent variability in the sensing process. Fig. 7 presents the distribution of sensor data for 9 agents.

4.3.1 Classification by ANN↩︎

Due to the overlap between classes, per-agent classification yields low accuracy. Table 4 presents the performance obtained using ANN to classify agent2 data samples. The per-agent accuracy ranged from 43% to 57.5%. The architecture of the ANN used is:

Since each agent is just a relatively small, noisy sample from that global distribution, training on a single agent’s sample set cannot recover the full decision boundary. The aggregation of samples from all 25 agents was then used to train an ANN model with the following architecture:

The combined model learns a population-level rule across agents with an accuracy of 93%. The significant difference between local and global separability is presented in Table 4.

4.3.2 Classification by TM Ensemble↩︎

The sensor measurements are inherently analog quantities that can be modeled as real-valued variables. Since the TM operates on Boolean-valued input features, these continuous variables must first be transformed into a discrete binary representation.

To achieve this, the continuous value range of each feature is partitioned into a predefined number of bins. The binarization strategy adopted in this study determines the bin thresholds via an order-statistic–based procedure, implemented as approximately evenly spaced selections from the sorted list of unique observed values. Concretely, all unique feature values are first extracted and sorted; the smallest unique value is then discarded, and the remaining unique values serve as threshold candidates. From this ordered set, thresholds are chosen at approximately uniform index intervals, yielding an approximation to Quantile Binning [@9923830].

Under ideal Quantile Binning, the continuous data range is partitioned into a given number of bins such that each bin contains approximately the same number of samples. Relative to binning schemes based on fixed, a priori thresholds (e.g., uniformly spaced in the value domain), quantile-based binning is generally more robust for skewed or otherwise non-uniform data distributions, as it promotes a more balanced allocation of samples across bins.

Each feature value is subsequently encoded in a cumulative (monotone) binary fashion with respect to the bin thresholds. For example, if the value range is divided into four bins defined by three thresholds,

the feature value \(x\) is encoded in three bits:

Each sensor sample is binarized to 8 bits (for a two-sensor agent) or 16 bits (for a single sensor agent).

4.3.2.1 A grid topology

The 16-bit input feature vector for each agent is first reshaped into a \(4 \times 4\) tile. A total of 25 such tiles (one per agent) are then arranged in a \(5 \times 5\) two-dimensional lattice. Fig. 8 shows several randomly selected examples of the resulting \(20 \times 20\) patterns, each representing the global feature set (all 25 agents). When data from all agents are pooled, a global class structure emerges in the 2D feature space.

In our setup, each of the 25 agents employs an Input Layer TM with 32 clauses and hyperparameters T=3, s=3. The Aggregation Layer TM is of 16 clauses and hyperparameters T=4, s=10. As reported in Table 4, the TM ensemble achieves 93% accuracy, the same level of accuracy achieved by the centralized ANN model. The classifier leverages the global class-conditional structure across agents, which yields substantially higher accuracy on the pooled dataset than on individual agents. The chart in Fig. 9 shows classification accuracy as a function of the number of participating agents for both the centralized ANN and distributed TM models. As the number of participating agents increases, the accuracy of the distributed TM progressively converges toward that of the centralized ANN.

Table 5 presents the resources used by each agent and the overall system in comparison to a baseline single TM. While each local TM is relatively small, the total number of clauses and automata in the entire system exceeds that of a single traditional TM baseline. Nevertheless, a system made up of multiple small agents can achieve the same classification accuracy by utilizing their private, non-overlapping local samples.

4.3.2.2 Connected graph topology

To assess the performance of the algorithm on a more general graph topology, as opposed to the regular \(n \times n\) two-dimensional lattice, we consider the network topology illustrated in Fig. 10. In this setting, agents possess one, three, or four immediate neighbors, implying that Aggregation Layer TMs associated with different agents operate over feature sets of varying cardinalities, thereby inducing heterogeneous model architectures across the network. As reported in Table 6, the obtained accuracy of 76% is comparable to that achieved by both the centralized ANN model and the two-dimensional configuration comprising 9 participating agents (see also Fig. 9).

5 Summary↩︎

We propose a hierarchical TM-based framework that supports distributed and decentralized learning, coupled with consensus-oriented inference procedures. The model assumes that training samples are vertically aligned and that the class labels are available to all agents. The framework is explicitly constructed to safeguard the privacy of both the model’s internal parameters and each participant’s locally held data. It naturally accommodates heterogeneous TM-based agents that may differ in their sensing modalities, data acquisition processes, underlying data distributions, or available computational power, thus enabling flexible integration and fusion of information in complex settings such as multi-modal sensing scenarios. During inference, the individual predictions produced by each agent’s local model are aggregated to form a shared, system-wide consensus estimate. Throughout this process, communication is restricted to exchanges between an agent and its immediate neighbors in the network topology, reinforcing both scalability and privacy. Experiments were conducted using the MNIST, Fashion-MNIST, and Sensor Network datasets, employing two-dimensional grid and connected graph network topologies. The classification accuracies achieved in these experiments are comparable to those of centralized models. Additionally, the models used by the agents are relatively lightweight, highlighting the potential for edge computing.

In our experimental study, we assume ideal (i.e., error-free and latency-free) communication channels. A more comprehensive evaluation under realistic communication conditions and deployment environments is left for future work.

6 Data Availability Statement↩︎

The data that support the findings of this study are available from the corresponding author, Y. R., upon reasonable request.

---

1. Y. Rudin, M. Yemini, and A. Fish are with Faculty of Engineering, Bar Ilan University, Ramat Gan 5290002, Israel. Emails: yehuda.rudin@biu.ac.il, osnat.keren@biu.ac.il, michal.yemini@biu.ac.il, alexander.fish@biu.ac.il.↩︎