The Missing Dimensions in Geo-Distributed Database Evaluation
May 28, 2026
Abstract
Geo-distributed OLTP databases are widely deployed across cloud regions, yet current evaluation practices do not cover the challenges of this aspect. Existing benchmarks assume stable network conditions; they lack explicit settings for data and client locality, and they largely ignore data transfer costs across regions. In addition, most evaluations rely on a limited set of geo-distribution patterns. In this paper, we propose Gaia, a comprehensive evaluation framework that addresses these gaps. We use Gaia to perform a comprehensive evaluation of existing geo-distributed OLTP systems. We deploy them across multiple cloud regions, using different geo-distribution patterns and variable cross-region network conditions. Among other interesting findings, our framework reveals that: \(i)\) most systems are sensitive to network instabilities, \(ii)\) network costs dominate cloud deployment expenses \(iii)\) multi-region fault-tolerance mechanisms incur measurable critical-path overhead that is often overlooked in prior evaluations. We argue that for the design of future geo-distributed databases, we must rethink the trade-offs between performance, fault-tolerance, and cost.
1 Introduction↩︎
Geo-distributed transactional databases are fundamental to modern applications across banking, e-commerce, and financial trading. By partitioning and replicating data across different geographical regions, these systems increase availability, tolerate regional failures, and provide low-latency access to users. However, ensuring strong consistency and atomicity across regions requires complex coordination protocols that may entail multiple communication rounds. Due to wide-area network (WAN) latencies and cross-region communication, transactions may incur significant overhead compared to single-region deployments. Such coordination and concurrency control protocols have been the subject of research in recent years [1]–[9].
To manage these trade-offs, many systems adopt deterministic execution models [3], [4], in which transaction order is determined before execution. Determinism is particularly attractive in geo-replicated settings, where timestamp-based approaches suffer from clock skew and message delays [10]. Without determinism, conflicting transactions may be aborted or reordered inconsistently, leading to replica divergence. By processing transactions in a predefined global order, deterministic databases enforce convergence and reduce coordination.
| Evaluation Scenario | Metric | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| 2-7 (lr)8-9 | Scalability | Access patterns | Access skew | Network conditions | Fault tolerance | Resource allocation | Data transfers | Monetary costs | |
| Calvin [4] | |||||||||
| SLOG [3] | |||||||||
| Detock [1] | |||||||||
| Caerus [5] | |||||||||
| Aria [11] | |||||||||
| CRDB [6] | |||||||||
| Spanner [12] | |||||||||
| Aurora [13] | |||||||||
| TAPIR [14] | |||||||||
| Janus [15] | |||||||||
| Mako [16] | |||||||||
| MaaT [17] | |||||||||
| EC [18] | |||||||||
| Q-Store [19] | |||||||||
| Deneva [2] | |||||||||
| CRDB-MR [20] | |||||||||
| DB sen. [21] | |||||||||
Despite the growing adoption of geo-distributed systems in industry (e.g., Spanner [12], Aurora [13], CockroachDB [6]), current benchmarking practices do not capture the intricacies of their deployments. Standard OLTP benchmarks such as TPC-C [22], YCSB [23], and SmallBank [24] were originally designed for traditional single-region deployments more than a decade ago. As we show in this paper, their schemata and data distributions offer limited support for modeling important geo-distributed transactional patterns. However, more complex geo-distribution patterns would better reflect the variations of real-world workloads. Moreover, most evaluations [1]–[4], [11], [14], [15], [21] focus on scalability and skewed data distribution, but they rarely model failures, unstable networks, or heterogeneous cloud infrastructures (1). Although WAN communication can be costly in cloud deployments, data transfer volumes and pricing implications are often overlooked in the literature. Given the increasing adoption of geo-distributed databases nowadays, novel benchmarking methods, evaluation scenarios, and metrics have become especially important and timely.
To address these issues, we develop Gaia, a benchmarking framework tailored to geo-distributed OLTP databases. We use Gaia to evaluate existing state-of-the-art geo-distributed databases across scenarios that were not tested in their original publications. Gaia integrates a range of OLTP systems and evaluates them under different scenarios derived from existing benchmarks such as TPC-C [22] and YCSB [23] (1). These scenarios model realistic conditions, including transaction locality, submission-placement asymmetry, hardware instability, and varying hardware configurations. In addition, Gaia introduces new metrics, including cross-region data transfers and monetary costs, to provide a complete picture of system behavior under varying deployment conditions. Gaia is designed to be extensible, allowing new geo-distributed protocols to be integrated with minimal effort.
Using Gaia, we evaluate five geo-distributed OLTP systems deployed across multiple cloud regions. We analyze system performance across eight scenarios and examine how design decisions affect performance and costs in WAN deployments. Note that our goal in this work is not to compare systems but to identify key limitations in current evaluation approaches and propose new metrics and evaluation scenarios.
In short, this paper makes the following contributions:
The first geo-distributed database evaluation framework that considers data transfers, and cloud deployment costs alongside the traditional latency and throughput metrics.
A taxonomy of scenarios for geo-distributed transactions.
A comprehensive performance evaluation using our framework of five database systems in different deployment scenarios, yielding new insights on geo-distributed database designs and their tradeoffs.
Identification of high-latency cross-regional data transfers as the primary cost for geo-distributed systems.
Quantification of how network instability affects the performance of existing systems.
Demonstration of the overhead that multi-region fault-tolerance mechanisms impose on transaction performance in geo-distributed systems.
The rest of the paper is organized as follows. We first provide an access pattern taxonomy (2), summarize geo-distribution modeling in existing benchmarks (3) and system evaluations (4). We derive a more holistic set of evaluation scenarios (5) and metrics (6). In (7) we discuss the insights our framework reveals about system capabilities and outline open challenges (8). (9) provides a related work overview.
Gaia is available at https://github.com/delftdata/gaia. We present additional metrics in our extended report [25].
2 Transactional Access Patterns↩︎
In addition to multiple works omitting specific benchmarking scenarios, we also observe a lack of critical workload characteristics in the two most popular benchmarks, YCSB [23] and TPC-C [22]. In this section, we define transaction types and analyze their prevalence in the state-of-the-art evaluation benchmarks.
Similar to [1], [3], [5], Gaia, classifies transactions based on two properties that determine their access patterns: \(i)\) the number of partitions accessed and \(ii)\) the number of regions involved. We distinguish between: Single-Partition (SP) transactions, which access data within a single partition, and Multi-Partition (MP) transactions, which access data in two or more partitions. However, unlike prior work which classifies transactions as single-home or multi-home, Gaia refines this categorization into three distinct classes based on the location of the client initiating the transaction. The different transaction types are summarized in 2. 2 illustrates the differences between these three classes on a simplified TPC-C schema. Let \(R(t)\) denote the set of regions involved in a transaction \(t\), and let \(O(t)\) be the origin region of \(t\), i.e., the region where the transaction was submitted. We distinguish the following three types of transactions:
Definition 1 (Local Single Home (LSH)). A transaction is considered LSH if the primary replica of all data objects it accesses resides within a single region, and this region matches the transaction’s region of origin. LSH transactions do not require cross-region round-trips as the data is located in the same region as the query. Formally: \(\quad |R(t)| = 1 \quad \text{and} \quad O(t) = R(t).\)
Definition 2 (Foreign Single Home (FSH)). An FSH transaction also accesses the primary replica of all data objects withing a single region, but that region differs from the region where the transaction was submitted. FSH transactions require the forwarding of the transaction to the respective home region of the data and sending back the result to the region of origin (i.e., at least one round trip is required). Formally: \(\quad |R(t)| = 1 \quad \text{and} \quad O(t) \neq R(t).\)
There are two key reasons for distinguishing FSH and LSH: \(i)\) they exhibit different communication patterns during transaction processing. An FSH transaction is submitted to a region that differs from the region where the data is actually mutated. FSH transactions may thus incur higher latency than LSH, since an additional round-trip is required for data collection and, \(ii)\) network costs are involved when exchanging data across the two regions for FSH.
Definition 3 (Multi Home (MH)). A transaction is MH if it accesses the primary copies of data across two or more regions and requires coordination across regional database instances. Formally: \(\quad |R(t)| \geq 2.\) MH transactions require at least one round trip to send the transaction and gather the results. In addition, they may require additional rounds to coordinate with other regions involved in the transaction to resolve potential concurrency conflicts.
Partitions vs. Homes. Note that a given benchmark assigns a set of partitions to each region. By default, the home of a given primary partition is assigned randomly and uniformly. A partition may be replicated in other regions, but this does not affect the definition of LSH, FSH, and MH transactions: coordination occurs across primary (also called master) replicas, not secondary replicas.
| Name | Transactional Access Pattern |
| Single-Partition (SP) | Within a single partition |
| Multi-Partition (MP) | Across two or more partitions |
| Local Single-Home (LSH) | Within a single region (the one where the transaction originates) |
| Foreign Single-Home (FSH) | Within a single region (different from the transaction’s origin) |
| Multi-Home (MH) | Across two or more regions |
Discussion. In real-world systems, challenges arise from the variations of transactions’ data access patterns. For instance, FSH transactions can occur when an e-commerce customer orders goods stored in a warehouse abroad. Systems like Detock [1] address this behavior by implementing home-movement functionality. If the system detects that most transactions touching a partition originate from a remote region, it relocates the home of that partition to the originating region. Thus, systems transform FSH transactions into LSH transactions. However, if a particular partition is frequently accessed by FSH transactions originating from multiple regions, relocating its home proves ineffective. For various reasons, including security, network disruptions, and costs, changing the location of data storage can incur additional overhead.
However, even with reduced cross-region traffic, ensuring atomicity and consistency remains challenging for MP read-write transactions: the system must either commit or abort. Within a single region, coordination is relatively inexpensive, often below 1ms. In a geo-distributed setting, however, inter-region latencies can be up to three orders of magnitude higher, making coordination a performance bottleneck.
3 Geo-distribution in Existing Benchmarks↩︎
In this section, we discuss the suitability of YCSB [23] and TPC-C [22] for evaluating different transactional access patterns in geo-distributed settings. At a high level, the suitability of a benchmark for evaluating geo-distributed transactions depends on the shape of their schemas, the distribution of values in their tables, and the locations of the records that transactions touch during execution.
YCSB. Due to its simplicity, the Yahoo! Cloud Serving Benchmark (YCSB) [23], can be adapted for MP and MH transactions. It consists of a single table with a primary key and a varying number of columns, with each column containing 100 random bytes by default. The main benefit of YCSB lies in its flexibility. Additional parameters, such as skew, MH, and MP percentages, are already present in prior work [1], [3], making it applicable to testing database systems under varying conditions. In Gaia, we have included tuning knobs to set the percentages for LSH, FSH, and MH transactions.
TPC-C and Geo-distribution. The TPC-C benchmark [22] simulates a realistic OLTP workload that resembles an online ordering and
delivery system. The benchmark includes five transactions (NewOrder, Payment, OrderStatus, Delivery, StockLevel) over nine tables. The most interesting transactions are NewOrder
and Payment because they can access data from remote regions, thereby inducing cross-regional coordination. Unlike YCSB, the default TPC-C benchmark specifications predefine a fixed transaction type composition.
The conventional way data is spread across regions to create MH transactions is to place warehouses, and their contained products in separate regions [1]–[6], [26], [27]. Typically, a NewOrder transaction involves an order of 10 items. Each item has a
99% probability of being sourced from a warehouse in the same region as the client and a 1% chance to be delivered from a remote warehouse, making the transaction cross-regional. Similarly, each Payment transaction defaults to a 1% chance of
involving a remote warehouse. Hence, production workloads with different access patterns, e.g., more MH transactions, cannot be easily modeled by TPC-C. To verify this hypothesis, we analyze the default TPC-C specification in practice and assess how its
design choices influence the transactional access patterns. We generate two million TPC-C transactions and report their composition in Table 3.
The breakdown reveals a striking imbalance: over 95% of transactions are LSH, around 4% are MH, and \(\leq0.5\)% are FSH. Interestingly, all observed FSH transactions correspond to Payment operations that
update a single warehouse issued from a remote warehouse. In theory, NewOrder transactions could also be FSH if all ten ordered items came from the same remote warehouse. However, following TPC-C’s default configuration, this scenario is
effectively impossible. In our setup consisting of eight regions (of which seven will be remote regions), the probability of generating such a transaction is \(7 \times (0.01/7)^{10} = 2.48 \times 10^{-28}\), meaning none
appear among our two million samples. In contrast, real-world workloads, such as those of a global retailer, naturally exhibit such access patterns, as customers frequently place orders for items stored in remote regions. This highlights a significant gap
between the synthetic behavior generated by TPC-C and that of realistic geo-distributed transaction workloads.
Discussion. In short, current evaluations only make the binary distinction between single-home and multi-home transactions [1], [3], [5], or do not distinguish between them at all [2], [4], [14]–[16]. In this section, we argued that transactions should rather be split into three categories (LSH, FSH, MH). We discussed how current benchmarks should be altered to enable different scenarios (e.g., MH-heavy workloads vs. LSH-heavy workloads) for evaluating geo-distributed databases.
| Transaction Type | New Order | Payment | Delivery | Order Status | Stock Level | Total | |
| LSH | 39.79% | 43.58% | 3.99% | 4.00% | 4.00% | 95.36% | |
| FSH | 0% | 0.44% | 0% | 0% | 0% | 0.44% | |
| MH | 4.21% | 0% | 0% | 0% | 0% | 4.21% | |
| Total | 44.00% | 44.02% | 3.99% | 4.00% | 4.00% | 100.00% | |
4 Evaluation Practices in the Wild↩︎
Modern geo-distributed databases expose rich instrumentation, including transaction metrics, network statistics, and resource usage. Yet, existing evaluations typically focus just on latency and throughput, providing only a partial view. In this section, we analyze five representative systems, derive a taxonomy of their design choices, review their evaluation practices, and highlight limitations we resolve with Gaia.
System Selection. To analyze core geo-distribution protocols under controlled conditions, we focus on five state-of-the-art open-source systems: Calvin [4], SLOG [3], Detock [1], Janus [15], and CockroachDB (CRDB) [6]. Our selection is guided by the following criteria:
Controlled comparison: Calvin, SLOG, Detock, and Janus share a common codebase [1], enabling an apples-to-apples comparison of OLTP protocols.
Protocol diversity: The systems span a range of approaches: deterministic ordering (Calvin), locality-aware execution (SLOG), decentralized deadlock resolution (Detock), consensus-integrated concurrency control (Janus), and optimistic concurrency control (CRDB).
Research vs.production: We include CRDB to demonstrate how Gaia applies to production-grade systems with strong fault-tolerance guarantees. Due to confounding factors such as proprietary query optimizers, network stacks, and security features, we focus on trends rather than direct performance comparisons.
Reproducibility: All systems are open-source and deployable in controlled environments, enabling consistent evaluation across scenarios.
Integrating Systems into Gaia. Gaia is designed to be extensible. Workload generation and metric collection are decoupled from the system under test, requiring only a thin integration layer. Specifically, a system must expose executed transaction characteristics such that Gaia can classify their type (SP/MP and LSH/FSH/MH). This design enables rapid integration of new protocols without modifying the core framework.
We focus on open-source systems to enable full control over deployment, instrumentation, and fault injection. Evaluating managed services such as Spanner [12] or Aurora [13] introduces practical limitations (e.g., restricted observability and lack of fault injection).
4.1 Properties of Geo-Distribution↩︎
From the chosen systems, we derive all key properties summarized in Table 4. This taxonomy captures the systems’ design decisions and their corresponding targeted scenarios. Replication is the most critical axis, addressing when, how, and where data is replicated. (i) Mode: Replication can be synchronous or asynchronous (some systems offer both). (ii) Scope: Data can be replicated fully or partially in remote regions. More specifically, although only Calvin is designed for full replication, most systems have been designed for partial replication (data only replicated to some or no remote regions). Finally, (iii) Placement: data may be geo-replicated if at least one secondary copy is placed in a remote region.
The replication strategy directly affects fault tolerance and performance. Fully synchronous geo-replication maximizes resilience but incurs latency, as all replicas must acknowledge each transaction. Conversely, partial asynchronous replication can yield higher throughput. Partitioning and geo-distribution are also common design choices between improving scalability and reducing latency by placing data closer to user demand.
Some systems further support dynamic home movement (also known as remastering) [1], [3], [27], [28], which adapts to changing access patterns by relocating data closer to the source of demand. This mechanism optimizes latency over time by converting FSH and MH transactions into LSH, thereby reducing the frequency of cross-regional operations. Lastly, CRDB, as the only system in 4, implements a fault-tolerance mechanism that allows healthy parts of the cluster to continue operating even in the presence of a failure. While it trades off performance relative to other systems, CRDB offers strong resilience for users. No data will be lost, albeit at the expense of additional synchronous replication and durability mechanisms on the critical path.
| Replication | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| 2-6 | Async | Sync | Partial | Full | Geo | Parti- tioned | Geo- distr. | Dyn. home movement | Fault Tolerance |
| Calvin | |||||||||
| SLOG | |||||||||
| Detock | |||||||||
| Janus | |||||||||
| CRDB | |||||||||
4.2 Existing Evaluations↩︎
We now survey the five selected database systems and analyze how their evaluations were conducted.
Calvin [4] is an early deterministic OLTP system designed to support MP transactions at scale. Its core design centers on a deterministic transaction ordering layer implemented via a centralized sequencer, thereby avoiding commit-time coordination logic. Calvin does not incorporate the concept of transaction homes and ensures strong consistency through full replication. As such, its performance remains unaffected by changes in the prevalence of FSH and MH transactions.
Focus: Calvin’s evaluation focus is on scalability. It solely makes use of NewOrder transaction from TPC-C and compares exclusively against a System R\(^*\)-style [29] baseline.
SLOG [3] reduces LSH transaction latency by decoupling their execution from global ordering; only MH transactions are globally ordered. This design reduces coordination overhead and improves latency in workloads dominated by LSH transactions. Experiments demonstrate that SLOG maintains throughput comparable to Calvin while significantly lowering latency for non-MH transactions.
Focus: The original evaluation includes both synthetic YCSB workloads and NewOrder transactions from TPC-C. It compares against Calvin, Spanner, and a two-phase locking (2PL) baseline. Overall, the evaluation emphasizes scalability
and latency improvements across varying LSH and MH ratios, and examines the impact of data locality and coordination costs.
Detock [1] eliminates global sequencing by relying on a distributed, graph-based deadlock-resolution protocol that runs independently on each server. By avoiding global ordering and minimizing aborts, Detock improves the throughput and latency of MH transactions, particularly under contentious workloads. Caerus [5] follows a very similar design.
Focus: The evaluation exercises synthetic YCSB workloads and a complete TPC-C benchmark, comparing to Calvin, SLOG, Janus, and normalized CRDB. The study emphasizes the scalability, abort-free behavior, and performance of MH transactions under asymmetric network conditions.
| VM Instance Type | vCPUs | Memory (GiB) | Network Performance (GiB/s) | Cost ($/h) in us-west-1 | |
|---|---|---|---|---|---|
| m5.2xlarge | 8 | 32 | \(\le10\) GiB | $ 0.448 | |
| r5.2xlarge | 8 | 64 | \(\le10\) GiB | $ 0.560 | |
| m5.4xlarge | 16 | 64 | \(\le10\) GiB | $ 0.896 | |
| r5.4xlarge | 16 | 128 | \(\le10\) GiB | $ 1.120 | |
| m6i.8xlarge | 32 | 128 | \(\le12.5\) GiB | $ 1.792 |
Janus [15] unifies concurrency control and consensus to avoid cycles in the serialization graph. Leveraging their shared requirements allows most transactions to commit in a single round trip, improving performance under high contention.
Focus: The evaluation includes YCSB-style microbenchmarks and the full TPC-C workload, and compares Janus against 2PL, optimistic concurrency control (OCC), and TAPIR. The study emphasizes scalability and performance across varying contention and workload skew, highlighting Janus’s ability to commit transactions in high-conflict settings efficiently.
CRDB [6] was designed to offer high resilience, availability, and strong consistency for geo-distributed applications. To provide fault tolerance, CRDB synchronously replicates data across different geographical regions at the expense of extending the critical path. CRDB can thus continue to operate in the remaining regions, even in the event of a complete regional failure. Unlike deterministic protocols, CRDB aborts transactions in the event of conflicts, but it does not require knowing the read & write set a priori.
Focus: The authors compare CRDB against other industry products, Amazon Aurora, and Spanner using YCSB and TPC-C, concentrating primarily on scalability and fault tolerance.
Discussion. Overall, we find that existing evaluations overlook five main aspects: \(i)\) monetary costs are not considered at all, \(ii)\) systems are only evaluated on a single type of cloud VM, \(iii)\) the evaluations are restricted to a narrow set of benchmarking workloads and scenarios, \(iv)\) FSH transactions have not been considered, and \(v)\) the effect of diverse network conditions is investigated insufficiently. The remainder of this paper addresses these aspects.
5 Towards Holistic Evaluation Scenarios↩︎
In this section, we devise three deployment scenarios: varying input throughput, resource allocation, and server geo-distribution, two benchmark-specific scenarios with respect to: access patterns and access skew, and finally three resilience scenarios: network latency & jitter, packet loss, and fault tolerance. We also justify why these scenarios are essential for evaluating geo-distributed databases.
5.1 Deployment-specific Scenarios↩︎
Varying Input Throughput. As a starting point to link with other related work, we develop a scenario to measure the number of requests that a system can handle without the latency exceeding a certain threshold. Consider an online retailer with a varying number of customers throughout the day. Gaia progressively increases the number of clients generating transactions at a fixed (per-client) rate to measure how well the systems can sustain the input rate and the associated latency.
| Server Geo-distribution (%) | |||||
|---|---|---|---|---|---|
| 2-5 Configuration | US West | US East | EU West | AP NorthEast | |
| uniform | 25% | 25% | 25% | 25% | |
| usw+ | 37.5% | 25% | 25% | 12.5% | |
| usw+ eu+ | 37.5% | 12.5% | 37.5% | 12.5% | |
| usw++ | 62.5% | 12.5% | 12.5% | 12.5% | |
| usw++ eu++ | 50% | 0% | 50% | 0% | |
Resource Allocation. Practitioners want to optimize the needed hardware resources and costs associated with achieving the desired performance. Four key resources drive the performance of a geo-distributed OLTP database: CPU, DRAM, Disk I/O, and Network bandwidth. The growing number of hardware offerings from major cloud providers makes it challenging for practitioners to find the cheapest setup that still meets their SLO. E.g., AWS currently offers over 850 EC2 instance types [30]. Yet, different concurrency mechanisms yield vastly different hardware requirements and the relationships with system performance are non-linear. Gaia deploys systems on different cloud hardware, under identical benchmark configurations, to measure the cost-effectiveness of each system on cheaper or higher-end hardware. In this work, we showcase the systems’ abilities on the subset of EC2 instance types listed in 5, providing a variety of different CPU, DRAM, and network bandwidth mixes.
Server Geo-distribution. Existing evaluations typically split data and transaction workloads equally across all regions. However, in practice, certain regions often handle more requests than others. To test the influence of different server geo-distribution scenarios, Gaia assigns servers, along with their associated data and transactions, to one or two dominant regions, to observe the effect on performance. 6 summarizes the server distributions tested.
5.2 Benchmark-specific Scenarios↩︎
Access Patterns. While TPC-C and YCSB are relatively static, production workloads may require different proportions of LSH:FSH:MH transactions. For the example of an online retailer, the access pattern should resemble customers purchasing more products from different parts of the world. As the FSH and MH percentages increase, more transactions will need to be coordinated to maintain consistency, triggering more conflicts. With more frequent conflicts, more transactions will need to be aborted, reordered, or repeated, decreasing throughput and increasing latency.
Access Skew. In practice, the partitions accessed by transactions are rarely distributed uniformly. Workloads can exhibit a small set of frequently accessed records, increasing contention and potential conflicts. Moreover, these access patterns may evolve over time, e.g., in a diurnal cycle [31]. For practitioners, it is paramount that databases remain unaffected by such behavior. Previous work proposed various skew implementations. E.g., Calvin and SLOG use a contention index, "HOT", to dictate the hot record set size [3], [4]. The evaluation of Detock instead defines "HOT" as the hot record set size reciprocal [1]. Other works instead declare a Zipfian coefficient \(\theta\) between 0.0 (uniform distribution) and 1.0 (extreme skew) [2], [11], [14], [15], [21], [32]. In Gaia we opt for the more widely used Zipfian coefficient option, as it does not require prior knowledge of the maximum possible hot record set size.
5.3 Resilience Scenarios↩︎
Network Latency & Jitter. Most database evaluations assume ideal and static network conditions. Yet, cloud and on-premise production environments exhibit significant fluctuation, leading to increased transaction conflicts, timeouts, and performance bottlenecks. Empirical studies reveal that over two-thirds of internet paths experience \(>\)100ms latency, especially under load [33]. The p50 latencies between on the slowest connections on AWS reach up to 600ms [34]. Geo-distributed deployments further amplify this with heavy-tailed round-trip time distributions [35] and high jitter [36]. Gaia’s network scenario induces configurable fixed propagation delays to simulate bandwidth overload and randomized jitter to mimic network instability and transient queueing effects. A practitioner can also use this scenario to investigate the system performance when servers are placed in regions locations closer together or wider apart. By setting the cross regional latency to 0ms, Gaia can also cover a potential single-region deployment.
Packet Loss. In real-world networks, congestion or external interference makes packets not only slower, but sometimes also lost entirely. Cloud providers typically maintain packet loss under 0.1%, yet under high load, or on some on-premise setups, users can experience significantly higher rates of 1–2% [37], [38]. Even modest packet-loss rates can degrade performance, as transactions are repeatedly blocked or aborted when critical coordination messages are lost. In the packet-loss scenario, Gaia randomly drops a user-specified percentage of outgoing packets at each server.
Fault Tolerance. Cloud providers such as AWS, GCP, and Azure offer very high availability and low mean time to recovery. Nonetheless, critical business applications cannot compromise on availability even in the event of short, transient hardware failures. Hence, a resilient OLTP database must not only recover from a failure successfully without corrupting the data, but also ideally allow (most) users to continue using the database during the failure [14], [16]. Gaia evaluates a database’s fault tolerance scheme by running two failure traces: i) the failure of a single server in the cluster, and ii) the failure of all database servers in a given region. Gaia will then analyze the throughput before, during, and after the failure in the affected region, as well as in the cluster as a whole.
6 Metrics↩︎
To assess database systems in both performance and efficiency, we devise five key performance metrics in Gaia: throughput, latency, aborts, data transfers, and cost. Throughput, and latency are well-established performance metrics. Yet, in production environments, practitioners must not only meet their SLOs but also minimize hardware costs. To assess the efficiency of the different systems, we additionally include data transfer volume and cost per transaction in our evaluation.
Throughput. We measure the system-under-test’s output throughput. Gaia submits pregenerated transactions at a constant rate adjusted to meet the needs of our experiments. Clients work asynchronously; they do not wait for a response to the previous request before submitting a new transaction.
Latency. Latency represents the time interval between a client submitting a transaction to the server and receiving a response. In contrast to prior work, Gaia distinguishes latency by transaction type (LSH, FSH, MH) for more fine-grained analysis.
Aborts. Some protocols may abort or restart a transaction, e.g., when a deadlock is found, or when it contains foreign key lookups on data changed in the meantime, like in PPS [2].
Data Transfers. For data transfers, we sum up all egress traffic from all database servers. We choose egress rather than ingress traffic because egress is specifically what major cloud providers (AWS, GCP, Azure) charge their customers for. Data transfers within the same cloud region are typically free of charge, while data transfers to another region are billed by data volume [30], [39], [40]. In the context of geo-distributed OLTP databases, a cost-efficient system must rely on light-weight internode communication algorithms. To measure data transfers between regions, Gaia uses an adjusted version of iftop [41], a lightweight packet-monitoring tool that uses packet sniffing to determine the amount of outgoing traffic.
Cost per Transaction. To calculate the monetary cost per transaction \(C\), Gaia models the three cost components of geo-distributed deployments: \(i)\) VM rental cost \(c_s\), \(ii)\) inter-region data transfer cost \(c_b\) (intra-region communication is typically free of charge), and \(iii)\) durable storage cost for data backups \(c_d\). These components reflect the standard billing models of major cloud providers (AWS, GCP, Azure) [42]–[45]. We assume the systems are deployed non-stop using on-demand VMs, thus the cost is independent of the transaction runtime. We use the following formula:
\[C = \frac{n_s \cdot c_s + n_b \cdot c_b + n_d \cdot c_d}{t}\]
where \(n_s\) is the number of server nodes, \(n_b\) the volume of inter-region data transferred, and \(n_d\) the amount of durable storage used. Finally, \(t\) denotes the system’s output throughput.
Resource Utilization. Finally, we also report back to the user the CPU, RAM, network, and disk I/O utilization. While not directly affecting performance or cost, resource utilization metrics help practitioners identify over- or under-provisioned resources, guide appropriate hardware selection, or reduce power consumption costs in an on-premise setup.
Figure 4: The YCSB and TPC-C throughput, latency, data transfers, and cost per transaction with increasing input throughput. Opaque lines in the latency figure represent p50 and bold ones p99..
7 Gaia in Action↩︎
We make use of Gaia to evaluate geo-distributed database systems across a range of scenarios. Using the systems introduced in 4, we apply Gaia to YCSB and TPC-C workloads to illustrate the types of insights it enables. Specifically, we use Gaia to study:
how systems behave under varying workload characteristics (e.g., throughput, access skew, and access patterns)
how efficiently systems utilize resources and the resulting cost implications
how systems respond to network instabilities and failures
Rather than direct system comparisons, our goal is analyzing how different system design choices (4) respond to Gaia’s evaluation dimensions. The results are not intended as definitive system comparisons, but as illustrative case studies of how Gaia exposes design trade-offs.
7.1 Methodology↩︎
Cluster Setup. Experiments run on r5.4xlarge VMs in AWS (16 vCPUs, 128 GB DRAM, \(\leq\)10 Gb/s bandwidth, $1.008–1.216/h depending on the region [us-west-1/2,
us-east-1/2, eu-west-1/2, ap-northeast-1/2]) [30]. Inter-region data transfers cost ¢1–9/GB [30]. Each region hosts four database servers (as in Detock [1]) and one client VM that generates transactions.
Network Profile. Measured RTTs vary significantly across regions (3), particularly for Asia-bound traffic. AWS routing further amplifies this effect: us-west\(\leftrightarrow\)eu-west is 1.7\(\times\) faster than eu-west\(\leftrightarrow\)ap-northeast, despite comparable
distance [46]. Some scenarios require controlled network impairments, Gaia thus utilizes NetEm [47]. All VMs are then co-located in a single region, using NetEm to inject delays, jitter, or packet loss based on the measured baselines.
| Workload Family | Access Pattern | Skew | |
|---|---|---|---|
| YCSB Default | LSH=50%; FSH=25%; MH=25% | \(\theta=0.0\) | |
| YCSB Access Patterns | LSH=0-100%; FSH=0-50%; MH=0-50% | \(\theta=0.0\) | |
| YCSB Access Skew | LSH=50%; FSH=25%; MH=25% | \(\theta=0.0-1.0\) | |
| TPC-C Default | LSH=95%; FSH=0.4%; MH=4.2% | \(\theta=0.0\) | |
| TPC-C Access Patterns | LSH=12-100%; FSH=0-44%; MH=0-44% | \(\theta=0.0\) | |
| TPC-C Access Skew | LSH=95%; FSH=0.4%; MH=4.2% | \(\theta=0.0-1.0\) |
Experiment Setup. Gaia supports combining multiple evaluation dimensions, but full grid search would be excessive. Instead, we adopt an ablation-style methodology to isolate the impact of each factor. 7 summarizes the workload families used. Unless stated otherwise, we use uniform skew. For YCSB, we set FSH+MH=50% and MP=50%; by default, touching 2 hot and 8 cold keys per transaction. TPC-C is configured with 1200 warehouses and a 1% probability of a remote item order or payment.
All systems store data in memory. Calvin and Janus replicate data across all regions; SLOG and Detock store data in a primary region (their most performant system variant); and CRDB uses multi-region replication for fault tolerance. We plot the output throughput, p50 (opaque) and p99 (bold) latency, data transfers, and cost per transaction. For CRDB, we use a separate y-axis as its performance is not comparable to the other systems. For the network latency and packet loss scenarios, we normalize results relative to default conditions. For space reasons, abort rates, resource utilization, and latency ablation are only discussed in the text, with additional plots available in our GitHub repository [25].
7.2 Varying Input Throughput↩︎
Figure 5: The YCSB and TPC-C throughput, p50 (bold bars) and p99 (opaque bars) latency, data transfers, and VM+storage cost (bold bars) and data transfer costs (opaque bars) per transaction across five AWS VM instance types..
We begin by using Gaia to identify the sustainable operating throughput of each system. To this end, we vary the target input throughput from 0 to 200k (YCSB) and 60k (TPC-C) transactions/s (4).
Finding the Maximum Throughput. As input throughput increases, all systems exhibit a characteristic saturation point beyond which latency grows rapidly. The location of this point varies per system and workload. For example, SLOG and Detock sustain higher YCSB input throughput (\(\sim\)100k) before their p99 latency explodes above 10s. Calvin and Janus reach their saturation points earlier in YCSB, with Janus scaling faster, but plateauing sooner. In TPC-C, memory pressure becomes the limiting factor at higher loads. Since TPC-C transactions are generally more complex than in YCSB, considerably more RAM is needed to store their in-flight data.
Positioning CRDB in the Evaluation. CRDB originates from a separate codebase, and operates in a different resilience regime than the academic prototypes. Among the evaluated systems, only CRDB is equipped with production-grade
multi-region fault tolerance. In contrast, the academic research prototypes evaluated here lack support for comparable fault-tolerance guarantees. This design choice explains why CRDB achieves lower peak throughput but provides stronger resilience. CRDB’s
observed peak throughput (4k for YCSB and 500 for TPC-C), is consistent with the tpmC (NewOrder transactions per minute) results in prior reports [6], [20], accounting for cluster size, replication, row count, and geographical node placement. Closest to our setup is an
experiment in [20], with TPC-C run over 10 regions, yielding \(\sim\)12k tpmC,
corresponding to \(\sim\)450 txns/s. For the remaining scenarios, we thus use the peak sustainable input throughput for each system and workload. Instead of absolute-number comparisons across systems, we focus on relative
performance trends.
Figure 6: The YCSB and TPC-C throughput, p50 (bold bars) and p99 (opaque bars) latency, data transfers, and VM+storage cost (bold bars) and data transfer costs (opaque bars) per transaction across five different server skew configurations..
Data Transfers. Throughput strongly correlates with data transfers. This is because increased transactional throughput leads to more communication demand for coordination, conflict resolution, and replication. Although the YSCB and TPC-C transactions themselves are relatively small, Gaia measures the entire system traffic, which surprisingly is on the order of GB/s. 4 also shows that Calvin’s and Janus’ concurrency protocols generally require less communication per transaction than SLOG and Detock.
Cost. The monetary costs per transaction are directly related to the throughput and data transfers. VM rental and durable storage have a fixed cost of around 1¢. The higher the throughput, the lower the fixed cost per transaction. When the systems reach their stabilized maximum throughput, the total cost per transaction ranges from 1.5¢ per 10k transactions (Janus and Calvin) to 2.5¢ (SLOG, Detock, and CRDB) in YCSB. Since data transfers make up the majority of costs, and with SLOG and Detock requiring almost double the data transfers, their cost per transaction is higher. Interestingly, in TPC-C, Detock and SLOG require only 25% more data transfers; the total cost for the first four systems converges to around 1.6¢ per 10k transactions. CRDB, with a much lower throughput, costs around 20¢ per 10k transactions. Note, however, that CRDB could likely achieve similar performance on weaker machines with a lower VM rental price.
Takeaways: SLOG and Detock scale the best in terms of throughput and latency. Janus’ performance is almost on par, but remains the least expensive, making a great tradeoff between performance and running costs. CRDB’s performance is comparatively lower, due to its hardened fault-tolerance and its support for interactive transactions.
Figure 7: The YCSB and TPC-C throughput, latency, data transfers, and cost per transaction with varying geo-distribution percentages. Opaque lines in the latency figure represent p50 and bold ones p99..
7.3 Resource Allocation↩︎
Next, we use Gaia to examine how system behavior changes across different hardware configurations (5) under the default YCSB and TPC-C workloads. We report monetary cost by separating fixed VM & storage cost (bottom component) from variable data transfer cost (upper component).
VM Price, Performance, and Cost. In 5, we order machine types by available CPU and RAM from weakest to strongest. We omit the m5.2x machine type for TPC-C, as it has
insufficient memory and crashes. Across systems, increasing hardware capacity generally improves throughput, although not strictly monotonically (e.g., m5.4x occasionally outperforms the more powerful r5.4x).
A key observation enabled by Gaia is that more powerful machines reduce the cost per transaction despite higher hourly prices. The higher attainable throughput amortizes fixed infrastructure costs and shortens the critical path of transactions. This effect is consistent across systems, although varying in magnitude. The cost breakdown further reveals different bottlenecks: for Calvin, SLOG, Detock, and Janus, data transfer dominates total cost, while CRDB spends a larger fraction on compute resources. Correspondingly, more powerful machines generally reduce latency through improved parallelism. The impact is most pronounced for SLOG and Detock in YCSB, while in TPC-C, Calvin and Janus benefit the most.
For the remaining experiments, we use r5.4x instances, consistent with prior work [1], allowing for direct comparisons.
Takeaways: In short, increasing hardware capacity can reduce cost per transaction by improving throughput and amortizing fixed costs. Gaia’s resource allocation scenario highlights that, for most systems, inter-region data transfer dominates overall cost, rather than compute and storage.
7.4 Server Geo-distribution↩︎
To examine the impact of the geographical server distribution, Gaia evaluates the systems’ performance in five different server distribution settings in 6. Across systems, we observe a negligible impact on throughput, latency, and data transfers. This behavior is expected since the access patterns of the workloads remain unchanged. Despite clustering data in hotspots, the LSH:FSH:MH ratio remains the same.
However, there is a clear impact on cost. In highly skewed geo-distribution (usw++eu++), more data transfers become intra-regional, not billed by cloud providers. Thanks to the reduced inter-region communication, the cost per transaction
for usw++eu++ is up to 2x cheaper than in uniform.
Figure 8: The relative YCSB and TPC-C throughput, latency, data transfers, and cost per transaction under extra network latency and jitter. Opaque lines in the latency figure represent p50 and bold ones p99. The dashed vertical line represents the average cross-region latency in the default geo-distribution setup..
7.5 Access Patterns↩︎
While YCSB natively supports varying the transaction access patterns, TPC-C defines a fixed LSH:FSH:MH transaction ratio. To provide such a test on TPC-C, Gaia varies the probability of targeting a remote warehouse in a NewOrder or
Payment transaction. In this manner, our TPC-C Access Patterns workload controls the geo-distribution percentage from 0% to 88%, making the workload increasingly harder for the systems. For YCSB, a geo-distribution of 100% means
having 0% LSH, 50% FSH, and 50% MH transactions. In the case of TPC-C, the maximum geo-distribution percentage of 88% is decomposed into 44% FSH and 44% MH transactions; the remaining 12% of transactions can only be LSH by design.
7 illustrates the diverse effects of increasing geo-distribution depending on system design. SLOG and Detock experience a steady decline in throughput as the cross-region share increases, accompanied by a 40% drop in CPU utilization. In contrast, Calvin, Janus, and CRDB remain unaffected, because they treat LSH, FSH, and MH transactions equally.
Latency. These differences are further reflected in latency. Calvin, Janus, and CRDB maintain relatively stable latency across geo-distribution levels, albeit at consistently higher levels. In contrast, SLOG and Detock show a significant increase in p50 latency as the workload becomes more geo-distributed, particularly for TPC-C. Their initial advantage stems from optimized handling of LSH transactions, but this benefit diminishes at 100% geo-distribution, once most transactions require cross-region coordination. As a consequence, Janus and Calvin also prove to be more cost-efficient as the workloads become more geo-distributed.
Latency Ablations. For a deeper analysis we isolate the latency of read vs. write and LSH vs. FSH vs. MH transactions. Again, Calvin and Janus do not differentiate between these transaction types, and their latency always remains high. However, Gaia reveals that SLOG and Detock are an order of magnitude more efficient for read and LSH transactions. Interestingly, the latency of FSH transactions grows from 20ms up to 250ms as geo-distribution increases and conflicts with MH transactions become more frequent. CRDB also offers faster response times for reads than writes. However, LSH transactions remain slow because they still need to be replicated synchronously to a remote replica.
Takeaways: SLOG and Detock optimize for LSH transactions, yet at higher geo-distribution percentages, their performance deteriorates and costs grow. Similarly, a highly fault-tolerant system like CRDB can perform well for local read transactions, but persisting writes to foreign-region replicas takes a toll on its performance. This entails that hybrid solutions are required. Janus strikes a reasonable performance-cost balance.
Figure 9: The relative YCSB and TPC-C throughput, latency, data transfers, and cost per transaction under extra packet loss. Opaque lines in the latency figure represent p50 and bold ones p99..
7.6 Access Skew↩︎
In the YCSB workload, Gaia defines hot and cold keys in the table. The workload’s built-in \(\theta\) parameter makes it suitable for evaluating access skew scenarios. A skew factor close to 0 results in a uniform
distribution of all keys, whereas values close to 1 concentrate transactions on a handful of hot keys. Our TPC-C access skew, models skew by limiting the pool of items and customers involved in the NewOrder
and Payment transactions. The smaller the pool of items and customers, the more contention in TPC-C. Our results are consistent with the latest literature [1], i.e., access skew does not affect the throughput and latency of deterministic databases considerably for our selected MP configuration. The only system that exhibits a notable impact is
CRDB, where extremely high contention rates trigger frequent aborts, leaving very little useful work. For brevity, we omit the plots.
7.7 Network Latency & Jitter↩︎
To test the effect of diverse network conditions, Gaia varies the average cross-region latency from 0ms (single-region deployment) to over 600ms (worst-case round trip time observed in AWS [34]). We also induce jitter of 10% of that average latency. This allows Gaia to identify breaking points, i.e., at what latencies does system behavior shift sharply. 8 depicts the relative performance compared to an average latency 111ms, our default geo-distributed setting. This scenario is workload-agnostic and can be applied to YCSB, TPC-C, or custom benchmarks.
Increasing latency exposes clear differences in how systems handle coordination. Janus exhibits almost no performance effect moving from 0ms to 111ms average cross region delay, and afterwards only degrades linearly. Even during the highest slowdowns, Janus sustains a meaningful amount of throughput. In comparison, SLOG and Detock degrade much more rapidly, reaching their breaking points at just \(\sim\)150ms average delay in TPC-C. The systems lose 90% of their throughput and increase their latency tenfold. Similar trends are observed in YCSB with all systems except Janus quickly dropping in throughput. We attribute the robustness of Janus to high latency to the fact that it can commit non-conflicting transactions in a single round-trip. As a consequence, the high latency affects the commit time of non-conflicting transactions less than in other systems. Gaia also uncovers which systems thrive in single region deployments. In particular, CRDB is able to double its YCSB throughput compared to our default setup.
Cost Implications. At high latencies, cost per transaction explodes for most systems. This is because very little useful work is still being done and the fixed infrastructure costs are amortized over fewer completed transactions. Systems that maintain some meaningful throughput under high latency conditions are therefore able to sustain lower cost per transaction.
Takeaways: Gaia reveals that none of the systems copes with network slowdowns effectively. Out of the five systems, Janus is the most robust and can keep the cost per transaction low, while SLOG and Detock struggle the most.
7.8 Packet Loss↩︎
We further evaluate robustness under unreliable networks by introducing 0–10% packet loss across all connections (9). Evidently, even modest packet loss (e.g., 2%) leads to noticeable 35–60% throughput decline and latency deterioration of Calvin, SLOG, and Detock. Only CRDB and, to some extent, Janus show more resilience to packet loss. As in the latency experiments, data transfer volume decreases with throughput, while cost per transaction increases due to reduced system utilization.
Takeaways: In summary, even modest packet loss can trigger major slowdowns in all systems, although CRDB proves to be the most resilient. With extreme packet loss, latency, and costs explode, even with reduced data transferrs.
7.9 Fault Tolerance↩︎
Finally, as an ultimate test of database resilience, we plot two failure throughput traces (10). We begin by warming up the cluster for \(\sim\)15s to reach a steady throughput of transactions and then kill a single node or an entire region. After another \(\sim\)30s, the node is restarted again. Among the evaluated systems, only CRDB was actually able to sustain such failures and recover correctly. The remaining systems require manual intervention and a restart to resume operation. Furthermore, thanks to the chosen replication policy (1 replica in a foreign region), CRDB can also maintain some level of operation for clients from unaffected regions during the failure.
Single Node Failure. When a single node (out of 32) fails, CRDB’s overall throughput drops by \(\sim\)40%. Clients in the affected region are temporarily unable to submit transactions, although for a few seconds, it may continue to receive responses to previous requests from other nodes. The remaining nodes must adjust to the missing 3rd replica for certain data items. After the node recovery recovery, the system requires another 10s to restore normal throughput and catch up with pending requests during downtime.
Region Failure. In a region failure, some data items only have one replica online, which is insufficient for a majority. Initially, \(\sim\)15s after the failure, clients in other regions can continue to execute transactions touching data in unaffected regions. Eventually, however, all clients block, waiting on unavailable partitions and bring the entire cluster to a standstill. Nonetheless, CRDB can still safely recover once the region comes back online. Its meticulous fault tolerance protocol prevents data loss, and blocked transactions can restart once the entire cluster is back in operation.
Takeaways: A sound fault tolerance mechanism is paramount for resilience. While impeding performance under normal conditions, CRDB’s fault-tolerance protocol ensures continued operation during a single-node failure and prevents data loss in a complete regional outage.
8 Lessons Learned↩︎
Our research yields several interesting insights into system design, benchmarking methodology, and open challenges in geo-distributed OLTP research.
Robustness of Transaction Protocols. Protocol design plays a critical role when systems operate under network instability. Protocols with fewer coordination rounds (e.g., Janus in our testbed) prove more robust. A broader concern is that fault-tolerance mechanisms introduce non-trivial overhead to end-to-end system performance. When evaluating concurrency control and transaction commit protocols in isolation and without fault-tolerance mechanisms in place, their latency and throughput characteristics may appear favorable. However, these advantages can degrade significantly once fault tolerance is introduced. Our experience with CRDB reinforces this point: the throughput of a production grade, fault-tolerant system is substantially lower than that of academic prototypes. We advocate for evaluating new protocols in systems where fault tolerance is a first-class concern, not an afterthought.
Transfer Costs: The New Bottleneck. Designing database systems around cost requires a fundamentally different perspective than optimizing for latency. High WAN latency often leaves RAM and disk resources underutilized, while data transfers dominate the operational bill. This tension calls for protocols that strike a balance between performance and cost efficiency, instead of optimizing for one dimension in isolation.
This observation complements recent findings in OLTP research, which identify communication overhead as the primary performance bottleneck [8]. While prior work emphasizes latency, our results show that in geo-distributed settings, communication also dominates monetary cost.
Multi-Parameter Optimization. Gaia focuses on throughput, latency, data transfers, and cost per transaction. Throughput and latency often exhibit non-linear and system-specific behavior. If multiple systems satisfy an SLO, non-trivial tradeoffs across performance, cost, and availability remain. This makes multi-parameter optimization particularly challenging. Frameworks like Gaia can empower architects to make well-informed decisions, uncovering a potential Pareto frontier.
Towards New Benchmarks. Production geo-distributed workloads may diverge sharply from TPC-C and YCSB [31], [48], [49]. They feature bursty loads, long-tailed access distributions, diurnal cycles, and transaction mixes unlike today’s benchmarks (e.g., FSH transactions are nearly absent in TPC-C). Tunable workload generators [50] move in the right direction. Yet, an in-depth study of transactions included and omitted from benchmarks such as YCSB, TPC-C, SmallBank [24], MovR [51], or PPS [52] is needed. Based on such an analysis, future work could produce a configurable, TPC-C-like benchmark with recently observed traits by major cloud providers, similar to recent OLAP benchmarks [53], [54].
9 Related Work↩︎
4 introduced the studied systems. Here, we highlight related evaluations and surveys, grouped by theme.
Concurrency Control Evaluations. The work most closely aligned with ours is [2], which found that existing protocols scale poorly in distributed, but single-region environments. Our work extends this line of inquiry to an eight-datacenter deployment spanning three continents, where WAN latencies dominate. Yu et al. evaluated seven concurrency control schemes on many-core architectures [55], advocating that database designs must be fundamentally rethought for novel hardware. At a finer granularity, [56] analyzed the instruction-level sources of OLTP latency, and [20] performed a deep dive into CRDB’s performance for geo-distributed databases.
System Configuration and Methodology. Leis et al. and Chang et al. investigated cost-optimal instance selection [57], [58]; however, geo-distributed deployments additionally require accounting for high network delays. Wang et al. examined the reproducibility of recent database evaluations [21], focusing primarily on throughput and a narrow set of scenarios. Finally, Nguyen et al. surveyed the OLTP system assumptions about the applications using them [59].
Benchmarking Beyond TPC-C. To align evaluations with production settings, Poess proposed TPC-DI [60], linking transactional with data integration workflows, e.g., bulk loading and transformation tasks. While valuable for mixed workloads, the adoption of TPC-DI remains limited in the community. Similarly, TPC-E [61], designed as a more sophisticated successor to TPC-C, has yet to attract mainstream adoption.
10 Conclusion↩︎
Our study demonstrates that performance-only evaluations are insufficient for geo-distributed database systems. Network instability, inter-region data transfer, and fault-tolerance mechanisms fundamentally shape both system performance and economic feasibility in real-world deployments. Therefore, we argue that a realistic geo-distributed database evaluation must explicitly incorporate: \(i)\) Local Single-Home / Foreign Single-Home / Multi Home workload composition, \(ii)\) inter-region data transfer volume and cost, \(iii)\) network delay, jitter, and packet loss, and \(iv)\) fault tolerance under node and regional failures. Only by evaluating systems across all these dimensions can we obtain results that reflect practical cloud deployments. We believe Gaia will encourage the community to embrace cost-, network-, and failure-aware benchmarking as a standard practice in geo-distributed database research.