1 Introduction↩︎

Physical memory allocation is a fundamental OS service that has a growing impact on the performance and energy efficiency in modern computing systems [1]–[4]. Each allocation typically involves handling a page fault in software: the processor traps into the operating system, which first assigns a physical frame to establish the virtual-to-physical mapping.

As we demonstrate in §3 and as recent studies show [5]–[8], allocation-related OS activity in modern short-running user-interactive workloads can account for up to 54% of their execution time and for up to 18% of their energy footprint.

To address the limitations of existing purely software- and hardware-based allocation schemes [5]–[7], we propose Valinor, a hardware–OS cooperative memory allocation substrate that combines the flexibility of software-based allocators with the low-latency and energy-efficiency of hardware-based allocation. Valinor introduces a programmable allocation engine (PAE) that runs compact software-defined allocation libraries directly in hardware. The OS loads and configures these libraries using process-level directives, allowing page faults to be resolved without entering the kernel’s slow fault-handling path. Unlike prior hardware allocators that implement a fixed, greedy strategy, Valinor enables the OS to select from a diverse collection of allocation libraries, each tailored to a different optimization goal or hardware environment. This way, Valinor enables various intelligent memory allocation policies while eliminating the costly overheads of the software page fault path, context switching, and pipeline flushes during allocation.

Mechanism Overview. Applications provide coarse-grained (per-process) or fine-grained (per-memory object) directives that allow the OS to associate specific memory regions with an allocation library. The OS then configures the hardware allocation engine accordingly, enabling hardware-level execution of that library’s policy. When a core encounters a page fault, the MMU determines whether the faulting memory object is bound to a library and whether it is a candidate for hardware-based handling. If so, the request is forwarded to the PAE, which invokes the library’s routines. The library itself decides whether it can (or should) allocate the object: on success, it selects a physical page, updates its metadata, and returns the resulting PTE directly to the MMU, completing the allocation entirely in hardware. If the library declines to allocate, Valinor transparently falls back to the standard OS page-fault handler.

At a high level, Valinor provides three key benefits:

(1) Hardware-class latency and energy efficiency. In latency-sensitive environments (e.g., serverless functions), Valinor bypasses the traditional software page fault path and satisfies many allocation requests directly via the hardware allocation engine. This way, Valinor avoids the overheads of context switches and pipeline flushes and executes allocator logic on a lightweight, energy-efficient in-order pipeline, substantially reducing both latency and energy consumption during allocation (see §9).

(2) Rich programmability and library extensibility. Valinor is not a fixed-function allocator. Instead, it exposes a programmable interface that allows the OS to load specialized allocation libraries into hardware. System developers can supply a wide library collection, including fast allocation, energy-efficient allocation, and more (see §7), without modifying the hardware. Libraries can manage their own metadata and define placement heuristics. This flexibility preserves the expressive power of software-based allocators while achieving high performance (see §9).

(3) Adaptivity to microarchitectural state. The PAE resides in the memory controller and maintains direct visibility into low-level signals (e.g., DRAM bank conflicts). This way, allocation libraries incorporate fine-grained hardware feedback to make adaptive placement decisions. This enables optimizations, such as steering allocations away from congested banks, that are infeasible in conventional software-based allocators (see §9).

Key Results. We prototype Valinor in a BOOM RISC-V soft-core running Linux (i.e., on a Xilinx ZCU106 FPGA [9]) to evaluate its feasibility and performance. We also implement Valinor in a full-system simulator (Virtuoso [10]+Sniper [11]) to perform a wide design space exploration for the allocation engine’s architecture (e.g., in-order core vs OoO vs CGRA) and to evaluate multiple allocation libraries.

Our evaluation yields four key results. First, on the RISC-V prototype, Valinor reduces minor page-fault handling time by 17\(\times\) compared to Linux, achieving 16% average end-to-end speedups and 5% average energy savings. Despite using a simple in-order programmable core, Valinor remains within 3% of fixed-function hardware while preserving full flexibility. Second, we evaluate Valinor across six allocation libraries and demonstrate that it can support low-latency, integrity-enforcing, speculative, and adaptive placement policies without any hardware redesign. Notably, the speculation-based library hides over 99% of page-fault latency. Third, Valinor’s telemetry-driven library uses real-time microarchitectural signals to steer allocations away from congested DRAM banks, restoring co-located latency-sensitive performance to within 1–2% of isolated execution while still accelerating allocation-sensitive workloads by 1.25\(\times\), outperforming both software-only and greed-for-speed hardware allocators. Finally, Valinor introduces modest hardware cost: in an 8-core BOOM CPU design, the PAE adds only \(<\)​1.5% area and 1.5–1.8% static power. These costs are small relative to the substantial reductions in page-fault latency, energy, and interference that Valinor provides. In this work, we make the following contributions:

• We perform an extensive characterization of the Linux page-fault path, quantifying its latency, energy cost, and bottlenecks across emerging workloads. This analysis motivates the need for hardware-assisted allocation.

• We propose Valinor, a hardware-OS cooperative memory allocation scheme that combines the flexibility of software-based with the efficiency of hardware-based allocation. Valinor introduces a PAE that allows the CPU to offload allocation requests directly to hardware, eliminating costly kernel traps and context switches while supporting diverse allocation policies with different optimization goals.

• We demonstrate a broad set of allocation policies enabled by Valinor’s programmable hardware–software interface, including low-latency allocation for short-lived objects, adaptive placement guided by microarchitectural telemetry (e.g., DRAM bank pressure), and integrity-checking of virtual-to-physical mappings.

• We implement Valinor in both a Linux prototype on a BOOM RISC-V soft-core and a full-system simulator (Virtuoso+Sniper). Across short-lived and placement-sensitive workloads, Valinor achieves 17\(\times\) reduction in page-fault latency, delivers 16% end-to-end speedups on average, and provides up to 8% energy savings, while adding less than 1.5% area to a 8-core BOOM processor. We will open-source our infrastructure to enable further use in the community.

2 Background↩︎

Modern processors implement address translation to bridge the gap between the virtual address space exposed to software and the physical address space used by hardware. Each memory reference issued by the CPU must be translated from a Virtual Address (VA) to a Physical Address (PA) before it can access memory. Address translation is performed by looking up per-process page tables managed by the operating system. Page tables define the mapping between virtual pages and physical frames at a fixed granularity (typically 4 KB). Each entry, called a Page Table Entry (PTE), stores the physical frame number and a set of metadata bits such as the Present bit, access permissions (read, write, execute), and state bits (accessed and dirty). A cleared Present bit indicates that the virtual page is not currently mapped in physical memory. In such cases, any access to that address results in a page fault exception, causing the processor to trap into the kernel’s page fault handler for resolution.

To minimize translation overhead, processors maintain a hardware unit called the Memory Management Unit (MMU) which is responsible for address translation. The MMU employs small hardware caches known as the Translation Lookaside Buffers (TLB) to store recently-accessed PTEs. On every memory access, the MMU first queries the TLB. A TLB hit allows the translation to complete with low latency. A TLB miss, however, forces the hardware to perform a page table walk, during which the MMU traverses the multi-level page table to locate the corresponding PTE. If the PTE is found and marked as present, the MMU updates the TLB with the new translation and resumes execution. If the PTE is not present, a page fault is raised, transferring control to the OS’ page fault handler routine.

2.1 Demand Paging and Page Fault Handling↩︎

The Memory Management Unit (MMU) handles address translation and uses Translation Lookaside Buffers (TLBs) to cache recent PTEs. On each memory access, the MMU checks the TLB. A hit resolves quickly. A miss triggers a page table walk (PTW) through the multi-level page table. If the PTE is found and present, the TLB is updated and execution resumes. Otherwise, a page fault invokes the OS.

Modern operating systems (e.g., Linux) use demand paging to manage memory: pages are brought into physical memory only when first accessed, reducing startup latency and allowing workloads to exceed available DRAM. When a process accesses a virtual address (VA) that is not backed by physical memory, the hardware triggers a page fault, and the kernel resolves it by allocating memory, updating metadata, and resuming execution. Fig. 1 shows the high-level steps for the simplest case: a minor fault on an anonymous page. When the fault occurs, control transfers to the kernel (), which identifies the faulting process and address (). The kernel locates the faulting region by walking the Virtual Memory Area (VMA) tree (). VMA entries describe contiguous regions of a process’s virtual address space (VAS) and their attributes. For a newly touched anonymous region, the kernel allocates a physical frame by querying the buddy allocator [12] () and then creates or updates the page table entries (PTEs) to establish the mapping (, ). It updates the VMA information as needed and propagates the new mapping to CPUs, potentially issuing TLB shootdowns. Finally, the kernel performs bookkeeping such as updating the LRU list () before returning to user space, where the PTW is restarted (). Modern operating systems (e.g., Linux) employ Demand Paging to manage memory efficiently. Rather than loading all pages of a process into memory at startup, pages are brought into physical memory only when they are first accessed. This design reduces startup latency and allows the system to execute workloads that exceed the available physical memory. When a process references a virtual address that does not currently have a valid page table entry, the hardware triggers a page fault. The kernel’s page fault handler is then responsible for resolving the fault by allocating memory, updating the relevant data structures, and resuming execution.

Fig. 1 illustrates the high-level steps involved in handling a page fault in Linux, focusing on the simplest case: a minor fault on an anonymous page. Such a fault occurs when the accessed page does not yet exist in memory but can be immediately satisfied by allocating a new physical page, without requiring disk I/O. When the fault occurs, control is transferred from user space to the kernel (), where the interrupt handler identifies the faulting process and the virtual address that caused the exception (). The kernel then determines the properties of the faulting memory region by walking the Virtual Memory Area (VMA) tree (). Each process maintains a set of VMAs that describe contiguous regions of its virtual address space with uniform attributes such as access permissions and backing source (e.g., anonymous or file-backed). The faulting address is located within this structure to identify the appropriate handling strategy.

In the case of a newly accessed anonymous region, the kernel proceeds to allocate a new physical frame. It first locates a free page by querying the buddy allocator [12] () and retrieving a pre-zeroed page, the kernel then constructs or updates the corresponding page table entries (PTEs) to establish the virtual-to-physical mapping (, ). At the same time, the VMA tree is updated to reflect the new mapping. The updated mapping is propagated to the relevant CPUs, performing TLB shootdowns if necessary (), ensuring memory consistency across cores. Finally, the kernel performs internal bookkeeping, such as updating the least-recently-used (LRU) list for page replacement (), before returning control to user space.

This sequence represents the fundamental fast path of page fault resolution. More complex cases, such as copy-on-write (CoW) or faults requiring disk access, extend this workflow with additional stages but rely on the same underlying mechanisms. Table 1 summarizes the most common fault categories.

3pt

3 Motivation↩︎

3.1 The Shifting Nature of Modern Workloads↩︎

Traditional applications such as scientific simulations [13]–[16], database servers [17]–[21], large-scale graph analytics [22]–[26] and genomics workloads  [27]–[41]are typically long-running and operate over large memory footprints. These workloads hide the overheads of software-based memory allocation by amortizing them over the billions of instructions they execute.

In contrast, the modern computing landscape is increasingly dominated by user-interactive, short-lived workloads. Examples include serverless functions executed in FaaS platforms  [5], [8], [42]–[74], microservices  [75]–[86], and large language model (LLM) inference queries  [87]–[95], perform limited computation per request, exhibit bursty request patterns, are stateless and often terminate within a hundred milliseconds. These workloads allocate and release memory frequently and unpredictably, and they may touch only a small number of pages per request. Because each request’s lifetime is short, there is little opportunity to amortize the cost of page allocation and initialization. As a result, even minor page faults and associated physical page allocation routines can become a first-order component of tail latency and energy consumption.

3.2 Examples: Serverless and Microservices↩︎

In FaaS platforms, a function invocation includes initialization, execution, and keep-alive [71], [72]. If a function cannot stay warm, snapshotting reduces cold starts but restores pages with CoW permissions, so first writes trigger CoW minor faults. Processing a new request that executes a function incurs additional faults by triggering first-touch accesses and allocating dynamic memory. For example, a JSON-processing pipeline repeatedly touches new intermediate pages with little data reuse, generating many minor faults [96]. Microservices show similar behavior. At startup they map large anonymous regions. Per-request handlers then deserialize inputs, build metadata, and emit responses. Each step touches fresh pages [75]–[86]. Although these services are long-lived, the short per-request work cannot amortize the resulting faults.

3.3 Cost of Software-Based Memory Allocation↩︎

To quantify the impact of software-based memory allocation on modern workloads, we perform detailed profiling on a real Intel Xeon server using DeathStarBench[76], vSwarm[97], and Bitnet [98] (see §8). Fig. 2 reports the fraction of total execution time and energy (measured with Intel RAPL [99]) spent in minor page-fault routines across our workloads. We make two observations. First, minor faults consume on average 13% (up to 54%) of execution time and 7% (up to 18%) of total energy. Second, even compute-intensive applications such as LLM inference in Bitnet still spend up to 11% of total time in minor faults.

3.4 Dissecting Minor Page Fault Overheads↩︎

To understand the source of these overheads, we dive deeper into the performance characteristics of minor page faults.

Latency distribution. We develop an eBPF-based tracing tool that records the latency and retired instructions of minor page-fault paths. First, our tool reveals that in these workloads, over 90% of the total minor page faults occur due to anonymous pages. Fig. 3 shows the CDF of fault latency in CPU cycles. The mean is  4.9K cycles. The tail is long. The 90th percentile is above 8.3K cycles, and the 99th percentile is over 20K cycles.

These outliers arise from extra cache misses in the allocator or page tables, or contention on shared structures (e.g., zone locks, LRU lists). Such latencies significantly degrade performance and inflate tail latency for short-lived functions.

Co-located workloads. To quantify contention on kernel-shared structures, we augment our eBPF-based tracing tool to measure how a victim workload’s page fault latency varies depending on co-located aggressors. Fig. 4 reports the page fault latency increase normalized to the victim running in isolation. The victim is pinned on CPU 0. Each aggressor occupies a separate core (e.g., 8 cores = 1 victim + 7 aggressors). We use three aggressor categories. (1) No THP aggressor: allocates 512 MB and touches every 4 KB page once in random order, triggering one fault per page. (2) THP aggressor (no fragmentation): allocates 256 MB and touches memory at a 2 MB stride to obtain huge pages. (3) THP aggressor (fragmentation): allocates 1 GB, touches every 4 KB page, then releases physical memory via MADV_DONTNEED to create a checkerboard-like fragmentation pattern. We then run aggressor (2) to mimic allocation in a highly fragmented system, which triggers memory compaction during fault handling.

We make three key observations. First, no-THP aggressors inflate the victim’s page fault latency minimally (5%, 7%, and 17% for 8, 16, and 24 cores), as they hold shared locks only briefly. Second, THP aggressors without fragmentation increase latency by 51% to 97% (7 to 23 aggressors), because they hold zone->lock for many cycles, serializing the victim’s fault handler. Third, under memory fragmentation, ubiquitous in cloud-native systems [100], the increase reaches 163% to 258%, as compaction on the buddy allocator’s critical path extends zone->lock hold times further.

Instruction count. Fig. 5 presents the CDF of the number of retired instructions per minor page fault, obtained from hardware performance counters. Each fault retires more than 5K instructions on average, with the tail reaching over 11K instructions. Such a high instruction count stresses the out-of-order core and contributes to both high latency and high energy consumption.

Software pipeline composition. Fig. 6 illustrates the breakdown of minor page-fault latency across its key software components: (i) page-table operations, (ii) bookkeeping (e.g., LRU and cgroup accounting), (iii) page-table and rmap updates, (iv) physical page allocation, (v) VMA lookup and permission checks (vi) synchronization and interrupt handling, and (vii) TLB shootdowns. We measure the execution time of over 150 kernel functions involved in minor faults using an ftrace-based [101] function-level tracer and aggregate them into these categories. Fig. 6 reports the average breakdown: constructing page-table entries and bookkeeping dominate (38% and 27%), and no single component determines total cost. Thus, minor-fault latency arises from the combined cost of many tightly coupled steps.

4 Limitations of Prior Work↩︎

Prior works proposed accelerating page allocation by offloading parts of memory management to hardware, avoiding software fault-handling overheads [5]–[7]. Tirumalasetty et al. [7] extend the MMU with a hardware-managed free-page queue that the kernel replenishes in the background, allowing minor anonymous faults to be satisfied without kernel traps. Lee et al. [6] show that faster storage increases the relative cost of software fault handling and introduce a HW engine that services major faults entirely in hardware. By removing context switches, pipeline flushes, and software overheads, these works show that hardware-based fault handling can reduce allocation overheads.

5 Valinor: Design Overview↩︎

We propose Valinor, a new hardware-OS cooperative memory allocation scheme that maintains the flexibility of software-based allocators while providing the low latency and energy efficiency of hardware-based ones. Valinor’s architecture centers around a programmable hardware allocation engine (PAE) that executes allocation libraries inside hardware. Fig. 8 shows an overview of Valinor’s end-to-end flow, which consists of four cooperating components, each bringing its own key benefit.

Application/OS Interface (Programmability). Applications explicitly express their allocation requirements, either coarsely for the entire process or finely for specific memory objects, by issuing special directives . These directives are conveyed to the OS via system calls, allowing applications to specify which library should govern which memory objects. The OS interprets these directives and selects the appropriate allocation library . This interface provides programmability: different regions of an application can be bound to different policies without modifying hardware, and the OS retains full authority over which library is used where.

Allocation Libraries (Custom & Intelligent Policies). Each allocation library defines a complete memory-management policy through a small set of functions. Libraries encapsulate policy (e.g., low-latency) and maintain their own metadata (e.g., hash-based sets, private free lists). This enables policy specialization that existing hardware allocators can not express. To activate a library, the OS loads its code and metadata into memory  and configures the PAE by passing the required pointers .

MMU Extensions for Library Binding (Eligibility). When the MMU encounters a TLB miss , the page-table walker (PTW) determines whether the faulting address belongs to a library-bound region. Valinor augments the PTW with a lightweight filtering structure that extracts the properties of the memory object (e.g., anonymous versus file-backed). This filter determines in hardware whether there is a library bound to the object (i.e., hardware allocation is applicable ). This preserves low-latency allocation by preventing unnecessary traps into the OS, while ensuring libraries only see faults they are expected to handle.

Programmable Allocation Engine (PAE) (Hardware-Class Latency + Adaptive Placement). If an object is associated with a library, the PTW forwards the request to the PAE , which executes the library’s translate routine to check whether the page is already mapped /. If not, it runs the library’s alloc routine , which may choose a physical page using policy metadata and hardware-visible state (e.g., DRAM bank load). On success, the PAE returns the physical page number (PPN) to the PTW, completing allocation entirely in hardware. If the library declines or cannot allocate, the request falls back to the OS page-fault handler. This design achieves hardware-class latency and energy efficiency while supporting policies that adapt to real-time microarchitectural conditions.

6 Valinor: Detailed Design↩︎

This section presents Valinor’s architecture. We first describe the application/OS interface that lets applications bind memory regions to hardware-backed allocation policies (§6.1). We then outline the structure of the allocation libraries that implement these policies (§6.2). Next, we summarize the MMU extensions that identify regions using hardware-based allocation (§6.3). Finally, we describe the architecture of the PAE (§6.3.2).

6.1 Application/OS Interface↩︎

Expressing Requirements. In Valinor, applications (or system libraries on their behalf) explicitly express allocation requirements. This keeps the mechanism simple and under application/OS control. In practice, higher-level runtimes, such as serverless platforms [102], microservice frameworks [103], memory allocators (e.g., TCMalloc [104]) can automatically insert these directives, enabling transparent use of specialized hardware-based allocation without source code changes.

Interface. Userspace applications specify which memory regions should use hardware-based allocation through custom libraries in two ways: enabling a library for the entire process (covering all its objects, e.g., via mmap), or binding a single object to a library. Valinor provides three system calls: enable_lib(<LIB_NAME>) and disable_lib(<LIB_NAME>) for process-wide control, and mmap_with_lib(..., <LIB_NAME>) to attach a single memory object. As shown in Fig. 7, an application can wrap a code region with enable_lib/disable_lib to indicate that it allocates short-lived objects and should use a low-latency library ,. Upon receiving this directive, the OS (i) allocates space for the library’s code and metadata , (ii) associates the process or VMA with the library ID , and (iii) programs a Control and Status Register (CSR) to inform the allocation engine about the library’s code and data locations.

6.2 Allocation Library Design↩︎

Library Structure. Valinor relies on custom libraries that guide the PAE’s operation. To enable a complete memory management solution, every library needs to implement specific functions, as well as specify to the OS how to initialize/configure it. These functions are summarized in Table 2.

Example: Low-latency Allocation. In Fig. 7, we illustrate an example of a library optimized for low-latency allocation via set-associative page placement.

3pt

Configuration: At startup, the library requests initialization information from the OS via an ioctl . The handler and specification are provided by the library writer and may include parameters such as the size of the autonomously-managed segment or the root of a process’s page table.

Request: On its first handled fault, the low-latency library requests a memory segment from the OS to use as its privately managed region, avoiding trapping into the OS upon each allocation/free . The library issues a request call with a type=request and the desired size. The OS allocates the segment and returns its starting physical address. The library then organizes the segment as a set-associative structure to simplify metadata management: it divides the region into multiple sets, each with a fixed number of pages, and initializes a tag array to track free/allocated pages.

Translation: When an allocation request is received from the PAE, the library first checks whether the requested virtual address (VA) has already been mapped . To achieve this, it uses a simple hash function on the VA to determine the set to look into , and then scans the tags of the pages in the set to check for a hit. If a match is found, the library returns the corresponding PPN and permissions .

Allocation: If the translation function does not find a mapping for the requested VA, the library proceeds to allocate a new page for it. To do so, it uses a hash function on the VA to determine the set to allocate from . The library then scans the tags of the pages in the set to find a free page. If a free page is found, the library updates its tag to indicate that it is now allocated and returns the corresponding PPN and permissions to the PAE . If no free page is found, the library, the allocation fails, and the library returns NULL to indicate that the fault needs to be handled by the OS .
Free: When the OS invokes the free function, the library scans through its tag array to identify and invalidate any mappings that fall within the specified range of VAs  and zeroes out the pages in its data segment corresponding to the freed mappings . In this way, the library reclaims the pages for future allocations without releasing them back to the OS.

6.3 Hardware Support↩︎

We will now describe (i) how the MMU discovers whether an allocation should be handled by the PAE, and (ii) the architecture of the PAE. Fig. 9 illustrates the key components of Valinor’s hardware design and flow of operations.

6.3.1 Discovering Memory Object Properties↩︎

Upon a TLB miss, the MMU invokes the PTW to resolve the virtual-to-physical translation  . If the PTW process fails to find a valid mapping  , it needs to determine whether the requested memory object is eligible for hardware-based allocation using the PAE. To do so, the PTW needs to discover the properties of the memory object being accessed (e.g., file-backed or anonymous, and its access permissions). The reason for this is twofold: (i) the PAE may be configured to handle only specific types of faults (e.g., minor faults for anonymous pages), and (ii) the PAE’s functions need the access permissions of the memory object being accessed. All this information is maintained by the OS in its VMA structures [105], which are kernel-specific (e.g., red-black tree [105] or maple tree [106]). Each process maintains a list of VMAs that cover its entire VAS.

VMA Filter. To obtain a memory object’s properties, the PTW must (i) locate the VMA covering the virtual address and (ii) extract its attributes. Valinor adds a hardware VMA Filter (VMAF) to perform these steps efficiently. The VMAF has two components: a small CAM that caches recently accessed VMA entries (VMA CAM) and a VMA Properties table that stores the attributes of cached VMAs. The VMA CAM holds several entries (16 in our design), each storing the bounds of a VMA and comparators to check whether an address lies within them. The \(i\)-th entry of the VMA Properties table holds the metadata for the \(i\)-th CAM entry.

Lookup Flow. When the VMAF receives a request for a particular VA from the PTW, it first looks the address up in the VMA CAM  . If the entry lies within the bounds of a CAM entry, then the corresponding properties are immediately fetched from the VMA Properties table  . Otherwise (on a CAM miss  ), the PAE executes the library’s vma_walk() routine to traverse the OS’s VMA structures. If no entry is found, then the requested VA is invalid. Otherwise, the routine returns the relevant VMA information, which is placed in an entry of the VMA CAM and Properties table, evicting an existing entry if necessary  . Finally, the properties are returned to the PTW.

6.3.2 Programmable Allocation Engine (PAE)↩︎

A lightweight compute unit implemented as a simple three-stage in-order pipeline with small (4 KB each) instruction and data caches [107]. The caches are backed up by the last-level cache (LLC) to provide low-latency access to larger data structures, instead of fetching them from main memory. To avoid the cross-core coherence, the PAE is placed inside the memory controller rather than within per-core MMUs.

Allocation libraries are executed using a compact RISC-V RV32I-compatible ISA [108] that supports arithmetic, logical, memory, and control-flow operations. We extend this ISA with custom instructions for reading microarchitectural telemetry (e.g., per-bank DRAM conflict counters), enabling us to implement adaptive, telemetry-driven allocation policies (see §7).

VMAF properties are used to check hardware-based allocation eligibility of the memory object. If the object is not eligible, the PTW directly falls back to the OS page-fault handler. Otherwise, it forwards the request and parameters (e.g., PID) to the PAE  . A deduplication CAM filters out the request, in case there exists an in-flight one which already target the same VA. Otherwise, the PAE runs the library’s translate function to determine whether the VA is already mapped  . If a mapping exists, it returns the PPN and permissions to the PTW  , which updates the MMU and resumes execution. Otherwise, the PAE executes the library’s alloc function  . On success, it returns the new mapping  . On failure, it returns NULL, indicating the OS must handle the fault.

6.4 System Integration and Multi-Tenant Deployment↩︎

Library Selection. Each process may enable one or more allocation libraries. The OS maintains a per-process (or per-VMA) association between memory regions and library identifiers. During a TLB miss, the PTW consults the VMAF to determine which library (if any) governs the faulting region. The selected library’s code and data segments are specified to the PAE via per-library CSR pointers. This indirection allows the PAE to switch between libraries on a per-fault basis, without flushing internal state or performing re-initialization.

Library Sharing. Valinor allows multiple processes to share the same allocation library. The library’s code is loaded once, and the PAE invokes it on behalf of any process that has bound a region to that library. During execution, the PAE passes the caller’s PID (and other request context) to the library’s routines (e.g., alloc). Using this PID, the library is responsible for ensuring correctness. For example, it distinguishes requests from different processes and manages their memory independently.

Coordination Across Multiple PAEs. In systems with multiple memory controllers, each controller hosts its own PAE. Because the physical frame is determined by the library’s alloc routine, library binding cannot depend on the physical address. Instead, Valinor binds each allocation library instance to a specific PAE at configuration time. When the OS loads a library, it selects a target PAE (e.g., round-robin or load-balanced) and records this binding. On a TLB miss, the PTW identifies the library for the faulting region and forwards the request directly to its assigned PAE. This keeps all library-related state local to one PAE, avoiding inter-PAE coordination, while allowing the OS to flexibly place libraries and tune performance without modifying the MMU or PTW.

Supporting fork / Copy-on-Write (CoW) semantics. Because Valinor maintains VA-to-PA mappings in both the OS page table and the library’s tag array, explicit synchronization is needed before fork() to preserve CoW semantics. Valinor addresses this with a lightweight drain step. Before forking, the OS walks the library’s tag array of the calling process, installs a standard PTE for each valid mapping, and reclaims the library’s physical segment. After draining, fork() proceeds using Linux’s unmodified CoW machinery. The kernel marks shared pages read-only, and subsequent writes trigger standard CoW faults, handled entirely by the OS. After fork, the library is back in the initial state: its tag array is empty and it owns no segment. Correctness follows directly from Linux’s fork()/CoW implementation, which operates over the page table. After the drain, all mappings reside in the page table, so the existing semantics apply. The required kernel changes are minimal: only a drain hook is inserted before fork(). The fork() code itself and the CoW page-fault handler remain unmodified. Since fork() already traverses the full VMA tree, the incremental cost of walking the compact tag array is insignificant.

When it comes to VMAF, the properties table can hold stale entries if the OS mutates VMAs concurrently (e.g., via mprotect()). Valinor exposes a programmable vma_walk function (Table 2) so that VMA lookup is portable across kernel VMA layouts without hardware changes. The fenced protocol that prevents a concurrent alloc from committing against stale VMA properties is defined formally as transition (3) in §[sec:sec:ownership].

Security Analysis. We assess the attack surface introduced by Valinor’s architectural extensions across three threat categories. Our attacker model assumes that the the PAE library code is trusted by the kernel.

7 Specialized Allocation Libraries↩︎

Valinor supports a wide range of specialized allocation policies besides the low-latency one. These policies are not intended to be the optimal or most sophisticated designs for their respective goals. They are simple, representative policies constructed and evaluated to show how diverse allocation strategies can be realized using Valinor’s programmable substrate. Below, we present three such libraries and evaluate them in §9.2. We additionally describe a that illustrates tier-aware placement (§[sec:sec:tier-aware]).

7.1 Library↩︎

7.2 Speculation-Enhanced Allocation Library↩︎

This library accelerates common sequential and strided allocation patterns by predicting and pre-establishing future page mappings, extending the low-latency allocator of §6.2 with two lightweight structures: a stride detector that tracks the deltas between successive faulting VAs per region, and a small speculative buffer that holds pre-allocated mappings not yet claimed by a fault. When the detector observes a stable stride across consecutive allocations (e.g., the linear page-touch pattern of a JSON parser streaming through a buffer), the library pre-allocates the next few pages from its private physical segment and records them in the speculative buffer, performing this work off the demand path while the PAE would otherwise be idle.

The value of speculation here is latency hiding rather than throughput. When a subsequent fault matches a buffered prediction, the library commits the pre-allocated page immediately. It returns the PPN with no allocation work on the critical path, so the fault is resolved in the time of a metadata lookup. This is why, in our evaluation (§9.2), the speculation library hides over 99% of page-fault latency even on the simple in-order PAE. The actual allocation has already happened before the fault arrives, leaving only commit on the hot path.

Speculation must not violate the single-ownership invariant of §[sec:sec:ownership], so a speculatively pre-allocated page is owned by the library but unmapped until a matching fault commits it. It is never installed in the TLB or page table on speculation alone. The deduplication CAM still admits only one in-flight resolution per VA, so a real fault and a speculative pre-allocation for the same VA cannot both commit. On a misprediction or stride change, the library simply reclaims the unclaimed buffered pages back into its segment’s free pool. No OS interaction and no rollback of installed state are required, because nothing was installed. If no prediction matches, the library falls back to ordinary allocation, or, if its segment is exhausted, lets the PTW invoke the OS. Freeing a page also clears any speculative state derived from it. The result preserves the low-latency critical path while opportunistically eliminating allocation cost for predictable workloads, at the cost of a small speculative buffer and the pages held in it.

8 Evaluation Methodology↩︎

Table ¿tbl:tab:simconfig? summarizes the configuration of the real RISC-V prototype, the simulated system used in our evaluation and the real server-grade CPU used for the experiments shown in §3.

RISC-V Soft-core Prototype We prototyped our proposed design and tested it on a Xilinx ZCU106 FPGA [9] to (i) showcase the feasibility of implementation of our key mechanism and (ii) evaluate the performance benefits of Valinor on a real system running a full-blown OS. To model the PAE, we employed a small 32-bit in-order RISC-V core [107]. We integrated this unit within the translation subsystem of the BOOM OoO core to accommodate allocations in hardware. The prototype runs Linux kernel v.5.11.6 [109].

Simulation We used Sniper [11] in combination with Virtuoso [110] to simulate a full-system x86 environment with detailed modeling of the MMU and memory hierarchy. We configured MimicOS (the lightweight OS modeled in Virtuoso) to mimic Linux’s minor page-fault handling behavior. Virtuoso was extended to model the Valinor architecture, including the PAE, allocation libraries, and their interactions with the OS and MMU. We will open-source the RISCV prototype and our simulation infrastructure to enable further research.

Workloads. We use workloads from three sources. DeathStarBench [76] is an open-source cloud microservices suite. vSwarm [97] is a set of ready-to-run serverless benchmarks. Bitnet [98] is an inference framework for 1-bit LLMs targeting resource-constrained devices. From DeathStarBench, we use all hotel-reservation microservices (hotel-res) and a recommendation service (rec-sys). From vSwarm, we use distributed compilation (comp), aes, authentication (auth), video analytics (video), compression (zlib), and image rotation (img-rot). We also add a Python json parser, a Python wcount function, and a C++ sqlite-based key–value store accessed over HTTP. From Bitnet, we use the b1.58 2B4T model (>2B parameters) and run four inference tasks using the benchmark’s generator: three with a 100-token prompt and outputs of 3, 10, and 20 tokens, and one with a 100-token prompt and a 3-token output. All workloads run to completion in both simulation and the RISC-V prototype, with warm-up to avoid cold starts (all required binaries, libraries, images, and models are preloaded).

9 Evaluation Results↩︎

9.1 RISC-V Soft-core Prototype Results↩︎

We first evaluate our proposed design on a RISC-V-based soft-core FPGA prototype as described in §8. We compare the performance of four different systems: i) Baseline: unmodified BOOM processor running unmodified Linux Kernel, ii) SW-only: unmodified BOOM processor with modified Linux kernel implementing the Low-Latency Allocation library (§6.2) solely in software, iii) Fixed-HW: modified BOOM processor with fixed hardware module that implements the Low-Latency Allocation library (§6.2) and handles allocations in hardware, and iv) Valinor: our proposed design with a PAE that implements the Low-Latency Allocation library (§6.2) and handles allocations in hardware. We evaluate all four systems across a subset of the benchmarks described in §8 (the ones we could successfully run on RISC-V BOOM), and 4 microbenchmarks that we designed to specifically stress test the allocation mechanisms. The short-lived allocation pool size is set to \(16MB\) for all experiments, unless otherwise specified.

Fig. 10 shows the end-to-end performance speedup measured on the FPGA prototype for our evaluated scheme. We make three key observations: (i) Despite making no modifications to the HW, the SW-only version already leads to an average speedup of \(9\%\), which is indicative of the allocation scheme’s efficiency compared to the baseline Linux’s page fault handler, (ii) the Fixed-HW version is the fastest, achieving an average speedup of \(16\%\), thus showcasing the benefits of offloading page fault handling in HW and, (iii) Valinor consistently outperforms both the baseline and SW-only by 16% and 7% on average, while still being only \(3\%\) slower on average commpared to Fixed-HW. We conclude that, despite employing a general-purpose programmable core, Valinor’s performance is still comparable to that of a fixed HW module. Our energy measurements with Synopsys [111] indicate that Valinor saves on average 5% (up to 8%) of energy over Linux (figure not shown due to space constraints).

Page Fault Latency Analysis. To illustrate the performance benefits of our design, Fig. 11 reports the average cycles spent on page-fault handling across systems. We make three observations. First, baseline Linux requires 5881 cycles per minor fault (consistent with our server-grade measurements from §3). Second, delegating allocation entirely to fixed-function HW yields a 48x speedup due to bypassing the page-fault handler. Third, although Valinor’s in-order core is  3x slower than Fixed-HW (344 vs. 122 cycles), it still provides a 17.07x improvement over Linux. Overall, even with a programmable fabric, moving allocation into HW offers substantial gains by avoiding the overheads of the OS fault-handling.

Sensitivity Analysis on Allocation Pool Size. To illustrate the benefits of hardware-assisted allocation, we evaluated two benchmarks (aes and transp) across a range of allocation pool sizes (4MB to 16MB). Fig. 12 compares the page fault latency reduction achieved by Fixed-HW, Valinor, and SW-Only, all relative to baseline Linux. We make two key observations. First, hardware methods are superior even for small pool sizes. With a 4MB pool for aes, the hardware-assisted Fixed-HW and Valinor achieve latency reductions of approximately 24% and 23%, respectively. This is already a distinct advantage over the SW-only approach, which reduces latency only by 13%. Second, the performance gap widens drastically with larger pools. For a pool of size 16MB, in aes, Fixed-HW and Valinor deliver latency reductions of 95% and 93%, respectively. In contrast, the SW-only method’s latency reduction is limited to 60%. The transp benchmark exhibits an even more accentuated trend, with Fixed-HW and Valinor achieving significantly greater reductions than the software-only solution.

9.2 Design Space Exploration in Simulation↩︎

To complement our FPGA prototype, we evaluate Valinor in simulation as described in §8.

Library and Microarchitecture Exploration. We evaluate four allocation libraries: Low-Latency Allocation (LLA) (§6.2), Integrity-Merkle, Integrity-MAC (§7.1), and Speculation-Based Allocation (SBA) (§[sec:design:specialized95libraries:spec]). We also prototype four PAE microarchitectures: a 3-stage in-order core, an OoO core, a CGRA similar to [112], and a fixed-function module. Libraries are ported to the CGRA using Morpher [113]. For fixed hardware, we use RTL-derived latencies via HLS and simulate memory accesses in Sniper [11]. Fig. ¿fig:fig:design95space? shows the average page-fault latency reduction across all workloads in §8, normalized to the baseline OS handler. We make four observations. First, SBA is most effective, cutting latency by over 99% and nearly hiding page faults even on the in-order core. Second, for the other libraries, the in-order core still performs well: 90% reduction for LLA, 93% for Integrity-MAC, and 73% for the more complex Integrity-Merkle. Third, Integrity-Merkle is the only case where the OoO core offers a major boost, improving 73% to 87% by exploiting ILP. Fourth, the CGRA and fixed hardware achieve the highest performance (96–99%) across all libraries, though CGRAs cannot guarantee that every library maps cleanly. Overall, Valinor’s PAE supports a wide range of libraries and delivers substantial gains even with a simple in-order core.

Telemetry-driven Adaptive Allocation Library. To showcase Valinor’s adaptability, a key benefit over fixed-function hardware, we evaluate the Telemetry-driven Adaptive Allocation (TAA) library. We configure Valinor’s PAE using the simple, in-order core design and run two applications colocated on the same CPU: (i) a DRAM-latency sensitive benchmark (pagerank) and (ii) an allocation-sensitive serverless function (compilation). Our goal is to demonstrate how TAA can adapt its allocation policy based on runtime telemetry to optimize data placement and reduce DRAM bank-level interference. Fig. 13 shows the end-to-end performance for both applications relative to running in isolation (the 1.0 line). We compare four scenarios: (i) the baseline OS, (ii) the OS with software bank-coloring, (iii) a Fixed HW allocator, and (iv) Valinor executing TAA. We make three key observations. First, in the Software Baseline, pagerank experiences a 17% performance degradation (0.83x performance) due to interference from the co-located compilation task. Third, both Software Bank-Coloring and Valinor (TAA) successfully mitigate the interference on pagerank, restoring its performance to  98-99% of its baseline. However, Valinor also accelerates the compilation task by 1.25x, a benefit the software-only bank-coloring approach cannot provide. This demonstrates TAA’s ability to adaptively manage placement to both protect victim applications and accelerate allocation-sensitive ones.

9.3 Area & Power Overheads↩︎

To quantify hardware cost, we used the Yosys Open SYnthesis Suite [114] to measure area and estimate power for both Fixed-HW and Valinor. For Valinor, we synthesized the 3-stage in-order PAE, assuming an 8-core system (Small BOOM cores) with a single shared engine, using the nangate45 library [115]. Fixed-HW adds only 0.12% chip area, while Valinor incurs a modest 1.5%. Static power scales with area, yielding a 1.5–1.8% leakage overhead for Valinor, compared to <0.2% for Fixed-HW. Overall, Valinor’s flexibility incurs minimal hardware overhead, thanks to a single unit shared system-wide.

10 Related Work↩︎

To our knowledge, Valinor is the first scheme to combine hardware-class allocation performance with the flexibility and programmability of software-defined policies. In this section, we qualitatively compare Valinor to prior efforts that extend hardware or software to accelerate memory management.

Hardware-Based Allocation. Prior works [6], [116]–[118] accelerate page-fault handling by moving parts of the OS handler off the critical path, maintaining a hardware-managed pool of free pages [6], or establishing mappings in hardware [117]. While these approaches reduce fault latency, they (i) consume system resources for background tasks, (ii) offer no control over placement policy, and (iii) still rely on software to manage physical-page reservation. In contrast, Valinor offloads the entire page-level allocation path to hardware, while allowing software-defined mapping policies.

Offloading Memory Management to Hardware. A large body of work accelerates memory management at object granularity [5], [119]–[126], often incorporating hardware-assisted or hardware-managed allocators and garbage collectors [127]–[133]. These systems require ISA extensions to notify hardware of allocation and free events, and focus on small, malloc-sized objects. Valinor differs fundamentally: it targets page-granularity allocation and relies exclusively on existing MMU events (TLB misses and faults), avoiding ISA changes. As such, it is orthogonal to prior work on hardware-assisted object allocators and garbage collectors, which operate at much finer granularity and require explicit ISA support.

11 Conclusion↩︎

We presented Valinor, a hardware–OS cooperative allocation substrate that pairs software programmability with hardware-class latency. Valinor introduces a programmable hardware allocation engine (PAE) that executes OS-supplied allocation libraries, enabling low-latency page allocation while supporting diverse policies, from short-lived object allocation and integrity enforcement to telemetry-guided placement. Implemented on a BOOM RISC-V soft core and evaluated in hardware and simulation, Valinor reduces allocation latency by 17×, improves end-to-end performance by 16%, and cuts energy by up to 8%, while supporting several allocation libraries. Valinor demonstrates that memory allocation can be both efficient and fully programmable, enabling a new class of adaptable, policy-aware virtual memory systems.

Acknowledgments↩︎

We thank the SAFARI Research Group members for their constructive feedback and for providing a stimulating intellectual and scholarly environment. We acknowledge the generous gift funding provided by our industrial partners (especially Google, Huawei, Intel, Microsoft), which has been instrumental in enabling the research we have been conducting on memory systems.

References↩︎