1 Introduction↩︎

Enabling robots to perform music poses significant challenges in dexterity and long-horizon rhythmic control. Drumming, in particular, is a highly complex task that demands precise timing, high-frequency intermittent contacts, dynamic motor coordination, and fine-grained control over acoustic outcome. While robotic manipulation has seen impressive advancements in recent years [1], these are often confined to goal-driven tasks with well-defined end states. In contrast, playing music is inherently process-driven, involving intricate timing, and the ability to adapt to changing rhythms. Success depends not on reaching a final configuration, but on sustaining an evolving sequence over a long temporal horizon. Even minor deviations in timing can profoundly affect the emotional quality and impact of a performance.

Reinforcement Learning (RL) has demonstrated remarkable success in enabling humanoid robots to master goal-driven behaviors, such as getting up [2], stabilizing [3], and tracking agile motions [1], [4]. In the domain of robotic musicianship, RL has been applied to piano playing with anthropomorphic dexterous hands [5], as well as to a simplified two-DoF underactuated drumming arm [6]. Despite these recent efforts, the challenges posed by process-driven tasks like music remain largely unexplored by policy-learning methods. To our knowledge, no prior work has investigated learning-based control for coordinated humanoid drumming.

In this work we introduce Robot Drummer, a general RL framework that enables a humanoid to perform drum tracks from MIDI (Musical Instrument Digital Interface) transcriptions. Our approach (i) formalizes drumming as a sequence of timed contact events (i.e., the Rhythmic Contact Chain), (ii) makes long performances tractable by training a single policy across fixed-length temporal segments, and (iii) uses dense contact-based rewards that penalize missed and incorrect strikes while rewarding correct hits. Together, these contributions enable policies that achieve high rhythmic fidelity across a diverse set of songs and exhibit emergent, human-like drumming strategies.

2 Related Works↩︎

2.1 Learning for Humanoid Control↩︎

Recent humanoid controllers largely follow a motion tracking-based learning paradigm, where policies are trained to reproduce re-targeted reference trajectories. These are typically drawn from motion capture datasets [7] or videos [3], [4], [8], often augmented by motion priors [9] and trained with curriculum schedules [2], [3] to enhance robustness and diversity. To tackle contact-rich and dynamic tasks such as parkour jumping, a sequential contact decomposition approach has been used in [10] that uses staged reference contacts to simplify reward design. Sim-to-real alignment efforts, such as those in [4], have improved fidelity for agile whole-body motion tracking policies in real environments.

While imitation-based approaches have shown impressive performance in acquiring loco-manipulation skills, they are tied to demonstrated motions in the datasets and optimize primarily over a few-second horizon. For drumming, persistent occlusions by the drum kit, high-speed stick impacts, and dense environment contacts make it difficult to obtain reliable human motion references, whether via motion capture or video-based methods. Moreover, drumming is a contact-rich, long-horizon task, where coordination skills are tightly coupled to temporal contact sequences, that are not well captured by frame-wise motion tracking objectives.

2.2 Learning to Play Music↩︎

Robotic musicianship has advanced from purely trajectory-based systems [11] to data-driven and learning-based paradigms [5], [6]. Early works used simple arms and rules-based improvisation to interactively play alongside humans [12]. More recently, RL has enabled end‐to‐end training of anthropomorphic hands to play piano pieces [5]. In drumming, a two-DoF underactuated arm was trained to generate rhythmic patterns under simplified kinematics [6]. Additionally, wearable robotic prosthetics have been developed to augment human drumming abilities, using muscle signals to adapt to the musical contexts [13].

All of the aforementioned works either focus on simplified, low DOF embodiments for drumming or on floating-hand setups for piano playing. In contrast, Robot Drummer treats percussion as a long-horizon skill that requires coordinated, high-DoF humanoid control and provides a general simulation framework to advance robotic musicianship.

3 Robot Drummer: Learning Rhythmic Skills↩︎

Our goal is to enable a humanoid robot to perform a musical repertoire by striking the correct sequence of drums (as illustrated in Figure 1), based on MIDI transcription of sheet music. Each drum is modeled as a collision surface which registers an impact when struck with a sufficient force, causing the synthesizer to produce the corresponding sound. This problem is challenging for three reasons. First, the task is contact-rich: each correct outcome depends on precise, intermittent impacts that are sensitive to small spatial and timing errors. Second, the task is long-horizon: songs can last from a few minutes to over six minutes, containing hundreds of hits, that lead to poor exploration. Third, effective drumming couples temporal and spatial planning: choices about which stick to use and where to position the hands affect the ability to execute upcoming contacts. To address these issues we represent a drum score as a Rhythmic Contact Chain (RCC) (Section 3.2), split each RCC into fixed-length segments that are trained in parallel (Section 3.3), and design task-agnostic contact rewards that provide graded signals for correct, wrong, and missed strikes (Section 3.4). An overview of our approach is illustrated in Figure 2.

3.1 Music Representation↩︎

While music can be represented as raw waveforms or spectrograms, these formats present challenges for robots, particularly when processing continuous data and designing appropriate reward functions. We use MIDI as a musical representation, which provides a standardized way to encode instruments’ timing in digital music. Specifically, we use publicly available community-created MIDI files, which are individual recreations of popular songs that may differ from original recordings in timing and instrument assignment. From a full MIDI sequence, we extract only the drum events, each containing the note, start time, and end time. To relate these events to a drum kit, we map distinct MIDI notes to corresponding drums (see Figure 1). This mapping is song-specific, as the same drum may correspond to different notes across songs. In addition, each drum can have multiple articulations, which are variations in how the drum is played, for example, open or closed hi-hat. These articulations produce distinct sounds and are represented as separate note numbers on a General MIDI percussion map ³. When multiple articulations appear for the same drum in a song, we select only the most frequent one to simplify the representation. Extending our framework to multiple articulations is left for future work.

3.2 Rhythmic Contact Chain↩︎

Humanoid drumming performance can be viewed as a sequence of precise and coordinated contact events, where each event corresponds to drum(s) strikes at a specific time. We model this sequence as the Rhythmic Contact Chain, which represents the timed contacts the robot must execute to perform a musical piece. Formally, we define it as:

\[\mathcal{C} = \{\,C_1, C_2, \dots, C_N\},\]

where \(C_i\) denotes the contact step at time \(t_i\) and may include multiple contact pairs corresponding to the target drum strikes. Each contact step is defined as:

\[C_i = \bigl(D_i,\,t_i,\,V_i,\,S_i\bigr),\]

with \[D_i \subseteq \mathcal{D},\qquad V_i \in \mathbb{R}^{|\mathcal{D}|},\qquad S_i \subseteq \{s_{\text{L}}, s_{\text{R}}\},\]

where \(D_i\) denotes the set of drums to strike at time \(t_i\), as specified by the drum track, \(V_i\) is a per-drum velocity vector aligned with the drum set \(\mathcal{D}\) (i.e., \((V_i)_d\) gives the MIDI velocity for drum \(d\) when \(d\in D_i\)), \(S_i\) optionally specifies which stick(s), left or right, may be used to execute the contacts, and \(\mathcal{D}\) contains all the drums in a drum kit as shown in Figure 1. In this work we do not condition the policy or rewards on \(V_i\), treating all strikes as equal intensity, and do not provide explicit assignments \(S_i\), letting the policy discover stick-to-drum coordination autonomously through contact rewards in Section 3.4.3. Both terms are retained to support extensions to expressive dynamics or explicit stick assignment.

3.3 Temporal Decomposition↩︎

Performing an entire drum track (several minutes long) in a single RL episode leads to sparse rewards and poor exploration, since thousands of timesteps must be rolled out sequentially before any reset, limiting parallelism and delaying exposure to later song sections. To address this, we introduce a temporal decomposition approach that partitions the full drum track into a sequence of fixed-length training segments, all trained in parallel. Each segment corresponds to a subset of the contact steps in RCC. Specifically, we divide \(\mathcal{C}\) into \(M\) segments:

\[\mathcal{C}^{(m)} \;=\; \bigl\{\,C_i : (m-1)\,P + 1 \le i \le m\,P \bigr\},\]

where \(P~=~N/M\) is the number of contact steps per segment, and \(m = 1,2,\dots,M\) represents the segments of a track. Each segment \(\mathcal{C}^{(m)}\) defines a temporally localized portion of the song. This decomposition transforms a single long-horizon task into multiple shorter training segments, significantly improving the sample efficiency of learning rhythmic skills.

3.4 Policy Training↩︎

We formulate humanoid drumming as a finite-horizon extended Markov Decision Process (MDP), defined by a tuple \((\mathcal{S, A, T, R, \gamma, C})\), where \(\mathcal{S}\) is the state space, \(\mathcal{A}\) is the action space, \(\mathcal{T}\) governs the transition dynamics, \(\mathcal{T}:\mathcal{S} \times \mathcal{A} \rightarrow \mathbb{P}(\mathcal{S})\), \(\mathcal{R}:\mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}\) is the reward function, \(\gamma \in [0, 1]\) represents the discount factor, and \(\mathcal{C}\) is the contact chain defined in Section 3.2. The objective is to learn an optimal policy \(\pi_\theta\) that maximizes the expected cumulative reward \(\mathbb{E}_{\pi_{\theta}}\left[\sum_{t=0}^{H-1} \gamma^t r_t \right]\) over the horizon \(H\). To optimize the policy, we employ Proximal Policy Optimization (PPO) [15], an on-policy RL algorithm known for its stability and efficiency in large-scale parallel training environments.

3.4.1 States↩︎

The state \(\boldsymbol{s}_t\) is composed of three components: the robot’s proprioception \(\boldsymbol{s}_t^{\mathrm{p}}\), spatial information related to the drums and sticks \(\boldsymbol{s}_t^{\mathrm{sp}}\), and contact goals \(\boldsymbol{s}_t^{\mathrm{c}}\). The proprioceptive state is defined as \(\boldsymbol{s}_t^{\mathrm{p}}~\triangleq~[\,\boldsymbol{q}_t,\; \dot{\boldsymbol{q}}_t,\; \boldsymbol{a}_{t-1}\,],\) where \(\boldsymbol{q}_t \in \mathbb{R}^{15}\) are the joint positions, \(\dot{\boldsymbol{q}}_t \in \mathbb{R}^{15}\) are the joint velocities, and \(\boldsymbol{a}_{t-1} \in \mathbb{R}^{15}\) is the action applied at the previous timestep. The spatial context \(\boldsymbol{s}_t^{\mathrm{sp}} \in \mathbb{R}^{24}\) encodes the positions of the drums and sticks. The contact goals state is defined as \(\boldsymbol{s}_t^{\mathrm{c}}~\triangleq~[\,\boldsymbol{c}_t, \boldsymbol{c}_{t+1}, \dots, \boldsymbol{c}_{t+L}\,],\) where \(\boldsymbol{c}_{t+i} \in \mathbb{R}^{6}\) is a binary encoding of target drum strikes at timestep \(t+i\) and \(L\) is the lookahead horizon over future steps to provide rhythmic context. The lookahead horizon controls how far into the RCC the robot can see. The contact goals state is akin to a time-phase variable in motion tracking policy learning [16] and effectively functions as a discrete phase marker, informing the policy which drum(s) it should be targeting at that moment in the musical piece. We evaluate the impact of contact goals in our ablation study by comparing it to a continuous time phase variable.

3.4.2 Actions↩︎

The policy’s actions \(\boldsymbol{a}_t \in \mathbb{R}^{15}\) correspond to the target joint angles for a Proportional-Derivative (PD) controller, which applies torques to the robot’s joints. The torque at timestep \(t\) is computed as:

\[\boldsymbol{\tau}_t~=~ \boldsymbol{K}_p( \boldsymbol{q}_{t}^{d} - \boldsymbol{q}_{t}) - \boldsymbol{K}_d(\boldsymbol{\dot{q}_t}),\]

where \(\boldsymbol{q}_{t}^{d}\) is the desired joint configuration, defined as \(\boldsymbol{q}_{t}^{d}~=~\beta\boldsymbol{a}_{t} + \boldsymbol{q}_{0}\); \(\beta\) is an action scaling factor, \(\boldsymbol{q}_0\) represents default joint positions of each joint, and \(\boldsymbol{K}_p\) and \(\boldsymbol{K}_d\) are the stiffness and damping coefficients of the PD controller, respectively.

3.4.3 Rewards↩︎

Effective drumming requires executing hundreds of precisely timed strikes on the correct drums while avoiding any wrong hits. Optimizing solely with the sparse rewards fails to guide the policy toward right rhythmic behaviors. We hypothesize that every contact event–correct, wrong, or missed–provides crucial information for learning the fine‑grained timing and coordination needed in drumming. We thus propose unified task reward as:

\[r_t = r_t^{\mathrm{contact}} + r_t^{\mathrm{reg}},\] where \(r_t^{\mathrm{contact}}\) provides dense feedback for contact events, and \(r_t^{\mathrm{reg}}\) contains regularization terms. The contact reward is defined as:

\[r_t^{\mathrm{contact}} = w_{\mathrm{c}}\,r_t^{\mathrm{correct}} + w_{\mathrm{w}}\,r_t^{\mathrm{wrong}} + w_{\mathrm{m}}\,r_t^{\mathrm{missed}} + w_{\mathrm{p}}\,r_t^{\mathrm{prox}},\]

where the terms respectively reward correct target strikes, penalize wrong and missed contacts, and encourage proximity to target drums at expected hit times. The first three terms directly correspond to the contact events in Rhythmic Contact Chain, defined in Section 3.2, while the last term is a proximity reward which encourages the agent for moving sticks closer to target drums at moments when the hits are expected. A full list of reward terms and weights is provided in the supplementary material. The weights are based on an intuition that missing a strike is worse than hitting a wrong drum, while correct hits should be rewarded more to encourage well-timed strikes.

3.5 Music State Initialization↩︎

State initialization plays a crucial role in training RL policies, where the key idea is to initialize the agent’s starting state at random phases along a reference motion trajectory. Recent works have shown the effectiveness of state initialization in tracking reference motions [4], [16]. To apply this idea in the context of drumming, we propose Music State Initialization, where we randomly sample contact steps within a segment \(\mathcal{C}^{(m)}\) of RCC and initialize the corresponding contact goals state at those steps. This effectively randomizes the starting point of the drum track and exposes the policy to high reward contact states early on in the training.

4 Experimental Setup↩︎

We evaluate Robot Drummer through a series of experiments in Isaac Gym simulation environment using a Unitree G1 humanoid robot. The robot is controlled through 15 degrees of freedom (DoF)—7 DOF for each arm and 1 DOF for torso—with drumsticks rigidly mounted to the hands and all striking motions generated via upper body. Our experiments are designed to investigate the following key questions:

• Q1. How does the Robot Drummer perform across songs of varying complexity?

• Q2. What role does each component of the Robot Drummer’s design play in its overall performance?

• Q3. How does a policy trained on multiple songs compare against specialist policies?

4.1 Quantifying Rhythmic and Spatial Complexity↩︎

To address Q1, we first identify the factors that affect the difficulty of drumming in a given song and then train specialist policies. The primary difference in songs complexity lies in the pattern of drum hits, both in time and space. To systematically assess these patterns, we introduce a set of features that capture the song’s key characteristics. These features, described in detail below, provide a quantitative measure of a song’s complexity.

4.1.1 Number of Drums Used↩︎

The number of distinct drums in a song directly affects its rhythmic and spatial complexity. More drums introduce variation into the pattern, requiring greater coordination and contributing to a more intricate rhythm. In this work, the maximum number of drums that can be used in a song is 6, as illustrated in Figure 1.

4.1.2 Beats per Minute↩︎

The Beats per minute (BPM) measures the tempo or speed of the song. It indicates how fast or slow the song is played, determining the pace at which drumming occurs.

4.1.3 Normalized Pairwise Variability Index (nPVI)↩︎

The normalized Pairwise Variability Index (nPVI) [17] quantifies the relative variability in timing between successive drum hits, known as inter-onset intervals (IOIs). For each adjacent pair of IOIs, we compute the absolute difference divided by their mean, then average these values across the entire song. A higher nPVI reflects greater temporal irregularity and thus a more challenging rhythm for the robot to execute.

4.1.4 Hit Distribution Entropy↩︎

Some songs concentrate drum hits on a few drums, while others spread them more evenly. We use Shannon entropy to quantify how dispersed the strikes are. A more even distribution of drum hits yields higher entropy, indicating that the robot must switch between different drums throughout the song and coordinate wider spatial movements between the hits.

4.1.5 Polyphony↩︎

Although the humanoid robot is physically limited to striking two drums at once (one per stick), MIDI drum sequences often encode complex polyphonic textures with three or more drums at the exact same moment, e.g., simultaneous hit on the snare, tom, and cymbal. We quantify this polyphony by measuring the fraction of time‐steps where the MIDI drum track requests more than two simultaneous hits. These overlapping notes signal moments in the song when the robot must prioritize which drums to hit.

4.2 Evaluation Metrics↩︎

To evaluate the drumming performance of learned policies, we use precision, recall, and F1 score by comparing ground‐truth sequence of desired drum strikes in RCC (see Section 3.2) to the robot’s actual hits at each timestep. Precision measures the proportion of the robot’s strikes that correctly align with RCC events (i.e., on the intended drums) over the entire song, while recall captures the fraction of ground-truth contact events the robot actually executes. The F1 score is the harmonic mean of precision and recall and provides a single, balanced metric of overall rhythmic fidelity on a 0-1 scale, with higher values indicating better performance.

5 Results↩︎

To evaluate the performance of Robot Drummer, we trained over 30 specialist policies on popular rock and metal tracks of varying complexities. Table 1 summarizes the complexity metrics for songs with zero polyphony (no more than 2 simultaneous hits) alongside corresponding F1 scores achieved by the policies. The results show that specialist policies consistently achieve high F1 scores. Songs with fewer drums and regular rhythmic patterns (low nPVI) yield near-perfect performance, while increased drum count and higher rhythmic variability (high nPVI) drops the F1 score. Extended results for songs with non-zero polyphony are provided in the supplementary material. Additionally, Figure 3 provides qualitative snapshots of challenging drumming sequences: a drum fill, which involves a rapid succession of hits across multiple drums to transition between sections, and a snare roll, which involves high-speed hits on a snare drum. These examples demonstrate the robot’s ability to perform both spatially and temporally demanding patterns with precise coordination.

To quantify which song characteristics most strongly affect performance, we also computed Spearman’s rank correlation between each feature and per-song F1 score across all specialist policies. We found that rhythmic irregularity (nPVI) shows the strongest negative correlation with F1 score (\(\rho = -0.52\)), indicating that songs with highly uneven timing between hits are most challenging. The number of drums used in a song also correlates negatively with F1 score (\(\rho = -0.42\)). This trend aligns with our expectations as each additional drum target amplifies the difficulty of policy learning due to a larger spatial manifold. The policy needs to coordinate larger movements across a wider space with a greater potential for incorrect and missed hits.

Figure 4 compares the performance of specialist policies on two representative songs (Nirvana - Lithium and The Police - Roxanne), by overlaying the target hits versus actual robot strikes alongside the original drum track. The two policies achieve notably different F1 scores despite similar values for all rhythmic complexity metrics. Lithium’s drum pattern is highly regular and evenly spaced, whereas Roxanne’s pattern has irregular cluster of hits with many switches between drums in quick succession (see Figure 4 (right)). The regularity in Lithium gives the robot sufficient time to swing each stick between target drums, resulting in an almost perfect F1 score (0.977). By contrast, Roxanne’s sequence interleaves distinct drum targets in short time windows. For example, the transition from cymbal2 to tom2 occurs rapidly, and as a result, the policy stays near cymbal2 and never attempts to hit tom2. Likewise, cymbal1 hits are squeezed into very narrow gaps between hi-hat strikes, and the policy tends to prioritize the more frequent pattern, staying locked in on hi-hat while missing the majority of cymbal1 strikes. These examples reflect how the policies learn a preference for spatially efficient actions under tight timing constraints.

5.1 Ablation Analysis↩︎

To assess the individual contributions of our design choices, we perform an ablation study of the contact reward components and the observation, reporting mean and standard deviation of F1 scores across five runs in Table ¿tbl:tab:abl?. In the reward ablation, we retain proximity and regularization terms and compare three variants: (i) correct-strike reward only, (ii) correct plus wrong strikes, and (iii) the full dense contact reward (correct + wrong + missed) strikes. The results demonstrate that the full reward formulation is essential for achieving robust rhythmic accuracy, as omitting either wrong or missed strike terms degrades F1 across five songs.

We next evaluate the effect of removing key components from the policy’s input state during training. We find that the policies trained without the contact goals collapse to near-zero F1, indicating they lack the ability to anticipate upcoming drum strikes. We also examine the role of an explicit time phase signal by replacing the contact goals with a continuous, normalized phase variable [16] ranging between \(0-1\). Phase-conditioned policies likewise perform poorly, suggesting that coarse temporal context alone is insufficient to acquire drumming skills. The time phase variable informs the policy "how far through the segment" it is but not "which drum hit is next", highlighting the necessity of explicit goal encoding. Collectively, these results show that humanoid drumming requires both dense contact rewards and structured state representation that encode future rhythmic targets.

Finally, we evaluate the impact of removing temporal decomposition scheme by training policies end-to-end on full-length tracks rather than on fixed-length segments. While the final F1 scores remain comparable in both settings, we observe a substantial increase in wall-clock training time: policies trained without decomposition require approximately 8–9 hours to converge, compared to just 2–3 hours with temporal segmentation on the same hardware. Detailed results are reported in the supplementary material.

5.2 Multi-Song Policies↩︎

We also trained policies to play multiple songs with a single controller. A policy trained on six songs from Table 1 performed substantially worse on each song than specialist policies (Figure 5). Songs with higher rhythmic irregularity or faster tempos, such as Livin’ on a Prayer and Fire, exhibited the largest performance drops. In contrast, specialist policies achieved F1 scores near or above 0.9 on these tracks. Training on all eight songs further degraded performance across all songs, indicating that naive multi-song training introduces trade-offs between tasks. Combining multiple rhythmic patterns (e.g. irregular timing of Livin’ on a Prayer and Fire’s fast tempo) in one policy produced negative transfer [18], reducing task-level performance. Songs differ widely in drum count and rhythmic structure, forcing the policy to accommodate conflicting requirements. As a result, the policy often adopts conservative strategies, focusing on more frequent drum targets while neglecting others.

We also conducted a small listener study with 15 participants to evaluate the perceived musical qualities of the Robot Drummer. Participants rated the performances on timing consistency, expressiveness, naturalness, and overall enjoyment using a 5-point Likert scale. The robot achieved above-average ratings for timing (3.6/5) and expressiveness (3.4/5), lower scores for naturalness (2.5/5), and moderate overall enjoyment (3.1/5). Full study details are reported in the supplementary material.

6 Discussion↩︎

Our experiments demonstrate that specialist policies can achieve high rhythmic fidelity for humanoid drumming (F1 \(\ge\) 0.9 on many tracks), and accurately execute complex drum fills and rolls (see Figure 3). Beyond timing accuracy, the policies exhibit emergent spatial strategies: for example, the robot executed multiple cross-over strikes while performing Livin’ on a Prayer. Such maneuvers were not encoded in the reward; they arose as the policy searched for efficient ways to reach distant drums. Our analysis further indicates that spatial complexity (number of drums) and temporal variability (nPVI) are strongly correlated with performance degradation. Songs with many quick switches between drums caused the robot to favor the most frequent targets and miss others. In addition, training a single policy across multiple songs led to a consistent drop in F1 scores, highlighting the challenges of multi‑task RL for rhythmic motor control. Together, these findings suggest that effective drumming requires tightly coupling temporal context with spatial planning (how the robot hits one drum directly influences its ability to reach the next targets). Our dense contact rewards and future contact goals encoding implicitly accounted for this. Indeed, we observed policies dynamically reassign sticks based on the musical context. For instance, in Livin’ on a Prayer, the policy typically used the right stick for the snare (a spatially efficient choice) but switched to the left stick when a cymbal2 hit immediately followed the snare hit, ensuring the right hand remained available for the cymbal while still executing the snare strike on time.

6.1 Limitations↩︎

Our experiments are conducted in simulation and do not model several physical aspects central to real-world drumming, including sound production, stick rebound dynamics, and grip compliance. As a result, the learned policies optimize for contact timing and spatial coordination, but do not reason about acoustic outcomes. We hypothesize that these unmodeled effects can be learned from real data. This points to a broader and relatively unexplored research question: can robots be trained to control the sound they produce? Optimizing for acoustic targets would force policies to implicitly control impact dynamics, closing the loop between motion and sound.

Transferring the learned policies to a physical system will require (i) a compliant hand–stick interface to better handle contact dynamics, and (ii) a residual control component that applies corrective action adjustments to compensate for unmodeled dynamics. A practical transfer pathway is to use an electronic drum kit that outputs MIDI events, enabling direct comparison between executed and target hits to fine-tune timing and strike intensity, thereby compensating for acoustic mismatch. While full acoustic modeling remains challenging, a sound model learned from real-world data could be integrated into simulation to train sound-aware policies.

7 Conclusion↩︎

In this work, we have introduced Robot Drummer, a general RL-based framework that enables a humanoid to perform complex drumming patterns. By modeling drumming as a rhythmic contact chain and leveraging temporal segmentation with dense contact rewards, our approach scales to long musical sequences and achieves high F1 scores on many songs. The learned policies exhibit emergent strategies, such as cross-over strikes and dynamic stick assignment, without explicit encoding. Future work will focus on sim-to-real deployment and adapting policies to match acoustic targets, advancing sound-aware control that could enable new forms of collaboration between robots and human musicians.

Acknowledgments↩︎

The authors thank Mirko Nava for insightful discussions and Marco Maccarini for setting up the training infrastructure, both at IDSIA.

APPENDIX↩︎

7.1 Implementation Details↩︎

7.1.1 Rewards↩︎

Table 2 lists both the contact reward terms and regularization terms together with the weights used in all experiments.

7.1.2 PD Controller↩︎

Table 3 lists the stiffness (\(K_p\)) and damping (\(K_d\)) values used in the PD controller for all joints.

7.1.3 Policy Training and Evaluation Protocol↩︎

We use a 3-layer MLP architecture for both the actor and critic networks, with hidden sizes of (128, 64, 32). For each song or training condition, policies are learned across 5 random seeds over 1500 PPO updates. Experiments are conducted on a server with Nvidia RTX 4090 GPUs, running 8192 parallel environments in Isaac Gym. After training, each policy is evaluated over 20 independent rollouts, and performance is measured using the mean F1 score between actual and target drum strikes across all timesteps. The final performance metric is the average F1 score and standard deviation across the 5 seeds.

7.2 Additional Results↩︎

7.2.1 Song-level Performance Analysis↩︎

Figure 6 presents song-level performance analysis across all specialist policies. (a) shows absolute Spearman rank correlations between individual song features and per-song F\(_1\) score, while (b–d) show the relationship between F\(_1\) and selected features using scatter plots with linear trendlines. Together, these plots summarize how performance varies across songs as a function of rhythmic and spatial attributes of the target drum patterns.

7.2.2 Temporal Decomposition Ablation↩︎

Table 4 reports per-song F\(_1\) scores (mean \(\pm\) standard deviation over five seeds) obtained with and without temporal decomposition. Temporal decomposition trains fixed-length segments concurrently rather than end-to-end on full tracks. Although the policies achieve similar F\(_1\) scores in both cases, wall-clock time is substantially reduced for each run with temporal decomposition (from 8–9 hours to 2–3 hours).

7.2.3 Listener Study↩︎

The listener study evaluates perceptual aspects of the drumming pattens performed by Robot Drummer using subjective ratings collected from human participants. Fifteen participants each watched and listened to three performance videos of specialist policies. After watching the videos, participants answered a series of questions, rating the robot’s drumming on a 1–5 Likert scale (1 = not at all; 5 = very much) for the following aspects:

• Timing Consistency - How consistent and well-timed was the rhythm of the drumming?

• Expressiveness - To what extent did the robot convey a sense of musical feel?

• Naturalness - How natural or human‑like did the robot’s drumming sounded?

• Enjoyment - Overall enjoyment of the performances.

Across all responses, we observed the following results:

• Timing Consistency: \(3.69~\pm~0.63\)

• Expressiveness: \(3.38~\pm~0.96\)

• Naturalness: \(2.46~\pm~0.78\)

• Enjoyment: \(3.08~\pm~1.60\)

These results show that listeners generally perceived the robot’s timing and musical feel in the upper mid-range, with lower ratings for human-like naturalness and mixed enjoyment responses reflected by greater variance.

References↩︎