1 Introduction↩︎

In human-in-the-loop systems, feedback describes the sensory elements designed to inform users about their interactions with the system. For example, a clicking sound on button press encodes the information that the system has registered the user’s input. Games are commonly designed with an abundance of responsive elements that encode important information, and they are used to not only support an understanding of the game’s state but also maximize players’ perception of rewards and steer performance. Within game interaction loops, feedback typically consists of the visual, auditory, and haptic information displayed to players, influencing their understanding and enjoyment of the game experience [1]. The merit of any information-display mapping can be considered from players’ ability to utilize them to improve their performance and in terms of correspondence – their perception of how well the game’s feedback reflects their performance. The quality of an information-display mapping depends on how well players can use it to improve their performance and their perception of how accurately the game’s feedback reflects their actions. For example, in Fruit Ninja [2], the speed of players’ strokes corresponds to larger and louder swings of their swords, creating a rewarding experience. However, it is unclear whether such feedback is merely an audio-visual embellishment that varies feedback to reduce monotony or whether communicating measurements in this indirect way can affect players’ performance positively or negatively. With the right feedback, game designers could guide Fruit Ninja players to make quicker, more expansive movements, enhancing their chances of successfully slicing multiple fruits. However, players might instead prioritize speed over accuracy, either in pursuit of satisfying feedback or because they interpret the feedback as endorsing their actions and thereby impair their own performance.

This highlights a knowledge gap for game designers, as the vast range of possible information-display mappings remains poorly understood in terms of their effects on player performance and behavior. To date, questions relating to information-display mappings have been addressed by relying on play-testing, gut feeling, or copying known practices. Game scholarship has contributed with design lenses to aid feedback design, like Hick’s “juicy” design framework [3], Deterding’s feedback design lenses [4], and Schell’s lens of feedback, juiciness, channels, and information [1]. However, although juicy feedback can demonstrably benefit task performance in non-game scenarios [5], game studies of juicy feedback suggested that it may not improve performance and may even worsen outcomes compared to minimal non-juicy feedback [6], [7]. Game designers and researchers are missing fundamental knowledge of the extent players can use information-display mappings to improve performance. Including the answers to questions like: How much do users read into the feedback received and use it to improve their performance? When will feedback induce a cognitive cost that affects performance negatively?

To answer such questions, we designed a study that evaluated players’ performance and experience with three performance metrics implemented into feedback happening at different points in time (from immediate feedback to more periodic feedback). Our study in Section 4 consists of the following contributions:

1. We unpack the design space of dynamic feedback by combining Wensveen’s couplings of time and dynamic with concepts from motor learning and game scholarship (Section 2.1).

2. We conceptualize a self-regulation model for pointing games, and draw upon information processing theory to propose how to measure key experiential dimensions affected by feedback design choices (Section 3).

3. We designed a low-level action-feedback game loop to study users’ ability to improve performance while receiving dynamic feedback based on three distinct metrics (Section 4).

4. We provide empirical evidence linking users’ performance self-regulation to their overall performance. Our qualitative results show factors affecting dynamic feedback’s legibility (Section 5).

Our systematic feedback design approach for pointing tasks (Section 2.1) can optimize feedback in a wide range of applied pointing-based games, including motion-controlled games, exergames, and rehabilitation games. Based on our quantitative analysis of tracked participant movements, self-report measurements, and semi-structured interviews with participants, we provide design recommendations for pointing tasks employing audiovisual real-time feedback. With the growing interest in designing video games for complex scenarios from rehabilitation and sport training to the use of artificial intelligence (AI), it becomes increasingly important to understand how feedback guides players’ game behaviour. Our study reveals valuable insights into how low and high-performing players derive benefits from dynamic, performance metric-driven feedback.

2 Background↩︎

In HCI, feedback constitutes a fundamental concept in interaction and product design, complementing notions like feedforward, signifiers, and affordance. Within game scholarship feedback closely relates to design lenses like game feel [8] and game-specific terminology like juiciness [1], [4] – a reference to non-functional feedback designed to evoke a visceral feeling through an abundance of (typically) visual effects [1], [6].

In motor learning, feedback is studied as a means to regulate user behavior. Feedback can, for example, improve performance or technique in tasks that involve the human motor system [5]. The field views feedback as knowledge and has developed sub-concepts specific to motor learning strategies, like knowledge of performance (feedback describing the technique or nature of the movement) and knowledge of results (feedback reflecting the interaction outcome) [9]. Compared to feedback terminology in educational contexts, knowledge of results is closely aligned with outcome feedback, while knowledge of performance aligns with process feedback. However, contrary to motor learning, educational contexts often seek to achieve deeper learning and higher reflection, focusing on cognitively demanding mental tasks (e.g. problem-solving, selection), that get executed through motor actions [10]. Functional feedback can also take the form of cues, which intend to provide users information and dissapear once they are no longer needed [11]. Like feedback notions in motor learning, cues can be provided at different levels of granularity from continuous (e.g. a color changing as a person moves [12]) to discrete (a color changing once the person moved correctly). Previous work have for example studied how cues can help players understand if they performed correct body movements [12], and explored how to use cues to direct players’ gaze [13].

The present study focuses on the motor learning goal of optimizing users’ motor skills to perform continuous movements (like tracking an object or steering a car). These movements are characterized by relatively low cognitive involvement, with the nature of the movement itself being a key determinant of success [9]. Motor learning refers to feedback dynamically driven by human performance as biofeedback [14], and is introduced into motion-controlled systems as augmented feedback, supplementing user’s inherent bodily feedback while they perform, to benefit skill learning and performance [15]. Games additionally employ dynamically driven feedback as a means to invoke curiosity, create sensual feedback, a more reactive environment, more variation, reward, and juiciness [1], [3], [4]. Fruit Ninja[2], Osu![16], and Beat Saber[17] continuously track players’ input devices and use it to drive audiovisual effects dynamically as users play. In Osu!, a green trail of particles follows the player’s cursor, which grows when players move the cursor at a higher speed. However, although the Osu! trail rewards high cursor speed, the rewarded behavior does not align with what is needed to obtain better performance or technique. These types of feedback are often less explicit than cues, providing information through ambient audiovisual effects driven by the characteristics of user actions. Game designers may consider feedback dynamics through, for example,  [1]‘s lens of channels and dimensions, which demonstrates dividing games’ interface into channels for game information [1]. However, as [1] admits: “This is usually done partly by instinct, partly by experience, and mostly by trial and error [..].” [1].

2.1 Design Space of Feedback↩︎

Although rarely used in game design contexts, [18]’s product design framework [18] offers a comprehensive starting point to unpack dimensions of feedback, illustrated in Fig. 1. [18]’s framework explain feedback design through six couplings: Modality (e.g. audio/visual/haptic sensory channels), Expression (e.g. color, size, form factor), Location (e.g. spatial placement), Direction (e.g. alignment to the action’s direction), Time (e.g. feedback’s temporal occurence), and Dynamics (e.g. how information is mapped to appearance).

Choice, combination and congruency of modality have been extensively studied within HCI and motor learning [5], [19]–[23]. For example, [19] demonstrated the performance benefits of congruent multi-modal feedback (repeating information across modalities in synchrony) [19]. Motor learning has studied forms of expression as positive/negative framing [9], natural or abstract representations (e.g. video feedback vs. numerical gauge), and granularity of representation (e.g. 5 shades of green versus a continuum of colour) [5]. Expression also represent game-centric notions like game feel [8], aesthetic appeal, and abundance (juiciness) [6]. The location of feedback is commonly manipulated by games in superimposed heads-up displays [24]. It is for example common to show an avatar’s health bar in the display corner, instead of next to the avatar itself. The feedback’s direction refers to the feedback’s way of changing compared to the direction of the user’s action [18]. For example, higher movement speed might yield better feedback (alignment), whereas longer movement time yields worse feedback (inverse alignment).

The time coupling represents the alignment between user actions and the feedback [18]. Time has multiple facets, which we illustrate as four sub-dimensions in Fig. 1: Duration, Coincidence, Frequency and Composition. Temporal duration refers to the amount of time the feedback is shown (e.g. in seconds or milliseconds). Temporal coincidence (or timing [25]) refers to how feedback coincides with user interaction from synchronous (immediate) feedback to delayed feedback. In games, synchronous feedback in games ensures that players know what they need to know at a given moment [1], whereas delayed feedback (e.g. latency) can disrupt players’ flow [4], [23]. Temporal frequency represents how often users receive feedback in response to their actions. Terminal feedback (single discrete feedback instances at a frequency) and concurrent feedback (feedback shown continuously during the movement) are frequently compared in motor learning [5]. Temporal composition represents the alignment of feedback to users’ tasks and actions in games. [26]’s core task framework, for example, provides a process to deconstruct gameplay into a hierarchy composed of tasks and actions. Composing feedback is the process of choosing when and where to provide it in the task hierarchy – at the end of each task, each action, or continuously during subconscious operation [26]. For example, Fruit Ninja [2] yields operation-level feedback (larger cutting trail) while the user cuts the fruit. Once the fruit is cut, the fruit becomes halved creating action-level feedback (fruit stains). The task ends when all fruits are cut, creating task-level feedback such as high score (number of fruits cut, and best fruit combo).

2.2 Dynamic Feedback↩︎

[18]‘s last coupling, Dynamics, refers to how the action maps to the system’s reaction. [18] exemplifies such a mapping process with their scissor example: “The speed of the cutting action determines the speed of the incision being made.” [18]. In digital games, dynamics refer to how players’ performance is mapped to feedback by undergoing an encoding process. Performance can for example map directly to a change in color, or be augmented through a logarithmic conversion. Such encodings are also known as information-display mappings within HCI, although they are not specific to visual displays. Within motor learning, dynamic feedback serves to improve user skill and learning [5]. Design considerations include how the underlying performance measurements are chosen (source performance metric) [15], the range of information used from the user interaction [26], and how it is technically processed (e.g. numerical conversion, sampling method). We expand the dynamic coupling into four sub-dimensions in Fig. 1. The first dimension, Information Scope refers to the context of information utilized in the dynamic feedback. [18]’s example of naturally mapped dynamic feedback only utilized information from the action instigating it [18], but it could just as well utilize information from all so far performed actions or from a selected range of fine-granular operations performed within the finished action [26]. Source Metric refers to the measured information source, such as the speed of the cutting action in [18]’s scissor cutting example. Choosing a source performance metric is often task-dependent [15] – for example, performance metric for real-time feedback in running disciplines were chosen by comparing the strength of evidence with running economy and injury [21]. Pointing tasks commonly employ movement time and target distance as performance metrics along with target size to calculate throughput from Fitts Law [27]. Sampling refers to how information from a given source metric is sampled and stored prior to encoding, including the available measurement unit (sampling reference), and information granularity (sampling frequency and type). Finally, encoding refers to how the information is eventually translated into a specific type of feedback – for example matched to a change in color.

Dynamic feedback has been indirectly studied in the context of exergames (games designed for physical exertion) and for novel input methods for games. For example, [28] studied how gaze, temperature, or galvanic skin response could be used as physiological source metric to control game elements, providing players dynamic feedback [28]. Their study focused on both direct and indirect continuous control via physiological input, influencing active elements (e.g. enemy targets) and passive elements (e.g. weather conditions) [28]. In both cases, feedback was concurrent with interactions and based on operation-level information, where feedback was synchronized in real-time to changes in users’ physiological state. Data underwent downsampling, smoothing, and normalization as part of encoding of the physiological data [28]. In a similar vein, [29] explored dynamic feedback in a virtual reality (VR) run and jump exergame by amplifying players’ inherent jump height [29], thereby augmenting their inherent feedback. They measured jump height from the ground floor (source metric) for each individual jump action (information scope) and used it as multiplication factor for encoding the resulting feedback. Both studies found experiential benefits of incorporating dynamic, be it as a dramatic device [28], or to increase intrinsic motivation and perceived competence [29]. However, it remains unclear what processes are involved when players regulate their performance with dynamic feedback, and what experiential effects design choices like choice of performance metric and timing have, particularly for more common tasks like pointing.

2.3 Research Gap↩︎

Players may find dynamic feedback rewarding in games, but it is unclear to designers when to deploy feedback schemes that help players improve and where they can exercise artistic freedom without causing performance interference. Previous work studied effects of juicy feedback [6], [30], dynamic feedback based on physiological input [28] and in motor learning contexts [5], [21], but to create a better foundation for the dynamic feedback design process, the relationship between feedback, performance metric, performance, and users’ sense of correspondence needs to be clarified:

• How does metric-based dynamic feedback improve or penalize people’s performance? For example, what improvements or penalties occur when players utilize dynamic feedback of their own peak speed during pointing?

• What cognitive factors exist in players’ self-regulation loop and do players feel dynamic feedback affects them?

To address this gap, we 1) conceptualized a model of performance regulation in a simple pointing game, and 2) utilized the pointing game to systematically compare changes in players’ performance and perception of dynamic feedback, driven by three performance metrics, promoting three distinct behaviors: lower completion time, higher peak speed, or straighter movement. By providing players’ knowledge and first-hand experience of how each metric worked, we hoped to create optimal conditions for measuring how players’ self-regulated with dynamic feedback.

3 Conceptual Model↩︎

Understanding dynamic feedback’s effect upon performance self-regulation requires knowledge of the cognitive processes involved in the underlying interaction loop between user and system. Human-computer interaction has commonly been conceptualized based on [31]‘s Stages of Action in which feedback partially reveals the system’s state to the user who interprets it to determine their next course of action [31]. Games similarly consist of loops, where a simple task is repeated over and over – also called’low-level action-feedback loop’ [30] or a mastery feedback loop [32]. Unlike progression-based game loops in which skills build on each other to form complex skill chains (e.g.  [33]), low-level action-feedback loops are characterized by high repetition and focus on enveloping players in success-dependent feedback [30], which reward players the more they master the task [32].

An example of a simple game loop, is the target-to-target pointing task, which consists of targets appearing one after another, resembling immediate-response tasks from information processing theory in cognitive psychology [34]. The immediate-response task is a subtype of the speeded response task, which aim to “produce a simple response dependent in some well-specified direct way upon the presentation of a stimulus.” [34] In target-to-target pointing tasks, the simple response consists of a quick movement from one target to the next. Looped tasks are said to be data-dependent – the output of the previous action becomes the input of the next. The loop between user and a system is also known as micro-level monitoring in self-regulated learning, where users use self-observation and system feedback to determine whether they are progressing appropriately to regulate their effort and affect (e.g. frustration or motivation) [35].

[34] categorizes immediate-response tasks into eight stages: perception (sensing the environment including feedback), encoding (parsing the information), attending (focusing on relevant details), comprehending (analyzing the information), setting the task (deciding what to do), intending it (determining when to act), decoding it (converting to commands), and finally movement (executing the action) [34]. If a stage fails (e.g. failing to perceive the feedback), it hinders the creation of a functional pathway from perception to movement in [34]’s model. Similarly in game contexts, [30] emphasized that feedback must be both legible and differentiated for fun and motivation to arise: “[..] even at low-level action-feedback loops, motivation and enjoyment arise from reducing uncertainty over action success, which depends on legible* and differentiated amplified feedback.“ [30] Legibility refers to the causal action-feedback link* (knowing I created an outcome) while differentiation expresses how well an action was executed.

We conceptualized a self-regulation model for pointing tasks (Fig. [fig:conceptualspace]), which encompass the system, world and mind, based upon [31]’s Stages of Action and [34]’s scheme for immediate-response tasks. We drew upon existing scales to develop and iterated upon the measures through pilot tests in parallel with our study’s apparatus and procedure, further detailed in Supplementary Material 1.

3.0.1 System Stages 1-3↩︎

Represents the underlying algorithms that map user performance to feedback. User performance is measured in real-time by obtaining data such as movement position at a given sampling rate from the VR headset and sensors in the sensing stage (1). A performance metric such as speed is derived from the sensor data, which is then normalized to an abstract form, like percentage in the metric stage (2). Finally, the abstract representation is translated into feedback with specific intensity and occurrence in time in the feedback stage (3).

3.0.2 Legibility Stages 4-5↩︎

Based upon [30]‘s notion of legibility, perceptual legibility, contrasting legibility, and analytical legibility are sub-concepts, derived from perception, encoding, attention and comprehension stages in immediate-response tasks. Users’ notice the feedback in the perception stage (4), influenced by design features in the system that help users see presented information (the sensory affordances) [36]. The contrast stage (5a) to follow, captures users’ ability to recognize changes to the feedback’s appearance over time. Then in the comprehension stage (5b), users mentally connect their action to the system’s reaction to make causal assessment, which is influenced by design features that help users facilitate knowledge and thought (cognitive affordances) [36]. Comprehension is similar to PXI’s Progress Feedback sub-measure “I could easily assess how I was performing in the game” and to Feedback Ambiguity in [6]’s framework for juicy game design (“Can information be connected to actions and only interpreted in one way?”) [3].

3.0.3 Preference Stage 6↩︎

Where users’ expectations and subjective feedback preferences are superimposed onto the evaluation process [30]. The quantity stage (6a) captures how much feedback players feel they receive – if feedback is too excessive, it can unintentionally reduce enjoyment and playtime, highlighting the importance of appropriate feedback amplification [30]. Perceptual appeal (6b) refers to ‘consummatory likeness’ [30] – the interest and enjoyment players derive from the feedback.

3.0.4 Internal Perception Stage 7↩︎

Refers to the internal information users store in memory of the action they performed – users’ sense of performance (7a). Motor learning conceptualized that as inherent feedback – the memory of how the movement felt, sounded and looked, which is accessible to people even before their actions are completed, enabling self-detection of error (or subjective reinforcement [9]). We introduced the correspondence stage (7b) to record how users sensed their internal performance matched externally perceived feedback. It relates to unambiguous feedback from the Flow State Scale [37] (“I had a good idea while I was performing about how well I was doing.”).

3.0.5 Performance Intention (8-10)↩︎

Represents the closest stages to motor system actuation and extends the intend and move stages in immediate-response tasks. It consists of a performance intention, which is decoded into a performance response, which is affected by the extent which feedback was encouraging or distracting. Encouragement and distraction are somewhat inverse to the positive effectance measure (users’ “perceived influence on the game world”) [38], which measure how effectively motor work yielded more feedback – by contrast, distraction represents the perceived cost of feedback, while encouragement represents the perceived gain of feedback.

4 Study↩︎

We designed a within-participant experiment to study 24 voluntary participants who played a virtual reality (VR) pointing game for 30-40 minutes with their right hand. We utilized pointing as a platform to study how well participants could adapt their performance, following three different source metrics as main study’s main variable – the primary component to create dynamic feedback in Wensveen’s design space in Fig. 1. To visualize the metrics, we created three feedback designs. We used three different compositions (Time coupling in Fig. 1) as the design anchor to make each feedback appear distinct and let the composition become an anchor for determining the appropriate expression and location. Performance and all cognitive factors from the self-regulation model in Fig. [fig:conceptualspace] were utilized as dependent variables, to measure players penalties or improvements, including perception, contrast, comprehension, quantity, appeal, correspondence, encouragement and interference.

Using an iterative feedback design process, we aimed to effectively unveil a broad spectrum of potential penalties and improvements in the conceptual model’s performance, legibility, preference, and performance stages in Fig. 3. The study employed differently timed dynamic feedback that provided different levels of exposure to the underlying metric. They leveraged metaphors from well known phenomenons – pointer trails used in most desktop accessibility settings to increase cursor visibility or recognizable and affirmative checkmark symbols. The feedback followed a fixed order, from high exposure to low exposure, ensuring that players’ had high familiarity with each metric, even with low access to feedback. The study made measurements across all three sessions to identify overall improvements or costs to performance.

The study was approved by the ethical review board of the lead author’s institution (REB #2023-505-00051). The study examined how dynamic feedback influenced players’ performance and experience using three metrics: distance, time, and peak velocity.

4.1 Apparatus: VR Pointing Game↩︎

The study’s pointing task consisted of playing a Whack-A-Mole VR game [39] on a HTC Vive head-mounted display while sitting down, shown in Fig. 3. Whack-A-Mole VR [40] was chosen because it was designed for continuous performance data collection for rehabilitation setting at high granularity; it provided calibration procedures, a simple distraction-free VR environment, and configurable activation patterns to make participant data comparable [39].

In Whack-A-Mole, players controlled a blue crosshair to hit round green targets on a 2D horizontal \(5m \times 9m\) grid shown in front of them in VR. Targets were activated by hovering over a target for 400ms – no button pressing was necessary to avoid the Heisenberg effect (parasitic motion from the motion-tracked controller when pressing its button down) [41]. Every target was active for 5 seconds and appeared immediately after each other (target-to-target). The crosshair moved in response to horizontal and vertical controller movements, unaffected by changes to controller angle – players pointed by moving their controller around within a small invisible \(35cm \times 25cm\) motor space, calibrated to be within reach from players’ seated position (e.g. as illustrated in [42]). We incorporated three performance metrics (completion time, peak speed, and travel distance) which we tested with distinctly different feedback in three sessions:

1. Trail. Continuous operation-level feedback which consisted of a trail, simulating a natural phenomenon observable on fast-moving objects in the real world, to maintain a natural expression and location coupling to the user’s cursor movements. The blue trail matched the cursor’s color, and the trail’s length and opacity were moderated by performance (from 0 to 9m length and 0-100% opacity). The trail emitted a continuous sound, whose pitch from 195hz to 320hz dynamically varied with users’ performance. Users received continuous haptic vibration in short 6ms pulses with variable amplitude, moderated by performance.

2. Checkmark. The player hovers over each green target to hit them and receives a green checkmark when completing an action. The green circular checkmark is a well-known acknowledgement symbol used in game and non-game contexts and was co-located with the target to maintain a natural location coupling. The checkmarks scaled 100-150% of their baseline size and changed color from pale to saturated green, moderated by user performance. An auditory “pling” sound changed to brighter pitches between 514 Hz and 817Hz continuously across 8 semitones. Players received a short 50ms haptic vibration with variable amplitude.

3. Heatmap. After all targets were hit, a heatmap appeared on the wall with a 3-second introductory animation, where targets each emit a checkmark by hitting order. With each checkmark, the player’s controller vibrated shortly for 100ms and they heard a brief “pling” sound at variable pitch forming a 3-second melody, which represented player performance. To give players ample time to see the heatmap, it was visible for 10 seconds.

All feedback in S1-S3 was multimodal, via sound from headphones, visuals in VR, and haptic vibration feedback in the hand controller. Expression-wise, the feedback changed size, color saturation, vibration intensity, vibration frequency, audio pitch, and audio volume. All feedback was located in proximity to the element of interaction and featured similar directionality – receiving more feedback always meant objectively performing better.

4.1.1 Combining Metric and Feedback↩︎

We designed the three performance metrics completion time, peak speed and travel distance shown in Fig. 3, bottom. The metrics were not designed as "fair" or ideal models of human pointing performance, for example as derived from Fitts Law [27] and studied by kinematic literature [43], [44]). Instead, each metric focused on an isolated aspect of human performance, making it possible to measure whether or not participants adapted their performance to the metric. The metrics’s information scope matched the feedback’s temporal composition - for example, action feedback only reflected performance information from the specific action. The metrics were measured as single samples of data with at 60Hz sampling frequency, and initially tested in textual form (e.g. “80%”, “2.3m”, “4.1s”) to ensure the correctness of measured performance. The metrics were implemented for each feedback type, like described in Fig. 4.

The time metric measured the time passed in seconds from action start to either the current cursor position (crosshair trail) or to the end of the action (checkmarks) – it rewarded players when they optimized their time usage and punished actions with high time usage. The peak velocity metric measured the current velocity at the cursor position (crosshair trail) or the highest velocity that happened during the action (checkmarks) – by design rewarding swift movements, and punishing slow movements. The distance metric measured the accumulated distance (measured in nearest whole cm) divided by the shortest Euclidean distance to either the current cursor position (crosshair trail) or to the end of the action (checkmarks) – by design it rewarded straighter movements (trajectory straightness) while punishing behavior that led to more accumulated travel like hand shaking or overshooting.

4.1.2 Adaptive Feedback System↩︎

To map metric to feedback, we implemented a feedback system, which measured and assessed users’ game performance in real-time, relative to observations of minimum and maximum performance peaks. the feedback system was calibrated to account for individual player performances, by determining floor and ceiling through averaging the five most recent worst and best performances. Using an average instead of a single observation or an absolute value increased the systems’ robustness against coincidental minimum/maximum performance outliers stemming from noise, learning effects or fatigue. Additionally, 1% of participants’ peak performance ranges were continuously subtracted from the minimum and maximum peaks every second, ensuring that the system continually obtained new minimum and maximum peak values over time. When the feedback system had a low number of observations (i.e. \(<\) 3 hits) it emphasized rewarding users by design, while capably providing more nuanced and accurate rewards when it had more observations to estimate user performance from. To provide a fair comparison between conditions, the feedback system reset with every condition. During play the feedback system’s floor, ceiling and assessments were tracked, and to help ensure they matched users’ performance.

4.2 Study Design↩︎

Following the objective of understanding how people’s adapted to three performance metrics, we designed a study constituting three counterbalanced sessions shown in in Fig. 4. Each counterbalanced session followed a Latin square, so each participant had a different performance metric order (e.g. Time \(\rightarrow\) Speed \(\rightarrow\) Distance). We captured performance measurements from participant actions, as they hit each target on the Whack-A-Mole wall. We used perceptually random performance balanced activation patterns, that maintained high repeatability across conditions (detailed in Supplementary Material 2). Completing the experiment required each participant to move their limb 44.9 meters Euclidean distance, verified through data logging.

We chose a fixed order for the feedback types, that maximized participants’ exposure to the underlying source metrics, by letting everyone experience continuous operation-level feedback first (most exposure). This way, participants were given more optimal conditions to self-regulate during short exposure to the non-continuous feedback that followed (end-of-action checkmarks, and end-of-task heatmap). This desirable learning effect was became the cornerstone of the study’s final within-subject design which economically balanced the total number of playthroughs (9 per participant) with the overall study length (30-40 minutes).

Numerous pilot tests refined the study’s apparatus (feedback system, types, metrics), and experimental design (conditions, control variables), to ensure consistent performance metric implementation across feedback types with adaption to all ranges of human performance. The feedback’s auditory, visual and haptic design was improved by obtaining impressions from pilot test subjects, consisting of game design students and two persons working in the game industry.

4.3 Participants↩︎

Fifteen male and nine female participants between 22-41 years old (\(M\)=\(27.2\), \(SD\)=\(3.9\)) volunteered their time. Almost all considered themselves experienced at playing video games (22/24), and the majority (19/24) had previous experience with VR. Half of the participants (14/24) wore glasses and were allowed to take them off if they could clearly read the instruction text without them. None of the participants reported having previously played Whack-A-Mole VR before.

4.4 Procedure↩︎

At the start of the study, participants filled out their consent form, their gender, and age. Participants were informed that we measured how they experienced variations of a virtual reality game and that their personal data would be anonymously processed as per the ethical and data law requirements. The facilitator asked participants verbally about their previous VR and gaming experience and expectations of the experiment. The facilitator showed them images of Whack-A-Mole VR and verbally instructed how to play, emphasizing that they were to perform their best. Participants were introduced to the different feedback and performance metrics, and could ask questions if in doubt. They then donned a VR headset and were calibrated to remain seated in the virtual scene’s center. If participants wore glasses, they either wore them inside the headset or took them off based on preference and whether the instruction text inside the HMD was legible. A short warmup with binary audio-visual feedback confirming target hits lasted for 1 minute (green target disappears with an unvarying “pling” sound). They then played the first session with trail feedback reacting to participants’ speed, distance, or time. Before each condition, the facilitator announced the upcoming performance metric. They had to hit 20 targets with each metric and were afterwards asked to rate their sense of correspondence (Fig. [fig:conceptualspace] ,7b) as shown in Supplementary Material 1. When the third and final metric was rated for trail feedback, they rated the feedback’s quantity (Fig. [fig:conceptualspace], 6a) and appeal (6b). They then proceeded to three conditions with checkmark-based and, finally, heatmap-based feedback where the procedure was repeated. After completing all conditions, the participants filled out the post-study questionnaire (Supplementary Material 1) regarding their perception (Fig. [fig:conceptualspace], 4), Contrast (5), encouragement (9a), and interference (9b). Finally, the participants were briefly interviewed about their experience and reported any felt discomfort, e.g. (cyber-)sickness or fatigue.

4.5 Data Analysis↩︎

Table 1 summarizes all quantitative-, survey-, and control measures. Participant movements were continuously recorded with 68Hz mean sampling frequency (30-95Hz, \(SD\)=\(0.008\)). Data was imported into R Studio for analysis. Movements from each condition were split into individual actions based on target hits, which marked one action’s end and the next action’s start. We interpolated samples to time-equidistant values at 100Hz (10ms per sample) and calculated speed and position at each point and smoothed speed according to a degree 2 polynomial, following [45]‘s suggested approach [45]. We calculated Fitts’ ID to quantify the challenge of each pointing task independently from target-to-target distance and target width. Our quantitative performance measures in Table 1 were inspired by [45]. Movement time, peak speed, and straightness map directly to time, peak speed and distance metrics, which we visualized as bar charts following [45]. Quantitative measurements were tested for normality using a Shapiro-Wilk test and for sphericity using a Bartlett’s test. Except for Throughput, all quantitative measurements were transformed prior to ANOVA analysis to follow a normal distribution. The appropriate transformation was determined through Box-Cox analysis, maximizing the log-likelihood function (See Supplementary Material 2). The quantitative measurements were analyzed using 3x3 repeated measures ANOVA with a Huynh-Feldt correction applied to p-values when sphericity assumptions were violated. Bonferroni correction with \(\alpha\)=\(0.05\) was applied when making pairwise comparisons. Effect sizes were calculated using generalized eta square for RM-ANOVA and Kendall’s W for Friedman tests. Learning effects were estimated using visual trend line analysis and assessed within each feedback block, to estimate its effect separately from the fixed feedback ordering. Self-report measurements did not follow normality and were analyzed using a non-parametric Friedman test.

Table 2 lists all experiential measures, which were based upon the conceptual model in Section 3. Likert scale measures were visualized as violin plots, showing mean values, distribution and data points per participant. Raw performance data points were aggregated to means per participant, and Likert scale data points reflected each participant’s rating. Qualitative data was collected via note-taking and was analyzed thematically using an open coding analysis approach [46]. We categorized responses into themes inductively without external review, and compared them to our own noted observations of participant reactions.

5 Results↩︎

The participants (N=24) took on average 16 min and 27 s (\(SD\)=\(3.13 min\)) to hit 190 targets in total by moving their right arm around 50.5 m on average (5.6 m above required Euclidean distance). Table 3 lists averages and standard deviation of measurements, categorized by metric and averaged across three sessions. We report our quantitative results with means (\(M\)), standard deviation (\(SD\)), effect size (\(\eta^2\)) p-values (\(p\)), inter-quartile range (IQR), correlation results with Pearson correlation coefficient (\(r\)), and survey results with median (\(Med\)) and interquartile range (\(IQR\)). Performance improvements refers to any change above 0%. .

5.1 RQ1: Self-regulation with Performance Metrics↩︎

Following our instruction while receiving time-, peak speed, and distance-based dynamic feedback, participants changed their performance in terms of movement time and straightness. It varied between participants whether their performance improved or worsened from the dynamic feedback – the majority of participants made straighter movements by 0-5% (15/24, \(M\)=\(1\%\), \(SD\)=\(1.2\%\)) and achieved higher peak speeds by 0-41% (14/24, \(M\)=\(12\%\), \(SD\)=\(9.7\%\)), while less than half of participants improved their completion time by 0-6% (9/24, \(M\)=\(3\%\), \(SD\)=\(2\%\)). The majority of players (21/24) improved their performance to one of the three metrics, except for three participants, whose performance with the metric worsened compared to their overall average. Performance differences shown in Fig. 6 tested with repeated measures ANOVA revealed significant changes between distance and peak speed conditions in terms of movement time (\(p\)=\(0.013\), \(\eta^2\)=\(0.013\)), and straightness (\(p\)=\(0.040\), \(\eta^2\)=\(0.010\)), and peak speed (\(p\)=\(0.021\), \(\eta^2\)=\(0.014\)). However, Bonferroni post hoc tests of peak speed did not reveal significantly different pairs. Players’ overall average improvement fluctuated between -5% and +11% on average (\(M\)=\(2\%\), \(SD\)=\(4\%\)).

Linear regression across each counterbalanced session suggested that people learned over time to lower their mean completion time by 12% (\(\beta\)=\(-12.25\), intercept=\(1335.98\)), increase their mean peak speed by 4% (\(\beta\)=\(0.04\), intercept=\(0.81\)), but only marginally improved overall mean straightness by 0.02% (\(\beta\)=\(0.002\), intercept=\(0.831\)). We normalized measures of completion time, straightness and peak speed and summed them up to form an overall performance measure. Players’ overall performance varied, approximating a normal distribution, where best performers performed 20% better than the mean and 40% better than the worst observed performance. People’s overall ability to adapt depended weakly on their overall performance, according to a spearman correlation test bordering significance (r = 0.23, p < 0.04797). We checked whether differences in participant performance followed differences in their ratings of correspondence. The Spearman correlation test showed a low correlation to action arm travel (\(r\)=\(0.21\), \(p\)=\(0.07\)) and straightness (\(r=\) \(-0.22\), \(p\)=\(0.07\)) and no correlation to other performance variables (\(r\)<\(0.15\), \(p\)>\(0.05\)). Participants rated correspondence of time-based metrics (\(Med\)=\(5\), \(IQR\)=\(3\)) significantly higher than distance-based metrics (\(Med\)=\(4\), \(IQR\)=\(3.25\)), according to a Conover’s F post hoc test. They did not rate correspondence with speed-based metrics (\(Med\)=\(4\), \(IQR\)=\(3\)) significantly different from time-based metrics (\(p\)=\(0.063\)) or distance-based metrics (\(p\)=\(0.445\)). Correspondence positively correlated to most experiential measurements including how encouraging participants felt at moderate level (\(r\)=\(0.51\)-\(0.72\), \(p\)<\(0.001\)), where the highest observed correlation was to ratings of feedback appeal (\(r\)=\(0.72\), \(p\)<\(0.001\)). Correspondence correlated weakly to ratings of distraction (\(r\)=\(0.26\), \(p\)=\(0.03\)).

When questioned, no participant reported predicting the pseudo-randomized activation pattern with full confidence. We did not find significant effects of age, gender, glasses, VR sickness, or overall experience. Supplementary Material 2 provides fully detailed test results from our ANOVA and post hoc tests. In terms of learning effects, visual trend line analysis within each session showed that participants decreased their average time to reach peak speed by 30ms (\(\beta\)=\(-3.3ms\)), and reduced their overall average movement times by 110ms (\(\beta\)=\(-7.9ms\)). Their throughput increased by 0.09 bits/s (\(\beta\)=\(0.01bits\)/\(s\)) – less than 0.05% – and their straightness increased by 0.02 (\(\beta\)=\(0.002\)).

5.2 RQ2: Impressions of Cognitive Factors in Self-regulation↩︎

Based on Likert scale ratings and the post-experiment interview, we analyzed participants’ experiences of self-regulation with dynamic feedback across the dimensions outlined in our conceptual model in Fig. [fig:conceptualspace] in terms of legibility, correspondence, preference, encouragement and distraction. Violin plots in Fig. 7 visually summarize how people rated the continuous trail, checkmark, and heatmap feedback, after experiencing them in that order.

5.2.1 Perceptual Legibility↩︎

People noticed checkmark feedback significantly more (\(Med\)=\(7\), \(IQR\)=\(0.25\), \(\eta^2\)=\(0.33\)) than both trail (\(Med\)=\(5\), \(IQR\)=\(1.25\), \(p\)=\(0.006\)) and heatmap feedback (\(Med\)=\(3\), \(IQR\)=\(5\), \(p\)<\(0.001\)), scoring the heatmap feedback binomially. The majority of participants paid much attention to the visual aspect of the feedback. They described that the blue trail following the cursor became “longer”, “thicker”, “clearer”, and changed in opacity (transparency). They noticed that checkmarks became greener when they did better and had different opacity. To four participants, the heatmap felt overwhelming: “I got very overwhelmed by the feedback. Too much sound and visual at once.” (P4). Half of the participants (11/24) commented on the changes in sound pitch based on their performance, which was present in all three feedback sessions. Some participants described the blue trail’s sound as a continuous “whooshing” or “whirring” sound: “As I move the cursor, it would grow a tail and make a whirring noise based on my timing/speed” (P6). Few participants (5/24) commented on the presence of haptic feedback, describing it as vibrations timed to the visual and auditory feedback.

5.2.2 Contrasting Legibility↩︎

People had a significantly harder time differentiating between performance metrics with heatmap feedback (\(Med\)=\(1.5\), \(IQR\)=\(1\), \(\eta^2\)=\(0.35\)) than with trail (\(Med\)=\(4\), \(IQR\)=\(3.5\), \(p\)=\(0.007\)) and checkmark feedback (\(Med\)=\(4.5\), \(IQR\)=\(4\), \(p\)=\(0.001\)). Participant scores were spread, except for heatmap feedback where the vast majority scored it low (>3). All 24 participants remarked that they sensed being measured through the metrics, although they could not always tell how. Six participants stressed that when the metric did not feel right, the feedback could feel unclear or weird: “The first one [speed] seemed inverted [e.g. grey when fast and green was slow], the second one [time] felt very accurate and the third one [distance] felt like it didn’t make much sense. The second one [time] felt really good and accurate but the two others were weird.” (P11). Three participants commented that, for them, their ability to differentiate it depended on the metric used to drive it.

5.2.3 Analytical Legibility↩︎

People found it significantly easier to comprehend checkmark feedback (\(Med\)=\(6.5\), \(IQR\)=\(2.25\), \(\eta^2\)=\(0.22\)), than trail (\(Med\)=\(3.5\), \(IQR\)=\(3\), \(p\)=\(0.026\)) or heatmap feedback (\(Med\)=\(3\), \(IQR\)=\(3\), \(p\)=\(0.002\)). Four participants remarked during interview that the checkmarks were best to clarify metric differences. Participants remarked the feedback was reacting to their timing, speed and to being fast. Four participants commented that the trail-based feedback was not as intuitive to hint at their performance: “It became longer I think, I did not realize how long. Sometimes it was completely gone. [..] It wasn’t clear how it changed.” (P16). The heatmap feedback was hard to comprehend and confusing to many participants (12/24): “I saw it after the task, which was kind of confusing the first time. It was very clearly showing where I was doing good or bad, but as it happens after the game, I don’t know why I did bad or good.” (P5). Participants stated that comprehending heatmap feedback required them to recall how they hit the targets. This made it difficult for participants to determine whether they agreed how the metric reflected their performance.

5.2.4 Preferences↩︎

Participants felt they received most feedback from checkmarks

(\(Med\)=\(7\), \(IQR\)=\(1.25\), \(\eta^2\)=\(0.20\)) compared to trail (\(Med\)=\(5\), \(IQR\)=\(2\), \(p\)=\(0.007\)) and heatmap feedback (\(Med\)=\(4.5\), \(IQR\)=\(4.25\), \(p\)=\(0.009\)). They did not agree whether heatmap feedback provided a lot or little feedback, scoring it binomially. Participants thought that the checkmark feedback felt best (\(Med\)=\(6\), \(IQR\)=\(1.25\), \(\eta^2\)=\(0.37\))), followed by trail feedback (\(Med\)=\(4\), \(IQR\)=\(2.25\), \(p\)=\(0.039\)), and heatmap (\(Med\)=\(2.5\), \(IQR\)=\(3.5\), \(p\)<\(0.001\)). The checkmarks were complimented by 16 participants as clear, easy to follow, accurate, and natural: “It made more sense to me, it’s almost like pushing a button thanks to the short sound. The color change was a great visual feedback.” (P8). The blue trail was described as satisfying, visually pleasing, and helpful: “I felt [the trail] helped explain how fast I was, especially the first one [speed], was very concrete.” (P18) Two participants commented that the trail gave them a better overview and indication of where their cursor was. Three participants were not happy with the trail’s sound: “The sound was confusing me but the visual worked very well, making it easier to see the cursor on the screen.” (P8) The majority preferred checkmark feedback (14/24), while others preferred trail feedback (6/24), and four participants indicated no preference. No participants preferred heatmap feedback, some commenting that it had resemblance to a “scoreboard” (P12), or being “graded” (P8) because they had no opportunity to action upon the feedback while playing. Still, some participants shared positive remarks, saying the heatmap felt like a big reward (3/24), and that its timing gave them space to focus when playing (2/24): “It felt nice to sit and look at an overview of how you did, showing the areas you did well or not.” (P2).

5.2.5 Encouragement and Distraction↩︎

Participants felt the checkmark feedback encouraged them most (\(Med\)=\(6\), \(IQR\)=\(2.25\), \(\eta^2\)=\(0.57\)) compared to trail (\(Med\)=\(5\), \(IQR\)=\(3.25\), \(p\)=\(0.035\))) or heatmap feedback (\(Med\)=\(1\), \(IQR\)=\(1\), \(p\)<\(0.001\))).Participants rated heatmap feedback as least distracting (\(Med\)=\(1\), \(IQR\)=\(1\), \(\eta^2\)=\(0.24\))),compared to trail (\(Med\)=\(2\), \(IQR\)=\(2.25\), \(p\)=\(0.042\))) and checkmark feedback (\(Med\)=\(3\), \(IQR\)=\(1.5\), \(p\)=\(0.001\))). One participant remarked: “I did not feel taken out of the game during [the heatmap] and didn’t need to think about feedback while playing.” (P2). Checkmarks were both the most encouraging and most distracting. Four participants stated that checkmarks disrupted their flow, made them feel anxious, or could throw them off when feedback was inaccurate. The time it took participants to reach their peak speed with checkmarks was on average 580ms (\(SD\)=\(112ms\)), which was 6% higher than the overall average time to reach peak speed (\(M\)=\(548ms\), \(SD\)=\(107ms\)).

6 Discussion↩︎

The metrics’ measurable impact on user performance, suggested that most users, provided with knowledge of the metric, could improve their performance through our non-textual dynamic feedback. Yet our results also highlight a potential performance penalty introduced by dynamic feedback in some cases. For example, the checkmark feedback shown at the end of each action was simultaneously the most preferred and most distracting, creating a measurable performance cost - an additional 6% time to reach peak speed. Motor learning studies have often suggested that learners, once having learned to perform, benefit from the withdrawal of the feedback or less frequent feedback [5] – highlighting the value of understanding potential performance costs of feedback. However, unlike motor learning, games face extra considerations when designing augmented feedback – motor learning primarily removes feedback to optimize performance and avoid over-reliance [5], while game designers use it to create variability, rewards ,and balance game challenge [1].

Across feedback with different timing we identified positive and negative impacts on player performance and their sense of correspondence. The end-of-action checkmark feedback was the overall most encouraging and preferred. However, checkmark feedback was simultaneously the most demanding to decode in this experimental setting that dual-tasked participants with play their best while evaluating the feedback. Checkmark feedback yielded the lowest peak speed and phase of movement from peak speed to target.

Our study of functional dynamic feedback adds to the understanding of the design tension between encouraging (e.g juicy) feedback that users appreciate, and the cost from possible distractions, similar to the performance penalties found in [6]‘s evaluation of juicy non-functional feedback  [6]. Our study showed a potential bias in participants’ internal sense of performance – users favoring checkmark feedback despite its measurable cognitive costs may indicate that they favored positive rewards over real performance improvements. But our study was not designed to answer to what degree these improvements were due to legibility difficulties in decoding and evaluating the feedback or the differences in how the metrics fostered better performance. The bimodal distributions of the quantity and perceivability of the heatmap feedback ratings in Fig. 7 suggested that the users might have interpreted or emphasized the questions differently. Participants, who rated the quantity and perceivability of heatmap low, argued that they did so since no feedback was present during the task.

Our conceptual model (Fig. [fig:conceptualspace]) theorized that feedback influences performance based on the extent to which individuals feel a sense of correspondence, encouragement, or distraction. In this study, distraction from feedback determined users’ performance more than its correspondence or encouragement. While participants preferred feedback with higher correspondence, its correlation to performance was negligible.

6.1 Contextualization of the work↩︎

[18]’s notion of naturally coupled feedback connects to notions like natural mapping and intuitive interaction, also discussed in the context of designing control interfaces, like motion-controlled gaming [47], [48]. They also utilize a notion of correspondence, which in contrast to this study, refers to how naturally the physical control interface maps to virtual effects in the game world at different levels [48], and quantifies perceived naturalness [47].

[30]‘s research on low-level action-feedback game loops found that excessive amplification can weaken the causal link between action and response, reducing legibility and user understanding [30]. They tied the efficacy of feedback to curiosity as an underappreciated engagement driver, of continued gaming [30] – the way players can explore the different possible states of animations, provided a novelty effect. [30] highlighted in particular the importance of providing players agency over the exploration, for example by tying feedback to skill rather than random states [30]. This makes feedback dynamics an important agency factor, which may be positively affected when dynamic feedback is tied to measures of user performance. However, effectance is typically framed from the perspective that games must respond to players’ input immediately to evoke strong and continuous experiences of effectance [38] – studies need to investigate how effectance is affected by loose or tight temporal couplings. Our study’s framing differs from [30]’s study, because all possible feedback states (how they might vary in size, opacity, etc.) were explained to users upfront, and because we asked users to make active assessments of what they saw – not only judging causality but judging the extent to which they felt the system’s feedback matched their own.

Dynamic feedback is related to corrective outcome feedback in instructional learning [49], which serves to direct users’ attention to key task aspects, making learners aware of the gap in performance, and apply correction strategies to progress [50]. Research within learning games use other models to represent cognitive processing of feedback, including notions like external and internal attention, to describe how incoming external feedback is organized and integrated with internally generated information [51]. [51] described how feedback’s salience is important for users to attend to feedback - which is controlled by its content, timing and modality[51]. Compared to studies of learning games where feedback operates at larger time scale, this study dealt with feedback and its ability to induce self-regulation in low-level action loops, functioning on a micro-level from a self-regulatory learning perspective. Likewise, fast-paced game genres, such as action or racing games, the need for immediate and clear feedback becomes crucial, as players rely on real-time information to make split-second decisions. In contrast, slower, strategy-based games may benefit from more gradual feedback mechanisms, allowing players to reflect on their actions over longer durations.

For cue research, the design space (Section 2.1) contrasts existing cue frameworks [11], which focus more on the feedback’s role (e.g. what it prompts users to do), rather than the dimensions of design.

6.2 Limitations↩︎

This study utilized three scenarios trail feedback, checkmarks and heatmaps, to quantify how people adapted performance based on specific metric with full knowledge of its function. The study was designed to compare metrics and not designed to quantify performance improvements from specific feedback types. The absence of a baseline condition limited what was known about players’ performance without feedback. By design, the study therefore cannot provide absolute benefits or costs of dynamic feedback. Designing dynamic feedback always involves multiple design choices in terms of expression, location, modality and timing, which can impact participants’ performance, encouragement, and distraction. Quantitative results and participant’s responses will reflect the unique interplay of design choices in the study’s feedback and the individualized feedback adaption system. The study did not not utilize pre-study trials, which limits the accuracy of the initial feedback.

We have hinted at potential player preferences in our qualitative research, but leave it for future work to quantitatively compare the added benefit of trail, checkmark and heatmap feedback.

This study has sought to exemplify a systematic exploration of dynamic feedback in a pointing game context. Our results are based on a target-to-target pointing task commonly found in shooting games but might not apply in games with longer self-regulation cycles leaving more reflection time during turn-taking or disruptions [52].

Lower ratings of correspondence for the straightness metric may stem from the use of the word ‘fast,’ whose connection to making accurate trajectories may have been unclear to participants. During pilot testing, ‘fast’ was clearer, while terms like ‘better performance’ lacked precision. Despite this, participants still acquired targets more accurately when given feedback based on the straightness metric.

6.3 Future work↩︎

Our study opens new avenues for exploring dynamics of feedback and its temporal placement in user interactions. Other metrics could be evaluated to understand participants’ ability to adapt to them, such as metrics that reward fast acquisition of targets (i.e., the corrective movement phase in [43]’s model) or rewards reaction time to the presence of the next target (visual search [43]). Our goal was to design metrics that were easy to explain and test whether they could moderate play behavior. Future work could investigate how advanced metrics based on advanced models of human performance or artificial intelligence fare in encouraging specific play behaviors.

Complex feedback that integrates multiple sources of information, risks being difficult to understand and predict, due to their black-box nature  [53]. This legibility issue has led to research in how systems can create explainable user interfaces by providing post hoc explanations, like exploratory data visualization to audit results [54]. Our findings suggests that complex feedback systems will need additional design features (e.g. meaningful supplemental textual feedback) to reach enough legibility (or algorithmic transparency [55]). Studies building upon our results could compare no-feedback scenarios, non-dynamic feedback, or measures participants twice per condition, to e.g. make it possible to measure players’ adaption to task feedback. Similarly, explorations of how other feedback modalities might fare e.g., textual feedback overlays such as percentage-wise feedback (e.g. “80%” as a distance from their best performance, which would be “100%”) or measurements in standardized units (“8m/s”, “2.3m”, “4.1s”), or situational feedback such as effects of combos, streaks, multipliers, or accumulative mechanics.

Our study results are based upon a simple pointing game in a VR environment, which offered experimental control and generalization. Our methodology could be applied for pointing in more complex game environments resembling off-the-shelf games like Fruit Ninja or Counter-Strike, which often feature ultimate goals (e.g. winning/loosing) and imperative goals (e.g. acquiring points/kills) within highly developed fictive worlds.

Our work and the conceptual model of interaction can be applied in future game scholarship, in, for example, motion-controlled VR games for entertainment and training applications, and beyond VR in HCI systems for which improvements in motor performance are a goal, such as improving eye-hand coordination [56], rehabilitation [57], or exercise [29]. Exploring optimal dynamic feedback strategies may also be relevant for competitive electronic sport. Elite esport performers and coaches constantly look for means of achieving superior accuracy and time trial performance. Behavioral monitoring behavior and feedback plays an important role as part of cyclical self-regulation to optimize esport performance [58]. By interpreting player movements and providing interpreted feedback, games can take a supplemental role to coaching, regulating players’ behavior to achieve a specific result [59].

Considering the limitations of the results and the design of this reported study, the authors are continuing to redesign and re-evaluate the metrics and feedback using a more elaborate, counterbalanced setup. This will allow us to more clearly disentangle the effects of feedback, instruction, and different learning and fatigue effects.

6.4 Design Guidelines↩︎

Based upon our results, we suggest the following design guidelines for feedback in pointing tasks. When designing performance metrics, designers should focus on designing metrics for the tightest temporal coupling first, as they translate better to looser non-continuous feedback through aggregation than in reverse. For minimal distraction and speed penalty, metrics should utilize tight temporal coupling like operation feedback. For optimal legibility, performance metrics should be tied to action feedback instead, but practitioners should be aware that it can create performance and distraction costs for subsequent actions, which may not be easily recognized by participants. Action feedback needs to be easy to decode and minimize interruptions. When designing task-level feedback, practitioners should be mindful of potentially low correspondence. To support players to improve, it should be coupled with action- or operation-level feedback.

Designers can choose to incorporate dynamic feedback as part of a cue, if the source metric information is meaningful for players’ task. Wensveen’s feedback dimensions may also help studies of cues to explore and explain possible mappings with greater detail. For example, [12]’s study of body posture cues [12], could situate their choice of cue design within a much larger design space than merely color. Likewise, experiential measures like correspondence could help them decide upon choice of metric to provide continuous body pose cues.

7 Conclusion↩︎

Our study evaluated how players’ adapted their performance to specific performance metrics, visualized as dynamic feedback in a simple VR pointing task. The study was scoped to quantitatively compare metrics, leaving form-to-form performance contrasts beyond scope. On average, participants improved their performance according to performance metrics of completion time, peak speed or movement straightness, which drove dynamic feedback displayed continuously, end-of-action and end-of-task. To achieve desired player behavior, game designers should carefully align performance metrics with a game’s objectives, to support players’ reaching their potential – e.g., use a peak speed metric if achieving higher peak speed is the goal. However, users may prefer mappings that do not optimize performance. Designers should also be aware of the performance cost feedback may induce – although feedback shown at the end of each action may be preferred by players, they also consider it distracting (e.g. impose a cognitive cost for subsequent actions.) Our results demonstrate how providing a conceptual self-regulation model can be used as a design tool to explain the effects of feedback collected in experiments and through playtesting. our work opens up a new avenue for exploring how basic dynamic feedback can be systematically studied and how it can impact player experience.

References↩︎