Small Latency Variations Do Not Affect Player Performance in First-Person Shooters

In interactive systems high latency affects user performance and experience. This is especially problematic in video games. A large number of studies on this topic investigated the effects of constant, high latency. However, in practice, latency is never constant but varies by up to 100 ms due to variations in processing time and delays added by polling between system components. In a large majority of studies, these variations in latency are neither controlled for nor reported. Thus, it is unclear to which degree small, continuous variations in latency affect user performance. If these unreported variations had a significant impact, this might cast into doubt the findings of some studies. To investigate how latency variation affects player performance and experience in games, we conducted an experiment with 28 participants playing a first-person shooter. Participants played with two levels of base latency (50 ms vs. 150 ms) and variation (0 ms vs. 50 ms). As expected, high base latency significantly reduces player performance and experience. However, we found strong evidence that small variations in latency in the order of 50 ms, do not affect player performance significantly. Thus, our findings mitigate concerns that previous latency studies might have systematically ignored a confounding effect.


INTRODUCTION
System latency, the time between user input and system response, is a inherent property of interactive systems.A system's end-to-end latency is comprised of delays added by hardware, polling rates, processing time, and the time needed to transfer data over a network.As interactions between humans and computers are e ectively feedback loops [6], where users constantly react to the system's response, high latency a ects user experience and performance.This e ect is especially problematic in real-time applications, such as video games [9,38,63], or in psychological studies [53][54][55], where participants' reaction time is measured.With e-sports having become a multi-million dollar business and the replication crisis in psychology [50], understanding and counteracting latency and its e ects is more important than ever.Consequently, manufacturers of gaming hardware advertise their products as "low latency" and even include latency-measuring technology directly into the hardware, e.g., Nvidia Re ex [59] in gaming monitors.
Research focusing on latency has a long history, starting with MacKenzie and Ware's seminal work in 1993 [49], in which the authors showed that adding latency increases movement time and error rate in pointing tasks.Since then, scienti c community and gaming enthusiasts developed a number of methods for measuring the latency of di erent systems (for example, desktop PCs [7,28,33,57], smartphones [15,37], virtual reality systems [16], or input devices [71]).In practice, the base latency of a system is not constant.It varies e.g., depending on the polling rate of the USB connection [71], game loops, processing times, and vertical display synchronization.However, in most studies which investigate the e ect of latency on user performance, a constant amount of latency is added to the system response for each condition, and only these constant values are reported.Rarely do authors measure or report how large the latency variations of their setup are.
This might be a problem.In case latency variation has a noticeable e ect on users' performance in real-time applications, consequences would be signi cant for several sub-disciplines of humancomputer interaction and psychology, including games research.As it is present in all interactions between humans and computers, latency variation could confound results of user studies investigating time-critical phenomenons.It could therefore, if not controlled for by thoroughly measuring the apparatus' latency distribution, invalidate ndings in the worst case.Therefore, e ects of latency variation on users could contribute to the replication crisis [54,55], leading to non-existent e ects being measured, reported, and published [62].
There is evidence that lower system latency with higher variation a ects users more than higher latency with lower variation [13,26,69], as users can compensate for known constant latency by adjusting their behavior.Clicking slightly before anticipated events or aiming slightly ahead of targets are examples of such compensation strategies.However, studies investigating varying latency frequently use blocks of constant latency with sudden changes or large latency ranges [13,26,69].In practice, latency variation is much more subtle with standard deviations clearly below 100 milliseconds around the mean latency [30].Currently, it is unknown how small variations in local latency, as they occur outside of a laboratory setting, a ect user experience and performance.
In this work, we investigate how small variations of local latency (latency jitter) in uence user experience and performance in gaming.Latency jitter is the variation of local latency that occurs with each input, as opposed to network jitter, which is caused by bu ering of packages in network communication [74].We conducted a within-subjects study ( = 28) to investigate how latency jitter in uences game experience and performance when playing a fast-paced rst-person shooter (FPS).We utilized a FPS since previous work showed that they are particularly negatively a ected by latency [9].To operationalize latency jitter, we varied the level of mean base latency (low = 50 ms vs. high 150 ms) and the level of variation (low = ±0 vs. high = ±50 ).To maximize internal validity, we used a system optimized for extremely low and constant end-to-end latency for our study apparatus.Our work aims to answer the following research questions: 1 : "How does base latency in uence performance and game experience in rst-person shooter games?" 2 : "How does latency variation in uence performance and game experience in rst-person shooter games?" 3 : "How does the interaction between base latency and latency variation modulate the e ects on performance and game experience?" The results of our data analysis consolidate previous ndings and show that a high latency a ects game performance and experience ( 1 ).However, we found no e ect of latency variation on neither the players' performance nor their experience ( 2 ).Furthermore, investigating the interaction between base latency and its variation, we found no e ect on most of our measures.However, we found that players derived a greater sense of meaning when playing the game with low base latency and high variation compared to a high latency with low variation ( 3 ).
Our ndings are crucial to latency and video games research since we show that local latency jitter does, generally, not a ect performance and experience, thus, validating previous work de ning latency as a constant.

RELATED WORK
A expanding amount of work addresses latency and its impact on users and video game players.In this section, we rst provide an overview of how latency arises in interactive systems and the problems users experience due to latency.We then focus on the impact of latency in video games and its e ects on player performance and experience.We highlight previous work investigating latency variability and how volatile latency adversely a ects users and players.Lastly, we conclude this section with a summary in which we spotlight why latency inherent variability needs to be accounted for when investigating its e ects.

Sources of Latency
A system's end-to-end latency, the time between user input and system response, is comprised of several partial latencies [7,8].When an input event is triggered, for example by physically closing the contacts of a mouse button, an event is transferred from the input device to the computer via USB.However, the input device itself contributes to end-to-end latency as it takes time to scan and de-bounce buttons and since USB polling rates are limited [71].The input event is registered by the operating system's kernel and passed on to the user space, where input callbacks of application toolkits are triggered.Task scheduling, high system load caused by background applications, as well as input handling of the application toolkit can delay this process [7].An application, e.g., a video game, then reacts to this input.
In network applications, such as multiplayer games, events also have to be transferred to a server that sends back a response.Depending on the type of connection, bandwidth, and physical distance to the server, network round-trip times can have a signi cant impact on latency.
Most applications update their state in a loop and re-paint regions if necessary.The latency added by re-painting depends on the used graphics toolkit or game engine, as well as the complexity of the rendered content [58].This step is highly resource-intensive for modern video games because of complex physics simulations and high-resolution models and textures.Once an image is rendered to a frame bu er, it is sent to a monitor and displayed to the user.In addition to the time it takes to transfer an image to the monitor (display response time [17,64]), the monitor's refresh rate also contributes to end-to-end latency.A monitor continuously updates its content at a xed rate.For a 60 Hz monitor, a new image is gradually displayed every 16.67 milliseconds.If vertical synchronization (VSYNC) is enabled, the system's frame bu er is synchronized to the monitor's refresh rate, so images are always drawn from top to bottom.Even though this prevents screen tearing, images might get delayed by up to one monitor refresh cycle in the worst case.

E ect of Latency in Interactive Systems
As latency delays the feedback loop of users reacting to a system's state and the system responding to users' input, it has a direct in uence on task di culty and task completion time.In an early study on latency in remote control tasks, Sheridan and Ferrel [61] show that task time and the number of open loop movements increase linearly with added delay.MacKenzie and Ware [49] show that latency increases task time and di culty in pointing tasks.They also incorporate latency as a factor for the index of di culty into the Fitts' Law model [22].Ever since, task time and throughput have become established measures for the e ect of latency on atomic actions, such as pointing.For example, Teather et al. [65] compared the e ects of latency and spatial jitter in pointing tasks with two di erent input methods.They found that latency deteriorates throughput and increases movement time signi cantly more than spatial jitter.Friston et al. [23] investigated the e ects of small latencies on Fitts' Law and Steering Law [2] tasks.Their results are in line with ndings in previous studies [49,52,65].Furthermore, Friston et al. found that latency in pointing tasks predominantly increases the time users spend correcting their movements.
The perceivable threshold for latency is strongly dependent on input modality and task.Jota et al. [32] found that the just noticeable di erence for latency for touch events is 64 milliseconds on average.Ng et al. [51] found that users can perceive a latency di erence of as low as one millisecond when dragging.Kaaresoja et al. [34] used empirical data to establish latency guidelines for di erent feedback modalities of virtual buttons on capacitive touch screens.They recommend latencies of 5 -50 ms for tactile feedback, 20 -70 ms for audio feedback, and 30 -85 ms for visual feedback.In the studies mentioned above, a custom-built apparatus with extremely low latency was used to measure perception thresholds as accurately as possible.

Latency in Video Games
As video games are real-time applications, oftentimes requiring quick reactions from players, they are especially prone to the e ects of latency.Accordingly, there is a large body of research on how latency in uences players, how much latency is tolerable in di erent genres, and how to counteract latency with predictive models either on the gaming system or in the game [24,29,40,60].
Eg et al. [18] used a simple 2D game to investigate how latency a ects players' performance and quality of experience (QoE).In their game, players had to follow a moving circle with the mouse cursor and click it as quickly as possible.In a within-groups study, participants played this game with di erent amounts of latency added to the the mouse.Higher latency lead to higher task time and lower QoE among participants.Beigbeder et al. [5] investigated the e ects of network latency and packet loss on player performance in the rst person shooter Unreal Tournament 2003.They controlled latency and packet loss for individual players with a network emulator.High latency signi cantly a ected players' precision and kill/death-ratios.
Claypool and Claypool [10] analyzed game genres in terms of susceptibility to latency and found that di erent latency threshold exist for di erent genres.They categorize rst person shooters as one of the genres that is most susceptible to latency.This is in line with ndings from Armitage [4] who determined the latency threshold for players of the rst person shooter Quake 3 to be around 150 milliseconds.However, this study is quite old and from today's standpoint, much lower latency thresholds should be applied.For example, Liu et al. [47,48] conducted two user studies in which participants playing the rst person shooter Counter Strike: Global O ense under di erent latency conditions.They found that latency linearly degrades game experience and players' scores starting at 25 milliseconds and that local latency a ects players more than network latency.In similar work, Liu et al. [44] showed that latency negatively in uences players' navigation capabilities in FPS games.Players playing with a higher latency required more time to move their avatar in a game-winning position, than players playing with a lower latency.
Due to the low perception threshold for latency on the one hand, and physical limits for data transmission rates on the other hand [4,38], new approaches for compensating latency with predictive models are emerging.For example, Halbhuber et al. [27] trained a CNN to compensate 50 milliseconds of latency by predicting the mouse position in a real time strategy game.Their system could signi cantly improve game experience among participants.

Latency Variability
If a system's latency is constant, users can compensate by pressing buttons early or clicking in front of a moving target.However, if there are variations in latency, the system's behavior becomes unpredictable, and compensation strategies no longer work.Therefore, a high but constant base latency might be better than a lower latency with high variability.Some studies have investigated the e ect of varying latency in di erent usage scenarios.
Weber et al. [69], for example, investigated the e ects of varying system response time (SRT) on user performance, task load, and user experience.In a within-group study, participants were asked to classify e-mails with a GUI dialog system with two levels of SRT variability (low vs. high).In the condition with low SRT variability, system response time after each user input was either 750 or 3000 milliseconds.In the condition with high SRT variability, system response time was randomly selected between 300 and 3000 milliseconds in 450 ms steps.Despite the higher total time on task with low SRT variability due to the higher mean SRT, task execution time was signi cantly lower in this condition.Weber et al. explain this e ect with temporal expectancy, as users can better predict the system's behavior when SRT variability is lower.
Davis et al. [13] investigated the e ects of xed and variable latency on driving performance and mental load in a driving simulator study.They rst gathered baseline data in a pre-study where participants drove without added latency.For their main study, Davis et al. compared high constant latency (700 ms) to varying latency (400 -1100 ms, mean: 700 ms).Latency variation was generated with a sinusoidal function.In both latency conditions, participants performed signi cantly worse than the baseline regarding lane o set, average velocity, and task load.With varying latency, lane o set was signi cantly higher than with constant latency.However, the latency condition had no main e ect for average velocity, task load, and motion sickness.
It is worth noting that both studies, Weber et al. 's [69] and Davis et al. 's [13], used extremly high values for mean latency and latency variation.Even though they could nd that varying latency impairs users' performance stronger than constant latency, this does not mean that their ndings apply to typical gaming systems with much lower latency ranges [30].
Halbhuber et al. [26] investigated the e ects of regularly switching latency on game experience and player performance in a browser-based 2D shoot-'em-up game.During a gaming session, latency was either constant or switched between zero and 33 or 66 milliseconds of arti cially added latency in di erent frequencies.They found that overall, the negative e ect of latency on game experience and player performance was stronger with switching latency than with a constant high latency.

Summary
Numerous studies have shown that latency directly a ects task di culty and task time -and in turn deteriorates users' performance [23,49,65].This e ect can be modeled accurately for atomic tasks, such as touch-based pointing [32] or target selection with a computer mouse [43].There is strong empirical evidence that this e ect also applies to more complex real-time applications such as video games [4,10,24,45,47].However, the impact of latency in video games strongly depends on the game's genre, with fast-paced, dexterity-based games, such as FPS games or racing games, being a ected more severely than strategy games [10].
In most studies, latency is assumed to be constant and constant amounts of delay are applied to users' actions to simulate higher system latency.Only a few studies have investigated the e ects of varying latency.In those studies, an e ect of latency variation on user performance could be found [13,26,69].However, extremely high base latencies and latency ranges were used, so those results are not directly applicable to real-life gaming systems with much lower latency variation [30].In conclusion, previous work does not answer how low-value latency variation with a high ecological foundation a ects performance and experience.

APPARATUS: GAME MODIFICATION AND LOCAL LATENCY
In our work, we used the open-source game Cube 2: Sauerbraten1 , a fast-paced rst-person arena shooter developed in 2004.In Sauerbraten, players control an avatar equipped with a virtual weapon.The game's goal is to navigate the avatar through di erent levels and shoot other entities, such as other players or AI-controlled bots, to survive and gain points.Despite its age, Sauerbraten enjoys a lively and consistent community, which recently even launched an o cial Steam fork of Sauerbraten called Tomatenquark2 .The rationale to use Sauerbraten in our work is threefold: Firstly, FPS games, such as Sauerbraten, are susceptible to latency as demonstrated by previous work [9,47,48].In FPSs, latency leads to players being less accurate, scoring fewer points, and having a reduced gaming experience.Secondly, Sauerbraten is an open-source project which allows us to modify and control every aspect of the gaming session, such as what weapons players are allowed to use, how AI-controlled bots behave, and which maps are played.Furthermore, in contrast to proprietary video games, we have direct access to the game's source code which makes low-level logging of game events straightforward.Thirdly, preliminary tests have shown that Sauerbraten is highly performant and has a very low impact on the system's end-to-end latency.High game performance is crucial in our work since we aim to investigate low amounts of latency variation.Hence, every uctuation, for example, induced by a game with high demand on system resources, may potentially bias our work.
In the following section, we rst highlight our modi cation to the game to make it t to be used in a study with high internal validity.Then, we elaborate how we measured the local latency of our setup since the local latency needs to be factored in all future investigations.

Modification of Cube 2: Sauerbraten
Sauerbraten is typically played against other humans or multiple bots.However, to prevent di ering player skills and play styles to confound our work, we modi ed the game so one player only faces one bot at a time.We used Sauerbraten's built-in level 75 bots 3 which corresponds to a medium to hard di culty.We set the di culty level of the bot in a way that players are neither under-nor over-challenged by it.
As the map and the player's weapon fundamentally change how the game is played, we restricted both for our study to prevent confounding e ects.We restricted rounds to the in-game map Teahupoo.Furthermore, we disabled all virtual weapons except the standard pistol which we modi ed to have unlimited ammunition, and prevented players from changing or picking up new weapons during the gaming session.
In the next step we removed all keyboard shortcuts which are not essential for our work, such as using a medikit or opening menus such as the map overlay.Lastly, we added custom logging functions to track di erent game events.We logged the number of shots, hits, misses, players deaths, and bot deaths.Figure 1 shows an aerial view of the game map Teahupoo (right) and the in-game view of the player (left).

Local Latency
To investigate the e ects of latency and latency variation on players of our modi ed FPS game, we needed (a) a way to reliably add latency to the test system, and (b) a system with very low and constant end-to-end latency.
To add latency to the system, we used a C program using the evdev4 library to capture and block input events from physical input devices, similar to Liu and Claypool's EvLag [42].For each input event from a mouse or keyboard, the program creates a thread which waits for a speci ed amount of time before invoking the input event with a virtual input device provided by evdev.The program allows for adding constant or varying delays with uniform or normal distribution.The process is illustrated as pseudo code in listing 1.Additionally, the program can be controlled via inter-process communication using a FIFO queue so added latency can be adjusted between conditions without needing to restart the program.For our study, we used a HP Pavillon Gaming 790 desktop PC 5 running Debian Buster 5.10 with proprietary Nvidia graphics drivers (version 470.103.01).In terms of periphery, we used an ASUS ROG Strix XG248Q at 1920 × 1080 pixels with 240 Hz, a Logitech G15 gaming mouse 6 , and a Logitech G213 gaming keyboard 7 .
We measured the end-to-end latency of our system with Schmid and Wimmer's Yet Another Latency Measuring Device (YALMD) [57], an Arduino-based device which electrically triggers a button click on an input device and measures the time until a brightness change on the computer's monitor is detected with a photo sensor.We used YALMD to trigger a mouse click leading to a gun shot in Cube 2: Sauerbraten and attached the photo diode at the center of the screen so a visible muzzle ash would stop the latency measurement.This way, we could also validate that our method for adding latency to the system is accurate and does not introduce unwanted additional latency.The measured end-to-end latency of our system running Cube2: Sauerbraten is between 6.2 and 15.5 ms (M = 9.11 ms, SD = 1.4 ms).Detailed measurement results are depicted in Fig. 2. All latency values reported in the remainder of the paper incorporate this local latency without explicitly mentioning it.

INVESTIGATING THE EFFECTS OF VARYING LATENCY IN FIRST-PERSON SHOOTER
GAMES To investigate how varying latency a ects game experience and player performance in video games, we conducted a within-subjects study with 28 participants playing our modi ed version of Cube 2: Sauerbraten.Participants played with two levels of mean base latency and two levels of latency variation.
The negative e ect of high latency on game experience and player performance has been shown in numerous studies [4,5,10,18,44,47,48]. Therefore, we expect to measure the same e ect in our study.Furthermore, previous ndings [13,26,69] suggest that high latency variation a ects game experience and player performance more than constant latency with the same mean, as players can adapt their behavior to compensate constant latency.Concluding, we hypothesize that both, high base latency and high latency variation a ect game experience and player performance.

Study Design
In our study we utilized two independent variables (IV) in a 2 × 2 within-design to control for mean base latency and latency variation: (1) B refers to the mean baseline latency participants played with.B has two levels: (I) low which refers to playing with 50 ms, and (II) high which refers to playing with 150 ms of base mean latency.The second IV is V and de nes how much the actual latency varied around the mean base latency B .V has also two levels: The rst level (I) low refers to no variation.The second level (II) refers to a variation of ±50 ms.This entails that the actual latency, for example, in a high B / high V round varied from 100 ms to 200 ms (150 ms ± 50 ms).Latency was applied to each input of the computer mouse (movement and clicks) and keyboard following a uniform distribution.Therefore, during conditions without V , all input events are delayed by a constant amount.During conditions with V , random delays are added to each event, resulting in jittering mouse movement.In those conditions, it is also possible that the order of rapid consecutive input events changes if high latency is applied to the rst event and low latency is applied to the second event.The levels of B are in line with previous work, which shows that latency in the wild reaches values up to 150 ms [30,31].Fig. 2. Results of end-to-end latency measurements for di erent latency conditions.The plot on the le hand side depicts the system's end-to-end latency running "Cube 2: Sauerbraten" without any added latency.The plot on the right hand side shows the system's end-to-end latency for the di erent conditions used in our study.Each measurement series consists of 500 individual measurements with random delays in between.
The measuring probe was a ached at the top le corner of the monitor and VSYNC was disabled.
However, the levels of V are constrained by the chosen levels of B since we are not able to decrease latency below 0 ms.Thus, the lower bound of B de nes the upper bound of V .As our method for measuring the system's end-to-end latency requires measuring probes to be attached to the input devices' circuit boards, as well as bright ashing regions on the screen, we did not measure the system's latency during the study.Therefore, we validated all latency conditions beforehand with the method described in section 3.2.Results can be seen in Fig. 2.
To measure the players' performance and game experience, we utilized a range of dependent variables (DVs).In line with previous work, player performance is measured in three DVs: (1) Hitrate [24] -which quanti es the ratio of total shots to successful hits, (2) KD-Ratio [25] -which refers to the ratio of player kills and deaths, and lastly (3) TotalKills [46] -which corresponds to the total amount of enemy kills per round.
To quantitatively evaluate participants' game experience, we used the 30-item Player Experience Inventory (PXI) [1,66].We used the PXI, since the instrument was rigorously validated and tested in in multiple studies [66].Given its multi-dimensionality the instrument allows for an in-depth analysis of player experience, contrary to other work, which, for example, uses single-item questionnaires to assess game experience.The PXI is divided into two categories: (1) functional consequences and (2) psycho-social consequences.The functional consequences dimension is built by ve subscales Ease of Control, Progress Feedback, Audiovisual Appeal, Clarity of Goals, and Challenge and encompasses essential game aspects such as gameplay mechanics, controls, and audio-visual elements.Latency substantially impacts these functional components.For instance, a delay in player actions being registered due to high latency can result in a diminished sense of control [72], a reduced responsiveness [5] or a feeling of an inappropriate challenge [26], ultimately leading to a less satisfying gameplay experience.
The psycho-social consequence dimension of the PXI delves into the social and psychological rami cations of gaming.The dimension describes second-order emotional experiences derived from playing and it also contains ve subscales: Mastery, Curiosity, Immersion, Autonomy, and Meaning.Potentially, latency a ects each of the psycho-social subscales individually and di erently as each of dimension shapes one crucial aspect of the overall gaming experience.

Procedure
Participants were met and greeted at the laboratory by an experimenter.Participants were not informed about the exact details of the study (to investigate the e ects of varying latency), to prevent a bias induced by the participants' expectations [25].Hence, participants were just told to test a game.Subsequently, participants gave informed consent to our data collection and were briefed on the further course of the study.After we explained the controls and the objective of the game, each participant played six rounds of Cube 2: Sauerbraten.Each round lasted for ve minutes.The rst and last rounds of the study were always played without arti cially added latency (B ) or variation (V ) to control for exhaustion induced performance degradation.In the remaining four rounds, we altered B and V .Each of the four rounds represents one of the unique combination of B and V .The nested rounds with changing B and V were counterbalanced using a balanced Latin Square design to prevent sequencing e ects.After each round, participants lled out the PXI on a separate device and had a short break, which allowed us to alter the game for the next round.Upon nishing all six rounds, participants lled out a demographic questionnaire and the study was concluded.In a short debrie ng, we informed participants about the exact purpose of the study.We estimated a total duration of one hour for participation.The study was designed, conducted, and analyzed following the research ethics policy issued by our institution and received clearance per the policy8 .

Apparatus and Task
As apparatus, we utilized the low-latency hardware setup described in section 3.2.Our modi ed version of Sauerbraten was executed in full-screen mode.
In each of the six rounds, the participants controlled an avatar equipped with a virtual weapon.In the game world, participants were free to roam and fought against one AI-controlled bot.The participants' objective in the game was to shoot the adverse bot as often as possible without getting shot by the bot.After shooting a bot three times, the bot died and respawned at a random location in the game world.If the bot hit the players' character three times, the player character died as well, and also respawned at a random location in the game world.Players obtained points for successfully killing an enemy bot.However, they did not lose points if they did not hit the bot or got killed themselves.This overall procedure was repeated six times (four times for each unique combination of B and V and two times with an unaltered game version).

Participants
Since previous work showed that the e ects of latency are reliably detectable with a relatively small number of participants (Halbhuber et al. [26]: 24 participants per condition, Liu et al. [45]: 25 participants), we recruited 28 participants (24 male, four female) using our institution's mailing list and advertisement in a local gaming club.The participants' age ranged from 20 years to 33 years with a mean age of 24.6 years (SD = 3.4 years).Participants' prior experience with FPS games ranged from 10 hours to 18 000 hours, with 2081 hours (SD = 525 hours) on average.Their self-reported skill level on a 10-point Likert-scale ranged from 2 points to 9 points (M = 5.2 points, SD = 2.1 points).
Students participating in our study were eligible to obtain one credit point for their course of study as compensation for their participation.

RESULTS
In this section, we report the results of our data analysis.We structure this section by IVs instead of DVs for better readability.Additionally, we only report p-values in body text.However, full inferential statistical data can be found in Table 1.We used an alpha level of .05 for all statistical tests (signi cance assumed if p < .05).The collected data were screened for normal distribution using Shapiro-Wilk tests (Gaussian distribution assumed if p > 0.05).All measures, except the Autonomy subscale of the PXI (p = 0.632), showed a violation of normal distribution (all p < 0.05).
For inferential assessment of non-parametric data for B (low vs. high) and V (low vs. high), we applied a rank-aligned 2 × 2 ART-ANOVA [73] with repeated measures on both factors.Analogously, we used a conventional 2 × 2 ANOVA for the analysis of parametric data.The participant's ID was entered as error term in both ANOVAs to account for random variation induced by individual participants.E ect sizes ( 2) are interpreted following the recommendation by Field [21]. .Participants performed significantly be er when playing with low L than with high L .They were more accurate, had a be er kill-to-death ratio, and killed more bots overall.There was no e ect of latency variation on any of the dependent variables in both B latency conditions.

Base. ART-ANOVA revealed signi cant main e ects of B
on Hitrate, KD-Ratio, and TotalKills (all p < 0.024).Participants performed signi cantly better when playing with low L than with high L .They were more accurate, had a better kill-to-death ratio, and killed more bots overall.Figure 3 depicts Hitrate (left), KD-Ratio (center), and TotalKills (right) grouped by unique combinations of B and V .ART-ANOVA and ANOVA showed signi cant main e ects of B on all subscales of the PXI (all p < 0.015).Overall, participants had a signi cantly better game experience when playing with low than high B .Participants rated the game as easier to control, were more satis ed with the progress feedback provided by the game, found the game to be more appealing on an audiovisual level, had an easier time grasping the game's goal, and found the challenge provided by the game to be more appropriate when playing with the lower level of B .Furthermore, players derived a greater extent of mastery, meaning, autonomy, and immersion in the low B conditions.

Variation.
We analysed the impact of V with the same systematic as B .ART-ANOVA revealed no signi cant main e ect of V on Hitrate, KD-Ratio and TotalKills (all p > High Base, Low Variation High Base, High Variation Fig. 4. Depicts the results of Ease of Control, Goals and Rules, Challenge, Progress Feedback, Audiovisual Appeal, Meaning, Curiosity, Mastery, Immersion, and Autonomy for each combination of B and V .In general, participants rated the game as easier to control, were more satisfied with the progress feedback provided by the game, found the game to be more appealing on an audiovisual level, had an easier time grasping the game's goal, and found the challenge provided by the game to be more appropriate when playing with the lower level of B .Furthermore, players derived a greater extent of mastery, autonomy, meaning, autonomy, and immersion in the low B conditions. 0.365).Furthermore, ART-ANOVA and ANOVA showed no signi cant main e ects of B on any subscale of the PXI (all p > 0.131).We found no signi cant e ect of V .Hence, V as an isolated factor did not alter players' performance or game experience.

5.1.3
Base × Variation.Lastly, we analysed the interaction B × V and its e ects on our measures.ART-ANOVA showed no interaction e ect on Hitrate, KD-Ratio, and TotalKills (all p > 0.157).Similarly, ART-ANOVA and ANOVA revealed no signi cant interaction e ect on all subscales of the PXI (all p > 0.174) except Meaning.
Participants playing with low L / high V derived a signi cantly greater level of meaning from playing the game (M = 4.154, SD = 1.306) compared to participants playing with high L / low V (M = 3.083, SD = 1.239) (Fig. 4).
Table 1.Results of the B and V ART-ANOVA and ANOVA (*) analysis.Each row represents one dependent variable and its analysis for either main e ects of B and V or the interaction e ect B × V . We found significant negative main e ects of B on all measures and no e ect of V .Investigating the interaction, we found that players derived a significant greater level of meaning from playing with low B / high V than playing with high B / low V .

Bayesian Inference
To further examine the e ects of latency and its variation, we performed multiple Bayesian 2 × 2 RM-ANOVAs with B (low vs. high) and V (low vs. high) as factors.Null hypothesis signi cance testing (NHST) detects di erences between distributions, thus, accepting or rejecting a null hypothesis.While useful for detecting statistical di erences in data, NHST cannot determine if an insigni cant di erence indicates similarity between the studied data.To explore similarity in our data, we utilized a Bayesian analysis, which estimates the probability that the null hypothesis (i.e., no di erences in distribution) is true, instead of rejecting it, as NHST does [11,68].Unlike NHST, Bayesian inference calculates probabilities for both 0 and 1 .
We used JASP [67] and followed the default prior probability distribution recommended by Wagenmakers et al. [68] for Bayesian inference.For post-hoc testing, we used Bayesian t-tests, and corrected the posterior odds for multiplicity using Westfall's approach [14,70].To interpret Bayes factors [35,39], which indicate the strength of evidence for 0 over 1 , we followed the guideline of Lee and Wagenmakers [41].

Base.
A Bayesian 2 x 2 RM-ANOVA found extreme evidence (0 < 01 < 0.01, error = 0.647 %) for a model that supports a true e ect of B on Hitrate, on KD-Ratio (0 < 01 < 0.01, error = 0.771 %) and TotalKills (0 < 01 < 0.01, error = 1.184 %), which indicates that the gathered performance data is at least a hundred times more likely in support of a distribution in which B alters Hitrate, KD-Ratio, and TotalKills.
In summary the Bayesian inference of the e ects of L consolidate our previous ndings, which demonstrated that the mean latency -B -fundamentally altered player performance and gaming experience.

DISCUSSION
The results of our NHST showed that a high base latency signi cantly a ects player performance and gaming experience.However, we found no signi cant main e ect of latency variation on either the player performance or the experience.While most of our measures were una ected by the interaction between latency and its variation, inferential analysis showed that players rated playing the game with low base latency and high variation better on the PXI's meaning subscale than playing with high latency and low variation.The meaning subscale, in general, refers to the level of meaning derived from playing the game.It quanti es how well players connected with the game, and how well they were able to resonate with what is important while playing the game on a psycho-social level [1].
We consolidated our ndings in regards to base latency ( 1 ) using a Bayesian analysis, which showed that our data is highly in favor of a model that acknowledges latency as a true e ector on player performance and game experience (all 0 < 01 <= 0.061).Furthermore, the analysis revealed moderate evidence that latency variation ( 2 ) does not alter player performance (5.062 < 01 < 6.758) and weak to moderate evidence in support of a model that postulates no e ect of latency variation on the gaming experience (1 < 01 < 5 for nine out of ten PXI subscales).Investigating the interaction between latency and its variation ( 3 ) revealed that our data is in favor of a model that supports no true e ect of the interaction on all measures with weak to moderate evidence (1.5 01 < 5.1), except on the meaning subscale of the PXI ( 01 = 0.11).In this section, we rst discuss and contextualize our ndings about the e ects of high base latency and latency variation on player performance and game experience, shed light on how base latency and its variation interact.We conclude by discussing our study's limitations and possible future work.

E ects of Base Latency, Latency Variation and the Interaction Regarding
1 , the in uence of constant latency on players, our ndings brace previous work, demonstrating that high latency leads to a decrease in player performance and game experience.Liu et al. [47], for example, showed that linearly increasing latency from 25 ms to 125 ms in the FPS game Counter-Strike: Global O ensive decreases players' accuracy and overall score.Other work, for instance, by Sabet et al. [56], illustrated that high latency also has negative e ects on the subjective gaming experience.Our work is in line with both ndings regarding performance and experience.In our work, players had a signi cantly reduced accuracy, total amount of bot kills, and a worse kill-to-death ratio, while simultaneously deriving a signi cantly lower quality of game experience when playing with high base latency.In line with previous work, we argue that the lack of responsiveness of the game induces the degradation of performance and experience.Playing with a high base latency led to a discrepancy between in-and output and, thus, to a decreased performance and experience.
To answer 2 , our analysis of the in uence of latency variation yielded no signi cant results.Using a Bayesian approach, we found up weak to moderate evidence that latency variation does not alter performance and experience.Our ndings, thus, are opposite to previous work investigating the e ects of network jitter on player performance and game experience.For example, Amin et al. [3] found that network jitter of 100 ms -which is comparable to our variation of ±50 , signi cantly increases task completion time in video games, compared to playing without jitter.Similarly, the authors found that network jitter also signi cantly decreases the overall gaming experience.Since, we were not able to replicate those ndings using local latency jittering, our ndings indicate that local and network-based jitter manifest their e ect on performance and experience fundamentally di erently.This is in line with previous work by Liu et al [48], who showed that local and network latency, generally, in uence player performance di erently.
Regarding 3 , the interaction between base latency and its variation, an inferential analysis suggests that all measures except one subscale of the PXI are una ected by the interaction.A Bayesian analysis consolidates this nding with weak to moderate evidence.The signi cant interaction e ect for the PXI's meaning subscale is supported by the Bayesian analysis with moderate evidence.As the evidence we found regarding the interaction e ect is rather weak, additional work is required to further investigate this e ect.However, the interaction between base latency and latency variation is likely to have very little relevance in most practical use cases.

Limitations and Future Work
While we found that latency variation does not a ect video game players, our study still has some limitations.Firstly, our sample of participants only partially represents the population of interest.Besides the strong gender imbalance among our participants, most of them were computer science students and, thus, not representing a high level of diversity.Future work, should aim to further generalize our ndings by investigating short-term latency variation with a more diversi ed participant pool including players from di erent ages, educational levels and cultural backgrounds.
On the same note, as latency is especially interesting in the context of e-sports, future research should rigorously control for the participants' skill levels.While we asked participants in our study to self-rate their skill level in playing FPS games, we did not quantitatively asses their actual skill.Self-rated assessments are heavily biased and typically not highly reliable.The participants' skills are of particular interest, since previous work indicates that more experienced players are more likely to perceive small di erence in latency [46].Hence, it is possible that latency variation, as investigated in this paper, does a ect expert players, but not players with a lower level of gaming skill.Furthermore, it is also possible that the player's individual skill level not only alters their performance, but also the felt gaming experience.Previous work indicates that players that perform better in a video game experience a higher level of enjoyment, and thus a higher level of gaming experience, compared to players performing worse [36].Hence, future research should either control player skill more strictly or measure a reliable metric to use it in the work's statistical analysis.
In addition, it is important to recognize our study's limitation presented by the sample size.With a sample of only 28 participants, there are limits to statistical power and precision.Although previous research suggests that latency e ects can be detected even with a smaller sample size, the limited number of participants in our study may make it di cult to identify small e ects.Therefore, future studies should investigate local latency variation using a larger sample to improve generalizability, reliability, and the probability to cleary detect e ects.Hence, future work should perform an a prior power analysis to ensure a certain power level (e.g., 1 − > .90[19,20]) to detect a certain e ect (e.g., 2 > 0.2).
Moreover, we only tested the e ects of varying latency for a FPS game.However, latency's e ect strongly depends on the game's genre [10].For example, in ghting games like Street Fighter, frame-perfect input is necessary to perform certain actions, such as blocking opponents' attacks, or successfully performing a combo.With a time window of only 16.67 milliseconds (assuming a 60 Hz game loop), latency jitter might a ect such actions much more than ghting a bot in a shooting game.Therefore, our ndings can not yet be generalized to the broad landscape of video game genres as further studies are needed.Hence, our study could be replicated with other games to learn about the in uence of latency jitter in di erent genres.

CONCLUSION
In this paper, we presented the results of a study (N = 28) investigating the e ects of local latency jitter on player performance and game experience in a FPS game.Participants played with two levels of mean base latency and two levels of latency variation.
Our work contributes to the extending body of work showing that a high latency reduces player performance and game experience.Furthermore, we highlight that latency's variation as an standalone factor does not signi cantly in uence video gaming session.A Bayesian analysis found that latency variation, as well as the interaction between base latency and latency variation, do not alter performance and experience.
Overall, our ndings can be seen as a sigh of relief for all past and future latency research, as the often overlooked factor of latency variance seems to have little to no practical relevance on the outcome of latency studies.Thus, our work illustrates that previous ndings may not be confounded by not factoring in latency variation.Nevertheless, we recommend that researchers accurately measure and report the latency and latency variation of their apparatûs for better replicability.Additionally, further research is required to investigate under which circumstances and to which degree latency variation can a ect users in di erent scenarios.

Fig. 1 .
Fig. 1.Shows two screenshots from the first-person shooter game Cube 2: Sauerbraten.The le depicts the player's viewport while playing.The screenshot shows the player's weapon, health and ammunition.The right shows an aearial view of the in-game map Teahupoo which was used for all gaming rounds.

Listing 1 .
Pseudo code for the delayed input events.f u n c d e l a y e d _ e v e n t ( d e l a y _ t i m e , keycode , v a l u e ) : w a i t ( d e l a y _ t i m e ) e m i t ( v i r t u a l _ i n p u t _ f d , keycode , v a l u e ) l o o p : keycode , v a l u e = r e a d ( i n p u t _ f d ) d e l a y _ t i m e = random_uniform ( m i n _ d e l a y , max_delay ) t = t h r e a d ( d e l a y e d _ e v e n t , d e l a y _ t i m e , keycode , v a l u e ) t .s t a r t ( )

Fig. 3 .
Fig. 3. Depicts boxplots of Hitrate (le ), KD-Ratio (center), and TotalKills (right) for each combination of B and V. Participants performed significantly be er when playing with low L than with high L .They were more accurate, had a be er kill-to-death ratio, and killed more bots overall.There was no e ect of latency variation on any of the dependent variables in both B latency conditions.
E a s e o f C o n tr o l G o a ls a n d R u le s C h a ll e n g e P r o g r e s s F e e d b a c k A u d io v is u a l A p p e a l M e a n in g C u r io s it y M a s te r y Im m e r s io n A u to n o m