Keywords

1 Introduction

As new video game console hardware is released, game developers have access to new types of input mechanisms. In this paper the use of the touchpad integrated into the game controller is investigated. A touchpad on the game controller is available on the standard Ouya controller, Dual Shock 4 controller, and on the planned Steam Controller. With touchpads becoming a standard interface mechanism on game controllers, it is important that their usability is understood.

1.1 Game Consoles

Ouya [8] is a micro game console released in 2013. Ouya runs Android as its base operating system, which should allow for simple methods of porting mobile games, or may even allow some mobile games to run unmodified. The micro console is relatively inexpensive ($99) and comes with its own controller. Ouya’s controller contains a directional pad, four action buttons, two analog sticks, two analog shoulder buttons, as well as a single touch touchpad centered on the top of the game controller. By default this touchpad is represented as a mouse on an Android device which allows games designed exclusively for touch to have some methods of interaction on the micro console without modifying the source code of the original game.

The Dual Shock 4 controller is the standard controller that ships with the Playstation 4 [9] game console. Like the Ouya controller, it contains a directional pad, four action buttons, two analog sticks, two analog shoulder buttons, as well as a single touch touchpad centered on the top of the game controller. The touchpad on the Dual Shock 4 controller has an expanded feature set when compared to Ouya’s controller. The Dual Shock 4 is multi touch capable, and it also contains a physical delineation of the location of the touchpad and it is a movable surface. That is when the touchpad is clicked, it can physically move downward.

The Steam Controller [11] is a future product to be released by Valve for use on Steam Machine systems. Steam Machines will allow players to play PC based games through a more traditional console gaming experience. Valve has not committed to releasing a version of a Steam Machine, however they have committed to releasing a controller that can be used with their software running on Steam Machines created by third parties. The Steam Controller contains four action buttons and two shoulder buttons. The standard analog sticks have been replaced with two track pads (one for each thumb). Similar to the Oyua controller and the Dual Shock 4 controllers, the Steam Controller contains a touchpad centered and near the top of the controller.

1.2 Related Work

Not many games have made use of the touchpad feature of the new consoles [5]. This research seeks to discover uses for the touchpad or possibly uncover why its use is so rare. The research space handling fine motion controls is defined by Fitt’s Law as shown in ISO 9421-9 [1]. This standard puts forth requirements for non-keyboard based input devices and also shows a user-based performance test. ISO 9421-9 has been used in analyzing different methods of video game input [7] and human computer interaction [6]. Natapov [7] looked into the performance differences of a Wii Remote, a standard gaming controller, and a computer mouse for a 2D target selection task. The Natapov study used mouse input as a baseline and then compared the two gaming control input options. Their study found that mouse based input performed better than controller or Wii Remote based input. This study will use the Natapov study as a baseline and draw parallels between Ouya’s standard controller and the controller used in in that study before investigating new control mechanisms.

Throughput has also been shown to be an effective method of comparing pointing based interfaces as shown in [4]. Mouse down based selection tasks were shown to have higher performance than tasks based on mouse up events. As a result of this, the design involves touch down based selections. Throughput will be calculated for all fine motion selection tasks to ensure confidence in the experimental setup.

1.3 Contribution

This paper presents two different user studies investigating the use of the touchpad on game controllers. Section 2 compares different touchpad control methods with standard thumbstick controls by investigating their usage in a game environment. Section 3 compares 2D target acquisition using different game controller touchpad hand positions.

2 Use of Touchpad in Games

In order to test the use of the touchpad on a game controller, a game originally created for a mobile Android phone was ported to be used on the Ouya console. The original mobile game was written in CoronaSDK [3] and was chosen as it had features for testing the touchpad interface, as well as it could be easily ran on the Ouya console due to CoronaSDK supporting the console directly. The game was a top down driving game where the player was in control of a garbage truck. The truck would automatically move forward, and the controls for the player were to steer the truck left or right to pick up bags of garbage located on the right and left sides of the street. At the conclusion of the round, players could win bonus points by selecting green garbage bags from a grid shown on the screen. This game was chosen to test touchpad controls because it offers both gross motion controls (steering the truck) and fine motion controls (choosing a bonus garbage bag).

2.1 Interface Design

Three different input methods were identified (mouse like, thumbstick, and mobile like) to control the game play aspects and they will be described in detail below. A user study was performed to determine which input mechanism has the highest performance.

Looking at control options to handle the large motions required to steer the garbage truck, results were utilized from a study investigating different types of control mechanisms for different viewpoints of driving games [2]. This study showed thumbstick control of an overhead view (similar to this game) outperformed first person or third high perspective which makes thumbstick control a viable option for this style of game.

2.2 Mouse Like Interface

Easiest to implement, the mouse like interface was the result of simply rebuilding the game to run on Ouya. In this interface, players could control the truck by using Ouya’s touchpad like a mouse on a laptop. Finger swipes would move a pointer on the screen, and when pressed a touch down event was triggered at the location of the pointer. A touch release event was triggered at the pointer’s position on the screen when the player removed his finger from the touchpad after pressing down. Simply swiping the finger across the touchpad had no effect on the gameplay as it only moved the pointer across the screen. After pressing and holding on the touchpad, touch moved events were also registered.

When the round was over and the player was prompted to select trash bags in order to earn bonus points, the player would move the pointer to the desired target by swiping across the touchpad which moved the mouse pointer on the screen, then pressing down to select the target. This is much like using a touchpad on a laptop to select a target.

2.3 Thumbstick Interface

A more traditional console experience was available due to Ouya’s controller. A thumbstick control mechanism was implemented where the truck’s navigation was controlled by moving the left thumbstick left and right. Pressing the thumbstick left would move the truck to the left, and pressing right would move it to the right. The controls appeared intuitive and simple to use.

An issue arose at the completion of the round where the player had to select different trash bags to earn bonus points. In the original mobile version of the game, players could make their choice by simply tapping on the screen where their desired bag was located. In the mouse like interface, players would scroll the mouse pointer over to the location of the bag to select and then tap on the touchpad. In the thumbstick version, there was no method for the player to select a bag as there was no pointer. To make that portion of the game playable, a highlight method was implemented, where the selected bag was highlighted by a semi-transparent glow. The player could move through the different choices by moving the thumbstick to the left or to the right and the highlight would follow. When the player had the desired bag highlighted, he would press the O button on the controller to select the bag.

2.4 Mobile Like Interface

With all the control options available on Ouya, a new type of interface was utilized. In this method, the direction of the truck was determined by the direction of the touchpad swipes. That is, as the player moved his finger across the touchpad on the controller, the truck would follow. When the player stopped moving his finger, the truck would stop. This was all done by scrolling with the touchpad, there was no need to press, and then move as required by the default mouse actions. This allowed players a more standard mobile touch interface on Ouya. Several rail shooters exist where swiping will move the player to a different rail, and although the game did not have predefined rails, the truck would move in the direction the player swiped. One downside to this method was that the mouse pointer was still visible on the screen the entire time, although it had no effect on the gameplay. This could be distracting to players.

At the end of the round, it would be possible to use the mouse like interface in order to select a bag, however that may have been confusing to players. In the regular gameplay the pointer icon on the screen had no effect on the game, and then changing it to have some meaning in the bonus selection may be confusing. It was decided to not have the pointer have any effect on the controls. In the bonus selection round, the same highlight method described in the thumbstick section was used, except for the moving of the highlight was slightly different. Moving the highlight was done by swiping the finger across the touchpad, just as the steering the truck was done in this section. Once the player highlighted the desired bag, he would need to tap anywhere on the touchpad to make the selection.

2.5 User Study

In order to determine how different touchpad input mechanisms compare to thumbstick input on a 2D driving game, a user study was devised to play the three different modes of gameplay. The user study is to identify which methods of input work best on gross motion conversions (steering the truck) and which methods work best on fine motion conversions (choosing the bag at the end of the round). Players were randomly recruited for this study and all had no self-identified impairments or any kind of disability that would prevent the use of a mobile device or controlling Ouya through its controller. The subjects played the game in isolation with only the administrator of the study present.

For the user study, the game consisted of three 50 s rounds. Each round consisted of a different control mechanism, and contained the same number of bags, however the side of the street the bags were on was a random sequence. The game randomly chose the sequence of the three methods for each player. Prior to beginning each round, the game would display a message indicating which mode the player was about to play. The administrator also gave a quick demonstration of the method of play. When the player was confident the control mechanism was understood, he was given the controller and pressed the O button in order to initiate play. Each collected bag in the steering portion of the game was awarded points. Although the bags were presented at a consistent rate, players were awarded a score for each bag based on their ability to steer the truck to the correct side of the street. The quicker the truck was in the correct position to collect the bag, the more points the player received. This created an environment where the player had the desire to quickly use each method in order to obtain a high score.

The bonus bag selection portion of the game presented players with a 4 × 4 grid of garbage bags with five of them randomly colored green. The player would need to select all the green bags in order to complete the bag selection portion of the game. The quicker the player selected the bags, the more points he would be awarded. The game logged the location and time of the selections. The distance from the previous selection was also logged. All player interactions were logged with millisecond accuracy and were used in measuring the different interaction techniques.

The bags of garbage were presented to the player on either the right or left side of the screen in a random sequence. In order to have an element of unpredictability, it was possible for two or more sequential bags of trash to appear on the same side of the road. This could lead to an artificially lower reaction time as the player would not need to react to the situation. To account for this in the data analysis, the data for any bag that appeared on the same side of the screen without the player moving was not considered a qualifying bag and the data associated with that bag was not used in the calculations for this study.

The user study consisted of 15 graduate and undergraduate students playing through three levels. Each level provided the opportunity to collect 10 bags of trash in the steering portion of the game and requiring five bags to be selected in the bonus portion of the game. Players had five seconds to react to a bag coming down the street. If the bag was completely missed by the player, it was counted as a five second reaction time in the analysis. The horizontal rate of travel was consistent across all control mechanisms.

2.6 Technical Evaluation

The time required for a player to either successfully or unsuccessfully capture a qualifying bag was averaged for each player for each control mechanism. The results of these sets of data was averaged to determine the better control option. The average seek time for the controls are shown in Table 1. An analysis of variance (ANOVA) showed significant difference between the control methods for seek time F(2,42) = 42.596, p < 0.01. A post hoc Tukey test at the 95 % confidence level revealed that the Thumbstick controls were significantly better than either of the touchpad controls (p < 0.01).

Table 1. Comparison of large motions
Full size table

In addition to timing, how successful the player was in actually grabbing each bag was analyzed. The results are summarized in Table 2. An analysis of variance (ANOVA) for miss rates showed a significant difference, F(2,42) = 25.21, p < 0.01. A post hoc Tukey test at the 95 % confidence level showed the Thumbstick control significantly outperformed both methods of touchpad control (p < 0.01). The ranking of all of the interaction techniques was consistent for the large motion steering portion of the game with thumbstick control being the most successful and mouse and mobile control trailing.

Table 2. Comparison of large motion miss rates
Full size table

Small motion controls followed a similar trend with thumbstick control being the most successful followed by mouse and mobile controls. To evaluate small motions, players were to select five green highlighted bags on the screen from a grid of 16 total. In mouse like controls, players were able to directly move the mouse pointer on top of the desired bag. The other two controls featured a semi-transparent glow highlighting the bag to select. In the mouse like control mechanism it was possible for players to make a selection that was not on any target. In this case there was no penalty. Similarly, in the other two control mechanisms, players were always selecting a bag, however it was possible to select a bag that was not shown in green. In either case players were not penalized for making an incorrect selection.

The normalized time – taking into account distance - required to select each green bag was recorded and saved for later analysis. An analysis of variance (ANOVA) showed significant variation on this data F(2,42) = 42.75 p < 0.01. A post hoc Tukey test with a confidence level of 0.95 showed the Thumbstick control was significantly better than both of the touchpad control interfaces (p < 0.01).

The Natapov study [7], compared the performance of mouse based input, Wii Remote based input and standard gaming controller based input using throughput as a metric. Throughput was calculated for the three control mechanisms and summarized the findings in Table 3. The throughput values in bits per second were calculated according to the mean of means equation [10]. Comparing the throughput for the standard control mechanism, Ouya’s thumbstick controls were found to have a throughput of 1.33 bits per second. The Natapov study found the standard gaming controller to have a throughput of 1.48 bps, compared to a mouse which was 3.78 bps. A different study [4] found throughput for mouse down selections to be 4.71 bps. The similar throughput for controller based interactions between this study and the Natapov study demonstrates the experimental setup is accurate.

Table 3. Throughput
Full size table

The mouse like controls using Ouya’s controller were found to have a throughput of 0.65 bps which is much lower than other studies found using a standard computer mouse. Although using the touchpad on Ouya as a mouse may be expected to have a similar throughput value, using the thumbs instead of the index finger may have had an influence on the results.

2.7 Game Interaction Recommendations

Although the game was quickly ported to run on Ouya, several issues were found when implementing different control mechanisms. In order to properly use the touchpad in a gaming environment, the following suggestions are made:

(1) Hide the cursor unless it is useful - Players in this study would focus on the visible mouse pointer even though it had no effect on certain control mechanisms.

(2) Return actual press locations - In order to better mimic touch based devices, the underlying drivers should always return the touch events including pressed, released, and moved in touchpad coordinates as opposed to screen mouse cursor coordinates.

(3) Improve on fine movements - Ouya’s controller appears to have issues with small movements on the touchpad. If a touchpad is to be used for fine movements, it should be able to report small movements from the player to the game layer such that players can receive immediate feedback on their motions.

(4) Add tactile feedback - Ouya’s controller may benefit from additional pieces of tactile information. A physical border around the active touch area may assist players in properly placing their thumbs on the touchpad.

3 ISO 9241-9 User Interface Study

To determine how hand position affected the usability of the DualShock 4 touchpad, a second user study was created where participants were asked to go through a series of 2 dimensional target selection tasks using four different hand positions. Hand positions have been investigated in different touch applications [12], however this study looks specifically at touchpad interfaces on game controllers. For this portion of the study, the Dual Shock 4 controller was used as it addresses some of the issues found in the previous game study using the Ouya console. The Dual Shock 4 contains tactile information about the bounds of the active touch area as well as a movable region that gives tactile acknowledgement of a successful click. In order to better represent how using a controller may work in a gaming environment, some of the exercises required players to also use the physical buttons as input.

The four hand positions were:

(1) Finger Press - Index Finger to move pointer and target selection by pressing the touchpad

(2) Finger Button - Index Finger to move pointer and target selection by pressing the X Button

(3) Thumb Press - Right Thumb to move pointer and target selection by pressing the touchpad

(4) Thumb Button - Right Thumb to move pointer and target selection by pressing the X Button

Users were presented with four different selection tasks for each of the hand positions as defined by the following:

  1. (1)

    15 Large targets around a large circle

  2. (2)

    7 Large targets around a small circle

  3. (3)

    13 Small targets around a small circle

  4. (4)

    23 Small targets around a large circle

The sequence of the selection tasks was always the same, however the sequence of hand positions was randomly chosen each time the game was run. At the conclusion of all four target selection tasks for all four hand positions, the user was presented with a score and a game over message. All participants in the study performed all 16 task combinations twice.

3.1 Target Selection Software

The user interface application software was created in Unity3D and handled the presentation of the targets, the random selection of input types, as well as a scoring mechanism. Although not directly relevant to the outcome of the user study, a score was given that represented how quickly the participant selected the targets. This was done to give motivation to the participants to select targets as quickly as possible. The score for each target selection began at 20,000 points, and one point was subtracted for each millisecond that it took for the participant to successfully acquire a target. If a target selection took longer than 20 s, the player was awarded 0 points for the selection of that target. Using this mechanism, a higher score was indicative of a better performing user. Players were notified of the highest score prior to beginning the first trial, and prior to beginning the second trial they were notified of the highest score as well as their previous score. Familiarity with the tasks and motivation to at the very least beat the player’s own trial one score should increase the scores, or decrease the time required to select targets in the second round.

The application followed the patterns set forth within ISO 9241-9. Targets moved around circle with targets alternating roughly 180 degrees apart. For example, the first target was located at the 12 O’clock position, the second target would be located at the 6:30 position, the third target would be located at the 1 O’clock position, the fourth target would be located at the 7:30 position and so on. This pattern of target selection is suggested in ISO 9241-9. Players selected large targets in both a large diameter selection space and a small diameter selection space, and then selected small targets across the same two diameter selections spaces.

15 users (11 male), participated in the user study. 14 of the participants reported they were right handed and the average age was 29.67 years (SD = 6.71). Six of the participants preferred to use a touchpad over a mouse in regular everyday computer use. Participants performed the study in a quiet room with an administrator present, and completed both sessions of the 16 input combinations and target sizes in one sitting. At the conclusion of the user study, participants were given a brief written survey.

3.2 Results

The four different target selection tasks were analyzed separately, and each of the control types were compared. All players played through each of the selection tasks twice, and all data for their trials were used. A total of 5760 targets were selected throughout the user study and were used for the quantitative analysis.

In the large target around the large circle task, a analysis of variance (ANOVA) showed significant variation F(3,116) = 4.02, p < 0.01. A post hoc Tukey test at 9\% confidence level showed the only significant difference at p < 0.05 in this target selection task was that the Finger Press selection task was significantly faster than the Thumb Press selection task. All other comparisons yielded no significant difference. In the large target around the small circle task, a analysis of variance (ANOVA) showed no significant variation F(3,116) = 2.52, p > 0.01.

In the small target around the small circle task, a analysis of variance (ANOVA) showed significant variation F(3,116) = 8.17, p < 0.01. A post hoc Tukey test at 95 % confidence level showed the significant difference at p < 0.05 for three hand position combinations. Finger Press selection was significantly faster than both Thumb Press and Thumb Button selection. Finger Button was also significantly faster than Thumb and Tap selection. All other combinations yielded no significant difference.

In the small target around the large circle task, a analysis of variance (ANOVA) showed significant variation F(3,116) = 11.78, p < 0.01. A post hoc Tukey test at 95 % confidence level showed the significant difference at p < 0.05 for four hand position combinations. Finger Press selection was significantly faster than both Thumb Press and Thumb Button selection. Finger Button selection was also significantly faster than both Thumb Press and Thumb Button selection. All other combinations yielded no significant difference.

At the conclusion of their second trial, participants were asked to rank their preference of the four control options, with a value of 1 indicating the worst control mechanism, and a value of 4 indicating the most preferred control mechanism. Overall, participants preferred the control by using the index finger and making selections with the x button. A summary of the results is shown in Table 4. An ANOVA showed significant variation in these results F(3,56) = 6.757, p < 0.01. A post hoc Tukey test at 95 % confidence level showed that for p < 0.01, both Finger control mechanisms are significantly preferred to the Thumb Press control mechanism, and at the p < 0.05 level, the Finger Button control mechanism is significantly preferred over the Thumb Button control mechanism.

Table 4. User preference of control types (Higher is more preferred)
Full size table

Using Fitt’s law, throughput was calculated for the 2D target acquisition test in the same manner as it was for the fine motor control section of the previously described game study. The results of the throughput test using the Playstation Dual Shock 4 controller show that the Finger Press interface combination produced the best throughput followed by Finger Button, Thumb Button, and the worst performing was the Thumb Press. The exact values are shown in Table 5.

Table 5. Dual Shock 4 Throughput (bps). Average is the average of all 4 target selection tasks. Large Circle/Small Target reports only on the last target selection task.
Full size table

4 Discussion

The results shown in this user study present an issue for game developers wishing to use this new piece of game controller hardware. The touchpad performs worse than the standard thumbsticks, and worse than the results of previous studies that looked at the usability of a standard mouse. The first game developed for this user study pointed out the fact that the touch pad was worse than the thumbsticks. The follow up study showed index finger usage was significantly better than using the thumbs. The design of the controllers for both the Ouya controller and the Dual Shock 4 controller allow for natural access to the center touch area by the thumbs, however actually using this method yields poor performance. Using the index finger by itself yields the best performance for 2D target selection tasks which may indicate that the touchpad should only be used for menu access if it needs to be used. Using the touchpad for seldom used tasks such as browsing the internet through a gaming console may warrant the inclusion of this device on every controller, however if a console application such as a web browser is to be frequently used, console owners may prefer the convenience of a dedicated mouse interface.

5 Conclusion

This paper explored the use of a new feature which is becoming common on game controllers, the touchpad. An initial game showed that touchpad based control of a 2D driving game was significantly worse than using the standard thumbsticks. A more thorough investigation of the touchpad found interacting with the touchpad while using the index finger to be significantly better than using the touchpad with the thumbs. This indicates that the use of this type of interface may be best utilized when moving the hand position from the standard game controller grasp will not effect the overall game play experience such as navigating menus, or browsing the internet through a console.