See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/369968942

The Challenge of Evaluating Player Experience in Tabletop Role-Playing Games

Conference Paper · April 2023

DOI: 10.1145/3582437.3582457

CITATION READS
1 189

2 authors, including:

Alena Denisova
The University of York
30 PUBLICATIONS 672 CITATIONS

SEE PROFILE

All content following this page was uploaded by Alena Denisova on 20 April 2023.

The user has requested enhancement of the downloaded file.

 The Challenge of Evaluating Player Experience
in Tabletop Role-Playing Games
Antonios Liapis Alena Denisova
University of Malta University of York
Msida, Malta York, United Kingdom
antonios.liapis@um.edu.mt alena.denisova@york.ac.uk

ABSTRACT computers and heavily influencing digital games [8]. The genre has
Tabletop Role-Playing Games (TTRPGs) offer players the opportu- expanded in terms of game systems and design philosophies since
nity to form imaginary gameworlds and stories within them, create the early inceptions of Dungeons & Dragons [86] in the 1970s. Many
community, solve problems, and explore identity. Designers and modern TTRPGs deviate from the aforementioned play experience,
researchers have tried to identify how aspects of TTRPGs facilitate including games with no Game Master, such as Fiasco [91], games
collaboration, immersion, creativity, and more. However, there has played with online participants [103], games mediated by software,
been no attempt to develop a formal assessment methodology for etc. While dwarfed by the digital games sector, new TTRPG systems,
player experience during TTRPG play. This paper argues that eval- niche settings, and steps towards improving representation and
uating TTRPG players’ experience can provide vital data for Game diversity [10] have stimulated unprecedented growth: in 2020 the
Masters to improve on their future games, for players to reflect estimated TTRPG sales in the U.S. and Canada exceeded a hundred
on their experience, and for TTRPG designers or event organizers million dollars for the first time in history [55]. TTRPGs have also
to collect and compare data. As a first step towards developing received extensive research interest1 with attention to character-
such an evaluation method, we identify important dimensions of izing the genre [62, 63] and associated experiences [17, 76, 90] as
TTRPG play that can be meaningful to track and actionable to im- well as to the impact of TTRPGs in education [30, 74], soft skills
prove upon. Moreover, we review player experience dimensions training [71], personal development [88], and more.
and evaluation methods in digital games, and explore similarities The experience of playing TTRPGs is arguably unique, tracing
and differences with TTRPGs. back to their identity as participatory story-creation systems [83].
The stories formed during a TTRPG are “systemic, improvisational,
CCS CONCEPTS and collaborative” [59]. On the other hand, these systems do include
rules that define the possible actions that in-game characters can
• Human-centered computing → Interaction design; Empirical
take within the story-creation sequence and — importantly — how
studies in interaction design; • Applied computing → Computer
success or failure of such actions is decided. This creates a tension
games.
between the storytelling and gaming aspects of TTRPGs [31]; which
KEYWORDS aspect is favored is as dependent on the chosen TTRPG system as
on the group (especially the arbitration style of the Game Master,
player experience, role-playing games, analog games, evaluation if any). In terms of the gaming experience, players may enjoy or
protocols, questionnaire have a hard time optimizing their gameplay or characters in order
ACM Reference Format: to overcome challenges in the game. In terms of the storytelling
Antonios Liapis and Alena Denisova. 2023. The Challenge of Evaluating experience, players may have a hard time envisioning the scenes
Player Experience in Tabletop Role-Playing Games. In Foundations of Digital described by the Game Master, may enjoy expressing their charac-
Games 2023 (FDG 2023), April 12–14, 2023, Lisbon, Portugal. ACM, New York, ter’s speech style and flair, or may try to push the story towards
NY, USA, 10 pages. https://doi.org/10.1145/3582437.3582457
a specific end-goal. In addition, since TTRPGs are social activities
where players cooperate towards the same story-creation goal, one
1 INTRODUCTION player’s experience can be impacted by other in-game characters’
Tabletop Role-Playing Games (TTRPGs) are a “type of game/game abilities and narrative end-goals, but also by other players’ social
system that involves collaboration between a small group of play- skills and relationships [105].
ers and a game master through face-to-face social activity with the Methods for measuring player experience (PX) have been de-
purpose of creating a narrative experience” [31]. TTRPGs have a veloped both within the game industry and academia, but so far
rich history spanning almost 50 years [112], preceding personal have predominantly focused on evaluating experiences of playing
Permission to make digital or hard copies of all or part of this work for personal or digital games and rarely on TTRPGs [108]. Evaluating TTRPG ex-
classroom use is granted without fee provided that copies are not made or distributed periences in some way would be useful feedback for Game Masters
for profit or commercial advantage and that copies bear this notice and the full citation to improve their session management and refereeing style and for
on the first page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or TTRPG designers to produce more intuitive rules that provide a
republish, to post on servers or to redistribute to lists, requires prior specific permission more immersive experience. Beyond its usefulness to stakeholders
and/or a fee. Request permissions from permissions@acm.org.
in the TTRPG, exploring dimensions of TTRPG experience can
FDG 2023, April 12–14, 2023, Lisbon, Portugal
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
1 Indicatively, the International Journal of Role-Playing has published 12 issues since
ACM ISBN 978-1-4503-9855-8/23/04. . . $15.00
https://doi.org/10.1145/3582437.3582457 2008.
FDG 2023, April 12–14, 2023, Lisbon, Portugal Liapis and Denisova

potentially contribute to our understanding of specific PX theories personal experience. Based on discussions on message boards, indie
and expand the field of PX research more broadly. The unique ex- TTRPG designer Edwards [47] proposed the GNS theory which
perience of TTRPG play as collaborative, emergent story-making distinguishes three types of goals during play: Gamism expressed
necessitates re-imagining certain dimensions of digital game PX in by competition among players (not PCs) by taking advantage of
order to better capture the impact and interactions between rules, the game’s rules, Simulationism expressed by attention to the game-
equipment (e.g. character sheets, miniatures), other players and the world’s internal logic and consistency, and Narrativism expressed
Game Master. In the following sections, we review TTRPG and PX by the creation, via role-playing, of a story with a recognizable
concepts and provide a preliminary set of PX dimensions that are theme. Based on their ruleset and setting design, some TTRPGs
both realistic to assess and useful to TTRPG stakeholders. may appeal more to one player type than another, although often
players can shift between types depending on the game, the GM,
2 WHAT IS A TABLETOP RPG? and the social context. There are several revisions to the GNS the-
Tabletop Role-playing games are shared social activities, usually ory as surveyed by Boss [15]. Ad-hoc typologies and motivation
between a number of players in the same physical space, sitting groupings are often proposed in GM guides, as knowing players’
around the namesake table. Each player controls a player char- preferred activities allows the GM to “tailor adventures that satisfy
acter (PC) and through collaboration with other PCs attempts to [their] preferences as much as possible, thus keeping them engaged”
accomplish a shared goal, overcome challenges along the way, and [32]. Indicatively, Laws [78] identifies eight TTRPG player types
create interesting stories through play. TTRPGs are often formal and provides a fairly well-rounded suite of possible motivations,
systems with published rules, mechanics, settings, and adventures. reactions, and coping mechanisms. The extensive descriptions pro-
A TTRPG session may encompass the entirety of the adventure vided by Laws could be considered (a priori defined) TTRPG play
(one-shot), or may be only one small episode of an ongoing cam- personas [26]. Similar to the GNS theory, Laws identifies the Power
paign. Players decide on their characters’ actions, but usually the Gamer (reminiscent of Gamism), the Method Actor and the Story-
outcomes of these actions are resolved through some stochastic teller (reminiscent of Narrativism, through differing perspectives),
process such as dice rolling, card drawing, etc. In most TTRPGs, and the Tactician (reminiscent of Simulationsism). However, Laws
the gameworld, non-player characters (NPCs), plot, pacing, and also includes player types based on less game-specific motivations,
rule arbitration is handled by a human Game Master 2 (GM) who such as the Casual Gamer (who is less invested in the game and
participates in play but does not control a PC. Other TTRPGs do not more in the out-of-play social bonds), the Specialist (who only
have a dedicated GM, and players share authorial control through plays characters of a specific trope), and the B*-Kicker (who plays
group decision-making towards interesting outcomes, as in Fiasco to blow off steam). Assessing the motivations of TTRPG players
[91], controlling more than one PCs, as in Legacy: Life Among the through a formal instrument similar to those prevalent in digital
Ruins [85], or controlling some or all aspects of the world, as in games [116] can be valuable for the preparation work of a Game
Across the Endless Sea [70]. This paper focuses on PX on the part Master, as evidenced by GM guides that already suggest this based
of players that control a PC. Therefore, the experience of the GM on ad-hoc typologies [78]. However, we consider that evaluating
is not considered as it is contingent on many more factors related PX of a (recent) TTRPG session can have broader and actionable
to system and adventure design, setting description, setup, rule impact than a one-time player profile assignment, and choose to
interpretation, arbitration, moderation, etc. Similarly, this paper focus on the former in Section 5.
does not consider games where players only control an emerging Another dimension of study is the relationship between players
narrative rather than characters within it, for example, storytelling and their PCs. Each PC is inescapably shaped by the player’s person-
games such as Once Upon a Time [75] or The Quiet Year [3]. ality, ethos, experiences, etc. Based largely on participant-observer
ethnographic research, Bowman [17] categorizes the reflection of
3 PLAYERS IN TABLETOP RPGS the player in their characters into nine types, based on the “same-
ness” between a player’s primary identity and their character’s.
While in digital games a gameworld and narrative exists in the
The nine types of “selves” identified by Bowman largely follow a
game’s executable, in TTRPGs a vast portion of the experience is
Jungian perspective; for instance, a PC as Oppositional Self repre-
built by the players themselves — along with the GM. At a high level,
sent the player’s Shadow qualities that have been repressed [52].
players have preferences in terms of genre (e.g. modern, horror,
These concepts are relevant for PX evaluation in TTRPGs when it
science-fiction, fantasy, etc.). Identifying the genre and themes that
comes to character attachment, discussed in Section 5.4.
the group wants to explore, the intended duration of the game, as
well as limits, sensitivities and taboo themes [104], requires that
the group is in constant and honest communication both before
and during a TTRPG session. 4 EVALUATING PLAYER EXPERIENCE
Player types and motivations have been well-studied in the con-
Borrowing from the broader concept of user experience, player
text of digital games, and many instruments have been proposed
experience (PX) aims to describe “the individual, personal experience
for profiling players of different genres, e.g. [92, 102, 116], including
held by the player during and immediately after the playing of the
computer RPGs [45] specifically. However, research into types of
game” [113]. Due to the differing goals of a game (to entertain,
players in TTRPGs is typically based on loose methodologies or
engage, etc.) compared to a productivity application or a website,
2 Notethat different games denote this person as Dungeon Master, Storyteller, Master and the different emotions that games elicit, “conceptualization
of Ceremonies, etc. of player experience requires differentiating specific dimensions like
The Challenge of Evaluating Player Experience in Tabletop RPGs FDG 2023, April 12–14, 2023, Lisbon, Portugal

(game-) flow, immersion, challenge, tension, competence, and emo- [5, 102, 117], immersion [68], engagement [20], flow [67], spatial
tions” [113]. Since both user experience and PX originate from the presence [110, 114], social presence [37, 65], and overall gaming ex-
Human-Computer Interaction (HCI) discipline, PX conceptualiza- perience [1, 66]. Specialized questionnaires assessing more nuanced
tion and PX evaluation has focused on digital games as its applica- PX include challenge [42], demand [16], attribution of failure [44],
tion domain. However, we contend that many of the assessment character attachment [81], character morality [54, 69], player-avatar
methodologies and dimensions of PX can and should be transferred interaction [6, 7], creativity [58], embodiment [95], uncertainty [98],
— with adjustments — to TTRPGs. Generic questionnaires have fantasy [27, 97], and more. Analyzing these experiences is beyond
been used in TTRPG research to ascertain differences between e.g. the scope of this paper; however, we will focus on the relevant PX
players and non-players [105] but not for the goal of assessing PX. dimensions and evaluation instruments for them when proposing
We should note that analysis of PX in analog games (board games possible directions for TTRPG PX in Section 5.
and TTRPGs) has been attempted so far via interviews [51, 106],
participatory observation [11, 17, 34, 50, 83], and thematic analysis
of forum discussions and message boards [2, 49, 73]. However, a 4.2 Why evaluate PX in TTRPGs?
standardized methodology and instrument for evaluation is lacking While there has been extensive research in defining or explain-
in analog games and TTRPGs, in particular. ing the experience of role-playing in TTRPGs [63, 90], the focus
To the best of our knowledge, the only published work on evalu- of such work has been on identifying components of the experi-
ating TTRPG experiences is by Tychsen et al. [108] in 2008, which ence [15, 47] or designing around and towards it [25], rather than
included an extended questionnaire on the FUN construct [93] operationalizing the experience itself. We identify that methodolo-
as well as questions on Group Dynamics and Player-Character gies and instruments for evaluating TTRPG PX can be valuable to
Relations. The questionnaire was applied, with modifications, to many stakeholders: PX researchers, Game Masters, TTRPG players,
compare experiences between TTRPGs (specifically, Dungeons & designers, and event organizers.
Dragons) and Computer RPGs (specifically, Neverwinter Nights). Given the extensive research conducted in operationalizing PX
While some of the PX dimensions evaluated by Tychsen et al. [108] in digital games, investigating PX in a new, arguably unique type
are close to our suggestions (see Section 5), this questionnaire was of gameplay in TTRPGs can lead to new insights and inform PX
not considering the needs of TTRPG stakeholders (as discussed research as a whole. While TTRPGs share common concerns that
in Section 4.2) and was not validated beyond the study itself. In have been well-studied in digital games (as discussed in Section 5),
addition, more recent work on PX evaluation in digital games could the medium and the experience itself is vastly different. The experi-
provide a more informed view of TTRPG experience beyond the ence is shaped by the presence of a Game Master and other players,
constructs of Newman [93]. Recent research on eudaimonic ex- the freedom afforded by the game’s fabula and loose ruleset, and the
periences [28, 29, 33] considers more nuanced and subtle player reliance on imagination and theory of mind rather than audiovisual
experiences ‘beyond fun’, such as emotionally moving [13], emo- feedback and discrete controls. Moreover, the uncontrollable nature
tionally challenging [14, 40, 42] and discomforting experiences [53]. of TTRPG sessions challenges traditional methodologies of PX re-
All these could be relevant to how players experience TTRPGs. search, such as controlled experiments. While in digital games the
game can be controlled by modifying the game executable [43] to
produce two or more versions that test a hypothesis on the impact
4.1 Evaluating PX in Digital Games of a dependent variable, in TTRPGs such a level of control is nigh
A plethora of research methods is used in the digital games industry impossible. Working with the medium and investigating how some
for evaluating PX, as surveyed by Medlock [87]. Importantly, differ- control can be maintained through e.g., a script for the GM, could
ent methods are applicable in different stages of game development. shed new light in research methodologies for PX and investigate
One way to assess one’s PX is by looking at the ‘objective’ data the impact of priming [41] on the emerging experience.
in the form of physiological responses from players, such as heart On the other hand, Game Masters already assess their group’s
rate or skin conductance. The downside of this approach is the lack experience throughout the course of play and explicitly during
of subjective context, i.e. why participants are feeling something the debriefing that follows after a TTRPG session. Usually, such
and what it is that they are experiencing or thinking exactly. To debriefing comes naturally, with players chatting around the table
address this shortcoming, qualitative evaluation methods can be about the session and their plans for next time. GMs may explicitly
used to complement these ‘objective’ responses, including inter- ask for feedback when trying something new, or use ad-hoc surveys
views, focus groups, and ad-hoc surveys; however, results from during campaign preparation to gauge what the players would like
these methods can lack standardization and comparability. Vali- to play next. Recently, a ‘standardized’ checklist for consent in RPGs
dated questionnaires, on the other hand, exist to address this issue. has been put forth by Monte Cook Games [104], which can help
Questionnaires are perhaps the most common method for as- GMs and players steer away from themes, stories, or actions that
sessing subjective experiences of playing digital games. These in- could make some players uncomfortable. In the same book, GMs are
struments can quantify experiences and compare these experiences advised to check-in with their players after each session to “discuss
between groups of players or between sessions. Questionnaires are anything that was emotionally charged or potentially problematic
usually created based on a specific theory with a view to test and in the game” [104]. Therefore, debriefing is ubiquitous and serves
refine this theory and to be able to compare experiences across multiple purposes including assessing — informally — the recent
different games, features, and player types. The most common ex- player experience. While a formal PX assessment may not fit the
periences that are measured through questionnaires are motivation relaxed style of many gaming groups, it could be a valuable tool for
FDG 2023, April 12–14, 2023, Lisbon, Portugal Liapis and Denisova

GMs to collect data on aspects they could improve in future sessions explanations of how GMs actively attempt to influence each PX
of the same campaign, or in new games. In addition, explicitly aspect. Each section also build bridges to existing instruments for
responding to questions could help players identify certain aspects evaluating similar experiences in digital games. As noted, the pro-
that they could also improve on (such as group dynamics), or simply posed directions form an initial set that should be substantiated
prompt them to reflect on their experience [79, 99]. and refined based on interviews with stakeholders (see Section 6).
Another obvious benefit of a methodology and/or instrument
for evaluating PX in TTRPGs is for designers of such games. In 5.1 Cognitive Challenge
digital games, most applications of PX evaluation are intended TTRPGs challenge the player’s mental resources in multiple ways.
for playtesting games [46]. Like digital game developers, TTRPG Players have to:
designers must consider how their rulebooks are interpreted on the
gaming table. Methodologies for running playtests can help TTRPG • estimate the optimal strategy for addressing a challenge,
designers collect better data on their games, including unclear based on the ruleset’s action and resolution system (strategy)
rules in the rulebook, edge cases, or how the game’s narrative or • keep track of the current state of the game, since it only
rules could lead to unintended play styles or disruptive behavior. resides in the GM and players’ mind (cognitive dissonance)
Developing a standardized method for evaluating PX can allow • keep track of the gameworld and the story so far (memory)
TTRPG designers to gathering large volumes of data fairly easily Strategic thinking in TTRPGs is largely dependent on the rules of
and quickly. The use of PX questionnaires ensures consistency and the game; action resolution may range from rolling a six-sided die to
uniformity of collected data, because the same specific aspects are consulting several tables and applying contextual modifiers. In most
considered by all participants in all studies. As a concrete example, TTRPGs, different character roles may be better suited for specific
Paizo released multiple early versions of their Pathfinder system types of actions, or may have additional actions available (e.g. spells).
[23] in 2008-2009 for public playtesting and solicited responses via More rigid systems with rules to simulate the world and decide on
the company’s forum. The response was enthusiastic, with over the exact outcomes and probabilities of success, such as Pathfinder
45,000 downloads of the rules and over 100,000 forum messages [22]. [12], allow the player to strategize, assuming their mental model of
In such a large-scale, multi-iteration playtest, a standardized TTRPG the game state matches that of the GM (more on this below). More
PX questionnaire (deployed online) could have collected more data freeform systems tend to have simpler resolution methods but give
and allowed for comparisons in terms of responses depending on more leeway to the GM to decide on an appropriate outcome, e.g.
the version of the system played. in Blades in the Dark [60], and therefore it is more difficult for a
Organizers of ‘organized play’ TTRPG events can similarly ben- player to estimate the outcome of their actions.
efit from easily deployable PX evaluation instruments. Such events Players’ decision-making also hinges on the mental model of
are often played on-location at conventions such as DragonCon3 in the game state that the entire group shares. Different TTRPG sys-
the USA, and feature many groups at different tables simultaneously tems have different material components to assist in this effort:
playing sessions on the same game system. For local organizers of miniatures, grids, maps, handouts, player or GM aides, software,
such events, having some way of assessing participants’ PX can help etc. Even the most intricate material setup, however, can still lead
them plan more effectively for future events, as well as for training to cognitive dissonance between what the GM and each player
or vetoing GMs. In addition, some of the major TTRPG publishers understands about the situation they are in: for example, some
have programs around international organized play. Indicatively, players may consider that intimidating a king would be effective
Paizo Organized Play 4 releases new campaigns with multiple adven- while the GM assumes that it is punishable by death. Minimizing
tures every year, where outcomes of individual groups’ adventures cognitive dissonance can be achieved through clearer communica-
feed into the narrative of the next year’s events. While the main goal tion between GM and players, such as the GM describing the scene
of these programs is to promote the publisher’s product, it could be thoroughly, and noting possible repercussions of each action.
beneficial to track not only players’ progress in the adventure but Finally, keeping track of a group’s broader history (rather than
also of their experience while doing so. Tracking this experience on the immediate surroundings) is done through players’ note-taking,
an international scale can act as an in-the-wild playtest that could e.g. writing down NPC contacts, unresolved goals, visited locations,
lead to corrections in the rulesets and the adventures. etc. This is largely dependent on the players and their personality,
but different types of games might rely on — and penalize lack of
5 INITIAL DIRECTIONS FOR PX EVALUATION — player memorization more than others (e.g. mystery TTRPGs).
IN TTRPGS The GM can also help with players’ memories through handouts
ostensibly given to the characters (such as maps or long messages)
While methods for evaluating TTRPG PX can be valuable for many
and by recapping the story so far at the beginning of each session.
stakeholders (see Section 4.2), this first paper focuses on the needs
Cognitive challenge has been highlighted in several question-
of Game Masters based on over 20 years of personal experience. We
naires: the Challenge Originating from Recent Gameplay Interac-
presume that PX evaluation is carried out after a TTRPG session,
tion Scale (CORGIS) [42] addresses it by name as challenge that
by participating players, as part of or complementary to debriefing.
arises “from the need for preparation, planning ahead, memorisation,
The following sections identify aspects of the player experience
effort and multi-tasking” [42]. Questions on cognitive challenge
that would be valuable (and actionable) to evaluate, and include
in CORGIS are not tied to the digital game medium and explicitly
3 https://www.dragoncon.org/things-to-do/gaming/#rollplaying address planning, general effort, memorization, preparation and
4 https://paizo.com/organizedplay more. Similarly, the cognitive demand scale of Video Game Demand
The Challenge of Evaluating Player Experience in Tabletop RPGs FDG 2023, April 12–14, 2023, Lisbon, Portugal

Scale (VGDS) [16] is associated with the consumption of cognitive avatar and gameworld. A recent paper conceptualizing immersion
resources in relation to direct and purposeful thinking, and can thus in board games [49], which highlights the key differences in this ex-
be another starting point. A more targeted aspect of TTRPG PX is perience when compared to digital games, could be a good starting
the stochastic nature of action resolution, which makes planning point to create a tool that assesses PX specific to TTRPGs.
and decision-making more difficult. The Perceived Uncertainty in
Games Scale (PUGS) [98] captures this via the Exploration, Prospect
and Randomness factors, although the Disorientation factor seems
more targeted to a digital play experience. Finally, the spatial pres- 5.3 Agency
ence questionnaire (SPQ) [110] may be a better starting point in Hammer [59] discusses agency (i.e. the ability to take action within a
terms of capturing understanding of the current game state, as it possibility space) and authority (i.e. the ability to enforce and judge
was originally aimed to capture, among others, presence in text the results of those actions) in TTRPGs. The researcher identifies
from books or film via items such as “It seemed as though I actually that while in theory there is no constraint on the possibility space,
took part in the action of the presentation” [110]. Other aspects of due to the freeform and improvisational nature of TTRPGs, there is
the cognitive challenge could be added to the above in order to both a fabulaic limit (due to genre conventions or intended narrative
better capture the often longitudinal, episodic nature of TTRPGs outcomes) and a zero-sum limit (due to collaborative play). While
(e.g. as questions regarding remembering the gameworld and story the latter is also pertinent to group dynamics around the table
in-between sessions) or its material components. (see Section 5.5), how much control the player perceives they have
on their character, the gameworld, and the story are important
factors for their enjoyment and implicitly impact other aspects
5.2 Immersion of the experience, such as immersion (see Section 5.2). Here, we
Borrowing from the definition of board game immersion by Calleja use the term agency to include notions of control, self-expression,
[25], our working definition of TTRPG player immersion is the empowerment, and autonomy [38]. A player may feel their agency
imagined habitation of a mechanically structured, fictional world stifled when e.g. other players take over the game and do not share
through embodiment in a single entity that is able to exert agency in the spotlight (see Section 5.5), when the GM does not allow for role-
accordance with the rules of that world. This is similar to the concept playing opportunities by speeding past them (see Section 5.6), or
of engrossment in TTPRGs proposed by Fine [50] as “the willing, when they do not see any (one or alternative) strategies for reaching
temporary acceptance of a fantasy world and persona as real, indicat- an intended outcome (see Section 5.1). Evidently, the perceived
ing this process as a necessary component to play” [19]. Immersion in control a player has over the game is contingent on many factors;
TTRPGs and live-action RPGs has been hotly debated [61, 64, 111], moreover, some TTRPG systems limit the player’s agency in order
often under the prism of the GNS theory (see Section 3), while in to trigger emotions of helplessness, as in the case of horror games.
digital games it has been central to many proposed frameworks However, the group and the GM can influence how much agency
[21, 24, 56, 82, 101]. Bowman [19] surveyed work on immersion, players feel they have by moderating how the spotlight is shared,
with a focus on computer RPGs, and concluded that “immersion is by setting up interesting role-playing opportunities that allow the
a multifaceted phenomenon with distinct levels of engagement, each player to express their character’s personality, beliefs, or flair, or by
with potentially gratifying elements” [19]. Bowman identified im- allowing players to change the course of an adventure compared
mersion into activity, game, narrative, character, and community; to the GM’s plans (or the book’s prescription).
many of these overlap with dimensions of involvement identified As noted, the term ‘agency’ in TTRPGs in this context is used
by Calleja [24]. Given its multifaceted nature, achieving immersion fairly loosely, because the breadth of actions available to players
on-demand during a play experience is rather unlikely. That said, falls beyond the sheer mechanical actions in a digital game; players’
TTRPGs offer many shortcuts for improving the immersion of par- actions include acting out a scene or even suggesting ideas to the
ticipants, such as addressing the players only with their character GM for an alternative way of resolving a challenge. Perhaps the
names, minimizing out-of-character communication [84], evocative closest mapping can be found with the Autonomy dimension of the
and multi-sensory scene descriptions by the GM, fleshed-out PC self-determination theory (SDT) [38], which has been evaluated
and NPC personalities, gestures and voice-acting, music [48], etc. using multiple PX questionnaires for digital games [5, 102, 117]. For
In digital games, immersion is often evaluated as part of general instance, Autonomy questions in the Ubisoft Perceived Experience
PX questionnaires, e.g. [66, 102], or using standalone question- Questionnaire (UPEQ) [5] are not tied to digital games and are
naires such as the Immersion Experience Questionnaire (IEQ) [68]. general enough to be applicable for TTRPGs. On the other hand,
The IEQ proposes that immersion is made up of five components: agency in TTRPGs is as much about self-expression as it is about
Cognitive Involvement, Real World Dissociation, Challenge, Emo- actions and their effects on the game-state. Therefore, questions
tional Involvement and Control. These factors arguably overlap regarding how much the player could role-play their character are
with many other aspects of TTRPG PX as discussed in this section, important to be added. Unsurprisingly, there is no questionnaire
with Real World Dissociation perhaps being the most relevant for for the experience of a player expressing their fictional character’s
our case. Even so, the IEQ is heavily geared towards digital games, emotions or morals. We can draw inspiration from the Creativity
assuming interaction with a screen and graphics, winning, controls, in Gaming scale [58] or the Creative Freedom factor in the Game
etc. It is likely that TTRPG immersion will need to be measured by User Experience Satisfaction Scale (GUESS) [96]; more ambitiously,
a domain-specific questionnaire that considers the dual nature of we can investigate how other forms of mediated self-expression
player and PC as situated play rather than a graphically depicted are evaluated, such as through art therapy [57].
FDG 2023, April 12–14, 2023, Lisbon, Portugal Liapis and Denisova

5.4 Attachment engage in the TTRPG activity by implicitly recognizing an egalitar-

A well-studied and critical aspect of TTRPGs is the player’s rela- ian structure of power with the other players [2]. In this spirit of
tionship with their character, as noted in Section 3. Identifying as collaboration, players acknowledge that “any participant must cede
the player character is almost necessary in TTRPGs in order to ac- at least some agency to other members of the group, or those other
complish immersion, but the cognitive and emotional shortcuts to members cannot meaningfully participate” [59]. Therefore, usually
achieve this are far from clear. Identification is “a psychological pro- players try to share the spotlight, giving other players leeway to
cess whereby the subject assimilates an aspect, property, or attribute express themselves and role-play their character, attempting to
of the other and is transformed, wholly or partially, by the model the reach consensus when taking an important decision, and generally
other provides” [77], and is very similar to the working definition maintaining a positive atmosphere.
of character attachment by Lewis et al. [81] as “internalization and On the darker side of groupwork, conflicts between players (not
psychological merging of a player’s and a character’s mind”. the PCs) can emerge when there is disagreement on rules, pacing,
A player’s relationship with their character is a personal ac- tactical or story-wide decisions, etc. This dysfunction [47] is usually
cord, likely verging on the subconscious. Therefore, at first glance due to incompatibility between different players’ expectations of
it would be less valuable to assess as it can not be addressed di- the game, as elaborated in Section 3. Another common reason for
rectly by a GM or by TTRPG designers — nor, arguably, even by dysfunction is a lack of shared mental model of the game state
the player themselves. However, at second glance, raising the emo- and gameworld (see Section 5.1) between players, which leads to
tional challenge of the player [42] can elicit feelings of sympathy or confusion and arguments. Difficult situations also emerge when
empathy [94] towards the PC, other PCs, or NPCs that can deepen more seasoned role-players overburden new players by not ceding
the player’s relationship with their character. Emotional challenge agency to others or by acting in a toxic fashion [35]. Clashes can
arises “from the emotions evoked in the player which might also have also be traced to out-of-character events that bleed into the game
implications for things they thought about outside of the game” [42] [18], such as relationship shifts between players, which are more
and is intimately tied to role-playing, as “deciding how the character difficult to mitigate.
should act forms part of the emotional challenge” [42]. The TTRPG Perhaps the most relevant questionnaire for this context is the
ruleset can solicit emotional challenge by codifying deep interper- Social Presence in Gaming Questionnaire (SPGQ) [37], which mea-
sonal interactions and their outcomes, as in Headspace [100], and sures two psychological involvement components (Empathy and
by describing game settings and characters that are emotionally Negative Feelings), and one Behavioural Engagement component.
vulnerable, e.g. in Monsterhearts [4]. The GM can also raise the These questions are not tied to digital games, and could be applied
emotional challenge by setting up situations with moral dilemmas, to TTRPG social dynamics. However, not all feelings captured by
raising tensions between individual wants and group needs, prompt- SPGQ may be relevant around the table (e.g. envy or admiration)
ing the player to role-play an emotional scene such as delivering and should be verified in the context of TTRPGs (see Section 6).
an obituary, make NPCs flawed and vulnerable, etc. Another relevant questionnaire assesses Competitive and Coop-
On this PX dimension, the closest evaluation instruments rel- erative Presence in Gaming (CCPIG) [65], which was validated in
evant to the TTRPG context are the Player-Character Sympathy digital games such as Mount & Blade and Dota 2. The collaboration
questionnaire for TTRPGs and computer RPGs [109] and the ques- questions of CCPIG seem to be a good starting point for TTRPG dy-
tionnaire for measuring Character Attachment (CA) in computer namics, but tend to skew towards goal-driven behavior (especially
RPGs [81]. The CA questionnaire measures four components: Iden- regarding the Motivation factor) which may or may not be cen-
tification/Friendship, Suspension of Disbelief, Control, and Respon- tral to the TTRPG. For instance, tactical games such as Pathfinder
sibility. However, some of these components overlap with other [12] may focus more on goals such as victory in combat; in more
notions such as agency (see Section 5.3) and cognitive challenge socially-driven games such as Monsterhearts [4] the players may
(see Section 5.1) which we consider distinct from attachment. The not consider other players as team-mates (nor competitors, per se).
dimensions of Identification/Friendship and Responsibility, how- Questions on social presence that are less focused on collaboration
ever, seem a good starting point and can be adapted to the types and competition may thus also be relevant, e.g. the social presence
of relationships players tend to have with their PCs as surveyed in module of the General Experience Questionnaire (GEQ) [66]. Relat-
Section 3. Additional inspiration can be gleaned from the Emotional edness in self-determination theory (SDT) [38, 102] is also relevant,
Challenge sub-scale of the CORGIS [42] and Emotional Demand in as it does not focus as much on the task at hand; relatedness in-
the VGDS [16]; while emotional challenge or demand dimensions stead is the “will to interact with, be connected to, and experience
do not map to attachment per se, they provide an important and caring for others” [9]. As mentioned above, multiple PX question-
actionable aspect of PX that forces the player to form a deeper naires assess SDT [5, 102, 117], although they do not address the
emotional connection with their character. dichotomy between player and PC. Another inspiration could be
the questionnaire on RPG group dynamics by Tychsen et al. [108].
Beyond the above dimensions, however, there are additional com-
ponents regarding emotion regulation and conflict resolution that
5.5 Group Dynamics would be valuable to be assessed in order to reflect on and address
While not all TTRPGs are group activities [80], a core aspect of player conflict. Good starting points for such evaluation could be
most TTRPGs is the “face-to-face social activity with the purpose conflict management questionnaires [115] or emotion management
of creating a narrative experience” [31]. Social dynamics are impor- questionnaires from psychology, e.g. the perspective-taking scale
tant factors that impact individual players’ experience, and players of the Interpersonal Reactivity Index (IRI) [36].
The Challenge of Evaluating Player Experience in Tabletop RPGs FDG 2023, April 12–14, 2023, Lisbon, Portugal

5.6 Refereeing This aspect of the TTRPG experience is the least grounded in ex-
Perhaps the most nebulous aspect of PX around the table is the isting PX measurement instruments, and will likely need extensive
impact that the GM has on the experience. Unlike other players, conceptual refinement and brainstorming before a pool of items is
the GM is not a collaborator but instead has a multitude of roles generated for it. As noted in Section 6, additional interviews with
[39, 107]. Moreover, the GM has significant power and authority in GMs should clarify the more general and useful feedback that such
shaping all aspects of the game (especially the outcomes, themes, a PX questionnaire could capture.
and narrative progression) in the implicitly “recognized structure
of power” [89] of a TTRPG group. The impact a GM has on a role- 6 DISCUSSION AND NEXT STEPS
playing experience is undeniable; if we assume that PX evaluation Based on the arguments laid out in the previous sections, PX eval-
would be most valuable to GMs, then this aspect is what they would uation in TTRPGs is valuable not only to PX researchers but also
be most interested in and could more immediately improve on. to GMs, players, TTRPG designers and event organizers. In order
We identify two ways in which the GM impacts the players’ to conceptualize a first set of aspects of TTRPG PX, we focused
experience around the table that do not significantly overlap with on those that could provide meaningful data to the above stake-
other aspects of PX listed above: moderation and pacing. holders so that they can act upon it. Based on a combination of
Moderation refers to how the GM uses their power to manage literature review and personal experience, we identified six PX com-
the players (rather than the PCs), both to empower each player and ponents: immersion in the imaginary gameworld, agency over their
to handle conflicts between players. This includes conflicts between character and over the story, attachment to the character, social
players and the GM, for instance when there is disagreement on the dynamics between players, decision-making based on the rules
application of a rule or confusion regarding the mental model of the and the game state, and GM direction. Some of these experiences
world (see Section 5.1). Handling clashes as they occur, detecting are already commonly evaluated in digital games. That said, it is
increased tension and steering away from a clash, and generally not straightforward to apply (or adapt) such PX questionnaires to
contributing to a positive atmosphere are all steps that the GM can TTRPGs due to differences in the medium or terminology, in the
take to better moderate the group dynamics during the game. Note player’s unbounded freedom and in the shaky consensus regarding
that GM moderation of players’ off-game behavior during the game the game state among players. In addition, some aspects such as
is distinct from group dynamics listed in Section 5.5. This is due to GM refereeing have no similar counterpart in digital games.
the fact that the GM is not a participant in players’ decision-making It should be noted that the above aspects of TTRPG PX are
and due to the power imbalance between a player and a GM; there likely not equally valuable to different stakeholders. Indicatively,
is an implicit understanding that moderation is one of the GM’s TTRPG designers may be more interested in cognitive challenges
tasks and that the GM has final say over the game’s rules or the due to over-complicated rules or lack of material components (e.g.
description of the gameworld [107]. miniatures) to communicate the game state. GMs may mostly be
Pacing refers to how the GM uses their power to move the story interested in feedback on their refereeing, while players may reflect
forward or to slow the action in order to draw attention to impor- on their emotions when answering questions on agency. However,
tant parts of the narrative or to give PCs opportunities for self- all aspects above — and probably more — intertwine and impact the
expression or decision-making. Borrowing from film-making, the experience of play in unclear and unforeseeable ways. Therefore,
GM controls pacing as a scriptwriter who sets the scenes and dia- even if different stakeholders can only act on a small part of the
logue and as an editor who can cut a scene short or let it run longer. TTRPG experience, having a holistic view thereof is important as
As an editor, the GM may fast-forward an uninteresting activity changes in e.g. the rules can impact immersion or group dynamics.
such as a long voyage (e.g. narrating a montage of minor events This paper reviewed related work on TTRPGs, mainly from the
during the activity) or may wrap up a scene that is running long digital humanities and social sciences, and attempted to identify
(either explicitly by asking the players or implicitly by triggering actionable and meaningful components of TTRPG PX in Section
in-game events that force the PCs to move on). The GM controls the 5. However, this is one of many steps towards developing and
pacing as a scriptwriter by choosing which scenes to bring up for validating a TTRPG PX questionnaire. Following questionnaire
role-playing. Unlike scriptwriters, GMs set the scene (with potential development guidelines by Kline [72], next steps include:
interesting dilemmas) and leave its resolution to the players. Good
pacing can be established by introducing scenes where there is a (1) conducting interviews with TTRPG players, Game Masters,
meaningful and informed choice to be made, where there are inter- and possibly other stakeholders, to solicit their feedback on
esting vistas to describe and explore, where there is an interesting actionable factors and fill in gaps in the literature
NPC to interact with, where an event or prop foreshadows events to (2) generating a broad initial set of questions (items pool) based
come, etc. In published adventures, such scenes may be pre-defined on the themes identified in the literature and interviews
by the RPG designers and thus evaluating pacing becomes relevant (3) running the items pool by experts (e.g. RPG designers, expe-
for those stakeholders as well. Some scenes may hinge on player rienced GMs and researchers) with a view to refine and trim
decision-making and role-play while others may be limited to GM the items pool to create the first version of the questionnaire
narration. Part of the challenge of pacing for GMs is balancing the (4) distribute the questionnaire to gather responses from a range
two, giving enough top-down gameworld description to stimulate of TTRPG players
immersion while giving players opportunities for high-stakes and (5) assess the construct validity of the first version of the ques-
evocative role-playing to ensure their agency. tionnaire and refine it further based on the results from the
exploratory factor analysis
FDG 2023, April 12–14, 2023, Lisbon, Portugal Liapis and Denisova

(6) further validation of the questionnaire through confirmatory games where there is no underlying executable that provides the
factor analysis and experimental studies gameworld, goals, and possibility space. Based on a preliminary lit-
erature review and personal experience, we identify six components
Exploratory factor analysis (EFA) and confirmatory factor analysis
for TTRPG PX that are valuable and actionable to assess: cognitive
(CFA) will be used on the players’ responses to the initial and
challenge, immersion, agency, attachment, group dynamics, and
refined questionnaire, respectively [72]. EFA ascertains whether
refereeing. The benefit of developing PX evaluation methodolo-
associations exist between the initial questions, and if so, where
gies (including questionnaires) specific to TTRPGs is two-fold: (a)
they lie and how they are grouped, while CFA tests the researcher’s
it can help game designers and event organizers collect succinct
hypothesis by confirming where the latent variables are and how
and actionable data from playtests and events, respectively; (b) it
much variance they account for. This approach suits the design
provides a framework for individual RPG groups to homogenize
process of a TTRPG PX questionnaire well: it will be possible to
expectations when setting up a campaign, and to debrief after each
solicit data collection from the many active online communities of
game session. The initial set of challenges and directions for PX
TTRPG players. Perhaps the least intuitive part of the above plan
evaluation presented in this paper need to be verified and refined
is the validation step. Indicatively, Denisova et al. [42] validated
via dedicated interviews with TTRPG groups (including players
their questionnaire on player responses on three games held out
and Game Masters), in order to build an initial set of items for
from the CFA and assessed the expected differences in PX between
evaluating TTRPG PX. This initial set will then need to be tested,
these games based on authors’ a priori knowledge of the games. In
refined, and validated through large-scale studies following estab-
theory, a similar method can be applied for TTRPG PX, using game
lished questionnaire development processes [72]. While the path
systems that differ, for instance, in terms of how their rules facilitate
forward is dark and full of terrors, as bold adventurers we ought to
creativity [11]. However, the game system is only one factor in
shed light on the important question of PX evaluation in TTRPGs.
the type of experiences occurring during TTRPG play, with other
major factors being the GM, players, adventure setup, game setting,
ACKNOWLEDGMENTS
and more. Therefore, validation of implicit assumptions that, for
example, a game system attracts specific play personas (see Section To all past and future Game Masters, for their patience and hard
3) overlooks that different groups may play the same game system work preparing adventures that player characters wrecked.
very differently, which could skew responses one way or the other.
A limitation of the work so far is the assumption that the TTRPGs REFERENCES
being evaluated have a Game Master. This is not only evident [1] Vero Vanden Abeele, Katta Spiel, Lennart Nacke, Daniel Johnson, and Kathrin
Gerling. 2020. Development and validation of the player experience inventory:
in the explicit refereeing component of TTRPG PX, but also in A scale to measure player experiences at the level of functional and psychosocial
the general assumptions made when brainstorming the important consequences. International Journal of Human-Computer Studies 135 (2020).
[2] Aubrie S. Adams. 2013. Needs met through role-playing games: A fantasy
and actionable aspects of the experience. Moreover, the next steps theme analysis of Dungeons & Dragons. Kaleidoscope: A Graduate Journal of
for producing an evaluation instrument will include interviews Qualitative Communication Research 12 (2013). https://opensiuc.lib.siu.edu/
with GMs, thus perpetuating our initial focus on these types of kaleidoscope/vol12/iss1/6
[3] Avery Alder. 2013. The Quiet Year. Buried Without Ceremony.
TTRPGs. Alternative versions of play exist, including games with [4] Avery Alder. 2021. Monsterhearts 2. Buried Without Ceremony.
no GMs, games with more than one GMs, games played virtually [5] Ahmad Azadvar and Alessandro Canossa. 2018. UPEQ: Ubisoft perceived expe-
or via teleconferencing software. The initial TTRPG PX evaluation rience questionnaire: A self-determination evaluation tool for video games. In
Proceedings of the International Conference on the Foundations of Digital Games.
instruments, developed with the assumption of a GM-based power [6] Jaime Banks and Nicholas David Bowman. 2016. Emotion, anthropomorphism,
structure, will later need to be tested on more varied games and realism, control: Validation of a merged metric for player–avatar interaction
(PAX). Computers in Human Behavior 54 (2016), 215–223.
determine whether the same theories and evaluation methods hold [7] Jaime Banks, Nicholas David Bowman, Jih-Hsuan Tammy Lin, Daniel
for other contexts and setups. Pietschmann, and Joe A. Wasserman. 2019. The common player-avatar in-
Another limitation is the explicit focus on the experience of play- teraction scale (cPAX): Expansion and cross-language validation. International
Journal of Human-Computer Studies 129 (2019), 64–73.
ers who control one player character. The suggested directions of [8] Matt Barton. 2008. Dungeons and Desktops. A K Peters.
Section 5 do not address the experience of a Game Master, a GM [9] Roy F. Baumeister and Mark R. Leary. 1995. The need to belong: Desire for
aide, or an audience. We focus on the players, because it is more interpersonal attachments as a fundamental human motivation. Psychological
Bulletin 117, 3 (1995), 497–529.
important to enhance their experience compared to the GM experi- [10] Whitney Strix Beltràn. 2015. Why minority settings in RPGs matter. https:
ence; the latter is also contingent on significant preparation work //www.tor.com/2015/04/27/why-minority-settings-in-rpgs-matter/.
[11] Karl Bergström. 2012. Creativity rules: How rules impact player creativity in
as well as on the good experience and role-playing of the players three tabletop. International Journal of Role-Playing 3 (2012).
themselves. Future work could explore other TTRPG stakeholders’ [12] Logan Bonner, Jason Bulmahn, Stephen Radney-MacFarland, and Mark Seifter.
experience, as collecting data on the GM’s experience could be in 2014. Pathfinder Core Rulebook (2nd ed.). Paizo Inc.
[13] Julia Ayumi Bopp, Elisa D. Mekler, and Klaus Opwis. 2016. Negative emotion,
turn valuable for TTRPG designers and event organizers. positive experience? Emotionally moving moments in digital games. In Proceed-
ings of the CHI Conference on Human Factors in Computing Systems. 2996–3006.
[14] Julia Ayumi Bopp, Klaus Opwis, and Elisa D. Mekler. 2018. “An odd kind of
7 CONCLUSION pleasure”: Differentiating emotional challenge in digital games. In Proceedings
of the CHI Conference on Human Factors in Computing Systems.
This paper is intended as a clarion call towards further research [15] Emily Care Boss. 2008. Key concepts in Forge theory. In Playground worlds.
in evaluating player experience in tabletop RPGs. Our motivation Creating and evaluating experiences of role-playing games, Markus Montola and
is not merely the lack of research in PX evaluation within this Jaakko Stenros (Eds.). Ropecon ry.
[16] Nicholas David Bowman, Joseph Wasserman, and Jaime Banks. 2018. Develop-
subdomain, but primarily the very different challenges that PX ment of the Video Game Demand Scale. In Video games: A medium that demands
evaluation faces when dealing with unscripted and improvisational our attention, Nicholas David Bowman (Ed.). Routledge, 208–233.
The Challenge of Evaluating Player Experience in Tabletop RPGs FDG 2023, April 12–14, 2023, Lisbon, Portugal

[17] Sarah Lynne Bowman. 2010. The functions of role-playing games: How par- [48] Timea Farkas, Alena Denisova, Sarah Wiseman, and Rebecca Fiebrink. 2022.
ticipants create community, solve problems and explore identity. McFarland & The effects of a soundtrack on board game player experience. In Proceedings of
Co. the CHI Conference on Human Factors in Computing Systems.
[18] Sarah Lynne Bowman. 2013. Social conflict in role-playing communities: an [49] Timea Farkas, Sarah Wiseman, Paul Cairns, and Rebecca Fiebrink. 2020. A
exploratory qualitative study. International Journal of Role-Playing 4 (2013). grounded analysis of player-described board game immersion. In Proceedings of
[19] Sarah Lynne Bowman. 2018. Immersion and shared imagination in role-playing the Annual Symposium on Computer-Human Interaction in Play. 427–437.
games. In Role-playing game studies: Transmedia foundations, José Zagal Sebas- [50] Gary Alan Fine. 1983. Shared fantasy: Role playing games as social worlds. The
tian Deterding (Ed.). Routledge. University of Chicago Press.
[20] Jeanne H. Brockmyer, Christine M. Fox, Kathleen A. Curtiss, Evan McBroom, [51] Amanda Flowers, Brian Magerko, and Punya Mishra. 2006. Gamemasters and
Kimberly M. Burkhart, and Jacquelyn N. Pidruzny. 2009. The development interactive story: A categorization of storytelling techniques in live roleplaying.
of the Game Engagement Questionnaire: A measure of engagement in video In Proceedings of FuturePlay: The International Academic Conference on the Future
game-playing. Journal of Experimental Social Psychology 45 (2009), 624–634. of Game Design and Technology.
[21] Emily Brown and Paul Cairns. 2004. A grounded investigation of game immer- [52] Erving Goffman. 1974. Frame analysis: An essay on the organization of experience.
sion. In Proceedings of the Extended Abstracts on Human Factors in Computing Northeastern University Press.
Systems. [53] Chad Phoenix Rose Gowler and Ioanna Iacovides. 2019. “Horror, Guilt and
[22] Jason Bulmahn. 2009. The beta playtest is officially closed. https://paizo.com/ Shame” – Uncomfortable Experiences in Digital Games. In Proceedings of the
threads/rzs2jbtw?the-beta-playtest-is-officially-closed. Accessed 20 Nov 2022. Symposium on Computer-Human Interaction in Play. 325–337.
[23] Jason Bulmahn. 2009. Pathfinder core rulebook (1st ed.). Paizo Inc. [54] Jesse Graham, Brian A Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and
[24] Gordon Calleja. 2011. In-game: From immersion to incorporation. MIT Press. Peter H Ditto. 2011. Mapping the moral domain. Journal of Personality and
[25] Gordon Calleja. 2022. Unboxed: Board game experience and design. MIT Press. Social Psychology 101, 2 (2011), 366–85.
[26] Alessandro Canossa and Anders Drachen. 2009. Patterns of play: Play-personas [55] Milton Griepp. 2021. Hobby game sales over $2 billion in 2020: Pandemic year
in user-centred game development. In Proceedings of DiGRA. ends on up note. https://icv2.com/articles/markets/view/48827/hobby-game-
[27] Beomkyu Choi, Jie Huang, Annie Jeffrey, and Youngkyun Baek. 2013. Develop- sales-over-2-billion-2020.
ment of a scale for fantasy state in digital games. Computers in Human Behavior [56] Mark Grimshaw, John P Charlton, and Richard Jagger. 2011. First-person shoot-
29 (2013). ers: Immersion and attention. Eludamos: Journal for Computer Game Culture 5,
[28] Tom Cole and Marco Gillies. 2021. Thinking and doing: Challenge, agency, and 1 (2011), 29–44.
the eudaimonic experience in video games. Games and Culture 16, 2 (2021), [57] Suzanne Haeyen, Susan van Hooren, William M van der Veld, and Giel
187–207. Hutschemaekers. 2018. Measuring the contribution of art therapy in multi-
[29] Tom Cole and Marco Gillies. 2022. Emotional exploration and the eudaimonic disciplinary treatment of personality disorders: The construction of the Self-
gameplay experience: A grounded theory. In Proceedings of the CHI Conference expression and Emotion Regulation in Art Therapy Scale (SERATS). Personality
on Human Factors in Computing Systems. and Mental Health 12, 1 (2018).
[30] Mike Cook, Matthew Gremo, and Ryan Morgan. 2017. We’re just playing: The [58] Johanna Hall, Christothea Herodotou, and Ioanna Iacovides. 2022. Measuring
influence of a modified tabletop role-playing game on ELA students’ in-class player creativity in digital entertainment games using the Creativity in Gaming
reading. Simulation and Gaming 48 (2017), 199–218. Scale. In Open world learning: Research, innovation and the challenges of high-
[31] Jennifer Grouling Cover. 2010. The creation of narrative in tabletop role-playing quality education, Bart Rienties, Regine Hampel, Eileen Scanlon, and Denise
games. McFarland & Co. Whitelock (Eds.). Routledge.
[32] Jeremy Crawford, Christopher Perkins, and James Wyatt. 2014. Dungeons and [59] Jessica Hammer. 2007. Agency and authority in role-playing “texts”. In A New
Dragons Dungeon Master’s Guide (1st ed.). Wizards of the Coast. Literacies Sampler, Michele Knobel and Colin Lankshear (Eds.). Peter Lang.
[33] Rowan Daneels, Nicholas D Bowman, Daniel Possler, and Elisa D Mekler. 2021. [60] John Harper. 2017. Blades in the Dark. Evil Hat.
The ‘eudaimonic experience’: A scoping review of the concept in digital games [61] J. Tuomas Harviainen. 2003. The multi-tier game immersion theory. In As Larp
research. Media and Communication 9, 2 (2021), 178–190. Grows Up, Morten Gade, Line Thorup, and Mikkel Sander (Eds.). Projektgruppen
[34] Stéphane Daniau. 2016. The transformative potential of role-playing games: KP03.
From play skills to human skills. Simulation & Gaming 47, 4 (2016), 423–444. [62] Tuomas Harviainen. 2008. A hermeneutical approach to role-playing analysis.
[35] Steven L. Dashiell. 2020. Hooligans at the table: The concept of male preserves International Journal of Role-Playing 1 (2008).
in tabletop role-playing games. International Journal of Role-Playing 10 (2020). [63] Michael Hitchens and Anders Drachen. 2008. The many faces of role-playing
[36] Mark H. Davis. 1980. A multidimensional approach to individual differences in games. International Journal of Role-Playing 1 (2008).
empathy. JSAS Catalog of Selected Documents in Psychology 10 (1980). [64] Matthijs Holter. 2007. Stop saying “immersion”! In Lifelike, Jesper Donnis,
[37] Yvonne De Kort, Wijnand Ijsselsteijn, and Karolien Poels. 2007. Digital games Morten Gade, and Line Thorup (Eds.). Projektgruppen KP03.
as social presence technology: Development of the social presence in gaming [65] Matthew Hudson and Paul Cairns. 2014. Measuring social presence in team-
questionnaire (SPGQ). In Proceedings of the International Workshop on Presence. based digital games. In Interacting with Presence: HCI and the sense of presence in
[38] Edward L. Deci and Richard M. Ryan. 1985. Intrinsic Motivation and Self- computer-mediated environments, Giuseppe Riva, John Waterworth, and Dianne
Determination in Human Behavior. Springer. Murray (Eds.). De Gruyter Open Ltd., 83–101.
[39] Guylain Delmas, Ronan Champagnat, and Michel Augeraud. 2009. From tabletop [66] Wijnand IJsselsteijn, Yvonne de Kort, and Karolien Poels. 2013. The Game
RPG to interactive storytelling: Definition of a story manager for videogames. Experience Questionnaire. Technical Report. Technische Universiteit Eindhoven.
In Proceedings of the International Conference on Interactive Digital Storytelling. [67] Susan A. Jackson and Herbert Marsh. 1996. Development and validation of a
[40] Alena Denisova, Julia Ayumi Bopp, Thuy Duong Nguyen, and Elisa D Mekler. scale to measure optimal experience: The Flow State Scale. Journal of sport and
2021. “Whatever the emotional experience, it’s up to them”: Insights from exercise psychology 18, 1 (1996), 17–35.
designers of emotionally impactful games. In Proceedings of the CHI Conference [68] Charlene Jennett, Anna L. Cox, Paul Cairns, Samira Dhoparee, Andrew Epps,
on Human Factors in Computing Systems. Tim Tijs, and Alison Walton. 2008. Measuring and defining the experience of
[41] Alena Denisova and Paul Cairns. 2015. The placebo effect in digital games: immersion in games. International Journal of Human-Computer Studies 66, 9
Phantom perception of adaptive artificial intelligence. In Proceedings of the (2008), 641–661.
Annual Symposium on Computer-Human Interaction in Play. [69] Sven Joeckel, Nicholas David Bowman, and Leyla Dogruel. 2012. Gut or game?
[42] Alena Denisova, Paul Cairns, Christian Guckelsberger, and David Zendle. 2020. The influence of moral intuitions on decisions in video games. Media Psychology
Measuring perceived challenge in digital games: Development & validation of 15, 4 (2012), 460–485.
the challenge originating from recent gameplay interaction scale (CORGIS). [70] Luke Jordan. 2017. Across the Endless Sea. gamesfromthewildwood.
International Journal of Human-Computer Studies 137 (2020). [71] Maciej Karwowski and Marcin Soszynski. 2008. How to develop creative imagi-
[43] Alena Denisova and Eliott Cook. 2019. Power-ups in digital games: The reward- nation? Assumptions, aims and effectiveness of Role Play Training in Creativity
ing effect of phantom game elements on player experience. In Proceedings of (RPTC). Thinking Skills and Creativity 3, 2 (2008), 163–171.
the Annual Symposium on Computer-Human Interaction in Play. [72] Paul Kline. 2014. An easy guide to factor analysis. Routledge.
[44] Ansgar E. Depping and Regan L. Mandryk. 2017. Why is this happening to me? [73] Mehmet Kosa and Pieter Spronck. 2018. What tabletop players think about
How player attribution can broaden our understanding of player experience. In augmented tabletop games. In Proceedings of the International Conference on the
Proceedings of the CHI Conference on Human Factors in Computing Systems. Foundations of Digital Games.
[45] Anders Drachen, Michael Hitchens, and Thea Marie Drachen. 2008. Motivations [74] Ken Koziej and Mikael Hellström. 2018. Year zero economics: Using edu-larping
for play in computer role-playing games. In Proceedings of the Conference on to explore economic systems in the ninth grade. International Journal of Role-
Future Play. Playing 8 (2018).
[46] Anders Drachen, Pejman Mirza-Babaei, and Lennart Nacke. 2018. Games User [75] Richard Lambert, Andrew Rilstone, and James Wallis. 1993. Once Upon A Time.
Research. Oxford University Press. Atlas Games.
[47] Ron Edwards. 2001. GNS and other matters of role-playing theory. http://www.
indie-rpgs.com/articles/1/.
 FDG 2023, April 12–14, 2023, Lisbon, Portugal Liapis and Denisova

[76] Petri Lankoski and Simo Järvelä. 2012. An embodied cognition approach for In Proceedings of the CHI Conference Extended Abstracts on Human Factors in
understanding role-playing. International Journal of Role-Playing 3 (2012). Computing Systems. 2839–2845.
[77] Jean Laplanche and Jean-Bertrand Pontalis. 2006. The language of psychoanalysis. [99] Bettina Renner, Michael Prilla, Ulrike Cress, and Joachim Kimmerle. 2016. Effects
Karnac Books. of prompting in reflective learning tools: findings from experimental field, lab,
[78] Robin D. Laws. 2002. Robin’s Laws of good game mastering. Steve Jackson and online studies. Frontiers in Psychology 7 (2016).
Games. [100] Mark Richardson. 2018. Headspace. Green Hat Designs.
[79] Seung-hee Lee. 2005. Design and analysis of reflection-supporting tools in [101] Marie-Laure Ryan. 2003. Narrative as virtual reality: Immersion and interactivity
computer-supported collaborative learning. International Journal of Instructional in literature and electronic media. Johns Hopkins University Press.
Technology and Distant Learning 12, 1 (2005). [102] Richard M. Ryan, Scott Rigby, and Andrew Przybylski. 2006. The motivational
[80] Samantha Leigh. 2022. Anamnesis. https://blinkingbirchgames.itch.io/ pull of video games: A self-determination theory approach. Motivation and
anamnesis. Emotion volume 30 (2006), 344–360.
[81] Melissa Lewis, Rene Weber, and Nicholas Bowman. 2008. They may be pixels, [103] Paul Scriven. 2021. From tabletop to screen: Playing Dungeons and Dragons
but they’re MY pixels: Developing a metric of character attachment in role- during COVID-19. Societies 11, 4 (2021).
playing video games. Cyberpsychology & behavior: The impact of the Internet, [104] Shanna Germain Sean K. Reynolds. 2019. Consent in gaming. Monte Cook
multimedia and virtual reality on behavior and society 11 (2008). Games.
[82] Jonas Linderoth. 2012. The effort of being in a fictional world: Upkeyings and [105] Lily Spinelli. 2018. Tabletop role-playing games and social skills in young adults.
laminated frames in MMORPGs. Symbolic Interaction 35, 4 (2012). Honors College Theses 192 (2018).
[83] Daniel Mackay. 2001. The fantasy role-playing game: A new performing art. [106] Jonathan Strugnell, Marsha Berry, Fabio Zambetta, and Stefan Greuter. 2018.
McFarland & Co. Narrative improvisation: Simulating game master choices. In Proceedings of the
[84] Frans Mäyrä. 2017. Dialogue and interaction in role-playing games: Playful International Conference for Interactive Digital Storytelling.
communication as ludic culture. In Dialogue across Media, Jarmila Mildorf and [107] Anders Tychsen, Michael Hitchens, Thea Brolund, and Manolya Kavakli. 2005.
Bronwen Thomas (Eds.). John Benjamins. The Game Master. In Proceedings of the second Australasian conference on Inter-
[85] Minerva McJanda and Douglas Santana Mota. 2018. Legacy: Life Among the active entertainment.
Ruins (2nd ed.). Modiphius. [108] Anders Tychsen, Michael Hitchens, Thea Drachen, Doris Mcilwain, and Manolya
[86] Mike Mearls and Jeremy Crawford. 2014. Dungeons & Dragons: Player’s Hand- Kavakli. 2008. Group play: Determining factors on the gaming experience in
book. Wizards of the Coast. multiplayer role-playing games. Computers in Entertainment 5, 4 (2008).
[87] Michael C. Medlock. 2018. User experience maturity levels: Evaluating and im- [109] Anders Tychsen, Doris McIlwain, Thea Brolund, and Michael Hitchens. 2007.
proving Game User Research practices. In Serious Games: Foundations, Concepts Player-character dynamics in multi-player role playing games. In Proceedings of
and Practice, Anders Drachen, Pejman Mirza-Babaei, and Lennart Nacke (Eds.). the DiGRA International Conference: Situated Play.
Oxford University Press. [110] Peter Vorderer, Werner Wirth, Feliz Gouveia, Frank Biocca, Timo Saari, Lutz
[88] Mikko Meriläinen. 2012. The self-perceived effects of the role-playing hobby on Jäncke, Saskia Böcking, Holger Schramm, Andre Gysbers, Tilo Hartmann,
personal development – a survey report. International Journal of Role-Playing 3 Christoph Klimmt, Jari Laarni, Niklas Ravaja, Ana Sacau, Thomas Baumgart-
(2012). ner, and Petra Jäncke. 2004. MEC spatial presence questionnaire (MEC-SPQ):
[89] Markus Montola. 2007. Tangible pleasures of pervasive role-playing. In Proceed- Short documentation and instructions for application. Report to the European
ings of the 2007 DiGRA International Conference: Situated Play. Community, Project Presence: MEC (IST-2001-37661).
[90] Markus Montola and Jaakko Stenros. 2008. Playground Worlds. Creating and [111] William J White, Emily C Boss, and Tuomas Harviainen. 2012. Role-playing
Evaluating Experiences of Role-Playing Games. Ropecon ry. communities, cultures of play, and the discourse of immersion. In Immersive
[91] Jason Morningstar. 2011. Fiasco. Bully Pulpit Games. gameplay: Essays on participatory media and role-playing, Evan Torner and
[92] Lennart E. Nacke, Chris Bateman, and Regan L. Mandryk. 2014. BrainHex: A William J. White (Eds.). 71–86.
neurobiological gamer typology survey. Entertainment Computing 5, 1 (2014), [112] William J. White, Jonne Arjoranta, Michael Hitchens, Jon Peterson, Evan Torner,
55–62. and Jonathan Walton. 2018. Tabletop role-playing games. In Role-playing game
[93] Ken Newman. 2005. Albert in Africa: Online role-playing and lessons from studies: Transmedia foundations, José Zagal Sebastian Deterding (Ed.).
improvisational theatre. Computers in Entertainment 3, 3 (2005). [113] Josef Wiemeyer, Lennart Nacke, Christiane Moser, and Florian ‘Floyd’ Mueller.
[94] Luca Papale. 2014. Beyond identification: Defining the relationships between 2016. Player Experience. In Serious Games: Foundations, Concepts and Practice,
player and avatar. Journal of Games Criticism 1, 2 (2014). Ralf Dörner, Stefan Göbel, Wolfgang Effelsberg, and Josef Wiemeyer (Eds.).
[95] Tabitha C. Peck and Mar Gonzalez-Franco. 2021. Avatar embodiment. A stan- Springer International Publishing, 243–271.
dardized questionnaire. Frontiers in Virtual Reality 1 (2021). [114] Bob G Witmer and Michael J Singer. 1998. Measuring presence in virtual
[96] Mikki Phan, Joseph Keebler, and Barbara Chaparro. 2016. The development and environments: A presence questionnaire. Presence 7, 3 (1998), 225–240.
validation of the Game User Experience Satisfaction Scale (GUESS). Human [115] Jason S. Wrench, Narissra M. Punyanunt-Carter, and Katherine S. Thweatt. 2020.
Factors: The Journal of the Human Factors and Ergonomics Society 58 (2016). Interpersonal communication: A mindful approach to relationships. Milne Open
[97] Courtney N. Plante, Stephen Reysen, Christopher L. Groves, Sharon E. Roberts, Textbooks, Chapter Conflict in Relationships.
and Kathleen Gerbasi. 2017. The Fantasy Engagement Scale: A flexible measure [116] Nick Yee. 2016. The gamer motivation profile: What we learned from 250,000
of positive and negative fantasy engagement. Basic and Applied Social Psychology gamers. In Proceedings of the Annual Symposium on Computer-Human Interaction
39, 3 (2017), 127–152. in Play.
[98] Christopher Power, Alena Denisova, Themis Papaioannou, and Paul Cairns. [117] Nick Yee, Nicolas Ducheneaut, and Les Nelson. 2012. Online gaming motivations
2017. Measuring uncertainty in games: Design and preliminary validation. scale: Development and validation. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems.

View publication stats