Abstract
“Early access” is a release strategy for software that allows consumers to purchase an unfinished version of the software. In turn, consumers can influence the software development process by giving developers early feedback. This early access model has become increasingly popular through digital distribution platforms, such as Steam which is the most popular distribution platform for games. The plethora of options offered by Steam to communicate between developers and game players contribute to the popularity of the early access model. The model is considered a success by the game development community as several games using this approach have gained a large user base (i.e., owners) and high sales. On the other hand, the benefits of the early access model have been questioned as well. In this paper, we conduct an empirical study on 1,182 Early Access Games (EAGs) on the Steam platform to understand the characteristics, advantages and limitations of the early access model. We find that 15% of the games on Steam make use of the early access model, with the most popular EAG having as many as 29 million owners. 88% of the EAGs are classified by their developers as so-called “indie” games, indicating that most EAGs are developed by individual developers or small studios. We study the interaction between players and developers of EAGs and the Steam platform. We observe that on the one hand, developers update their games more frequently in the early access stage. On the other hand, the percentage of players that review a game during its early access stage is lower than the percentage of players that review the game after it leaves the early access stage. However, the average rating of the reviews is much higher during the early access stage, suggesting that players are more tolerant of imperfections in the early access stage. The positive review rate does not correlate with the length or the game update frequency of the early access stage. Based on our findings, we suggest game developers to use the early access model as a method for eliciting early feedback and more positive reviews to attract additional new players. In addition, our findings suggest that developers can determine their release schedule without worrying about the length of the early access stage and the game update frequency during the early access stage.













Similar content being viewed by others
Notes
E.g., related news for the Dota 2 game: http://store.steampowered.com/news/?appids=570
References
Al-Ani B, Trainer E, Ripley R, Sarma A, Van Der Hoek A, Redmiles D (2008) Continuous coordination within the context of cooperative and human aspects of software engineering. In: Proceedings of the 2008 international workshop on cooperative and human aspects of software engineering. ACM, pp 1–4
Allen P (2016) Is Steam’s Early Access a ticking time bomb? http://www.gamasutra.com/blogs/paulallen/20160211/265598/is_steams_early_access_a_ticking_time_bomb.php, (last visited: Apr 18, 2017)
Bierzerkers (2016) DEVBLOG 5. http://archive.is/XHOjf, (last visited: Apr 18, 2017)
Cards C (2015) Free to play coming soon! http://store.steampowered.com/news/?appids=360730&headlines=1 (last visited: Apr 18, 2017)
Chambers C, Feng Wc, Sahu S, Saha D (2005) Measurement-based characterization of a collection of on-line games. In: Proceedings of the 5th ACM SIGCOMM conference on internet measurement (IMC), USENIX Association, pp 1–14
Chambers JM, Hastie TJ (1991) Statistical models in S. CRC Press, Inc
Clifford C (2014) Crowdfunding Generates More Than $60,000 an Hour (Infographic). http://www.entrepreneur.com/article/234051?newsletter=true, (last visited: Apr 18, 2017)
CPx (2015) Pool nation FX - lite NOW AVAILABLE. http://store.steampowered.com/news/?appids=314000&headlines=1 , (last visited: Apr 18, 2017)
Damodaran L (1996) User involvement in the systems design process-a practical guide for users. Behav Inform Technol 15(6):363–377
Dictionary O (2016) Oxford dictionary definition of Crowdfunding. http://www.oxforddictionaries.com/us/definition/american_english/crowdfunding, (last visited: Apr 18, 2017)
EuroGamer (2014) DayZ week-one sales rocket past 400,000. http://www.eurogamer.net/articles/2014-01-02-dayz-week-one-sales-rocket-past-400-000, (last visited: Apr 18, 2017)
Fine D (2016) Double Fine - What is Double Fine? http://www.doublefine.com/about/, (last visited: Apr 18, 2017)
Foley MJ (2015) Microsoft launches new Office Insider test program. http://www.zdnet.com/article/microsoft-launches-new-office-insider-test-program/, (last visited: Apr 18, 2017)
Gallivan MJ, Keil M (2003) The user–developer communication process: a critical case study. Inf Syst J 13(1):37–68
Galyonkin S (2016) SteamSpy - All the data and stats about Steam games. http://steamspy.com/, (last visited: Apr 18, 2017)
Greg D (2014) Double fine- action forums. https://archive.is/vfdqw, (last visited: Apr 18, 2017)
Jacobs M, Sihvonen T (2011) In perpetual beta? On the participatory design of facebook games. In: Think design play: the fifth international conference of the digital research association (DIGRA)
Johnson J (1999) Turning chaos into success. Softw Mag 19(3):30–34
Johnson J (2000) Chaos in the new millennium: The ghost of christmas future: the standish group. West Yarmouth, MA
Kujala S (2003) User involvement: a review of the benefits and challenges. Behav Inform Technol 22(1):1–16
LadyAijou (2014) About the early access - edge of space general discussions. http://steamcommunity.com/app/238240/discussions/0/45350790838400003/#c45350790880063101, (last visited: Apr 18, 2017)
LadyAijou (2015) ETA On getting out of early access? - edge of space general discussions. http://steamcommunity.com/app/238240/discussions/0/41973820433803052/#c594820656453333967, (last visited: Apr 18, 2017)
LeBreton J (2014) JP Lebreton’s twitter. https://twitter.com/vectorpoem/status/536933544472240128, (last visited: Apr 18, 2017)
Lin D, Bezemer CP, Hassan AE (2016) Studying the urgent updates of popular games on the steam platform. Empirical Software Engineering
Logfeller (2014) Release date? - Underrail general discussions. http://steamcommunity.com/app/250520/discussions/0/35222218811239879/#c626329186828487604, (last visited: Apr 18, 2017)
Long JD, Feng D, Cliff N (2003) Ordinal analysis of behavioral data. Wiley
Löwgren J, Stolterman E (2004) Thoughtful interaction design: a design perspective on information technology. MIT Press
Maalej W, Happel HJ, Rashid A (2009) When users become collaborators: towards continuous and context-aware user input. In: Proceedings of the 24th ACM SIGPLAN conference companion on object oriented programming systems languages and applications. ACM, pp 981–990
Manes S (1984) Taking a gamble with word vision. PC Mag, pp 211
Martin W, Sarro F, Jia Y, Zhang Y, Harman M (2016) A survey of app store analysis for software engineering. RN 16:02
McDougall J (2010) Community heroes: notch, for minecraft. http://www.pcgamer.com/community-heroes-notch-for-minecraft/ http://www.pcgamer.com/community-heroes-notch-for-minecraft/ (last visited: Apr 18, 2017)
Medierra (2015) When is this game coming out? - Grim dawn general discussions. http://steamcommunity.com/app/219990/discussions/0/530649887209076036/#c530649887215003016, (last visited: Apr 18, 2017)
Minecraft (2016) Minecraft official website. https://minecraft.net/, (last visited: Apr 18, 2017)
O’reilly T (2007) What is web 2.0: design patterns and business models for the next generation of software. Commun Strateg (1):17
Orland K (2011) Minecraft draws over $33 Million in revenue from 1.8m paying customers. http://www.gamasutra.com/view/news/33961/minecraft_draws_over_33_million_in_revenue_from_18m_paying_customers.php, (last visited: Apr 18, 2017)
Orland K (2014) Introducing steam gauge: ars reveals steam’s most popular games. http://arstechnica.com/gaming/2014/04/introducing-steam-gauge-ars-reveals-steams-most-popular-games/ (last visited: Apr 18, 2017)
Pavel Djundik MB (2016) Steam DB - steam database. https://steamdb.info/, (last visited: Apr 18, 2017)
PCGamer (2014) Tim schafer explains spacebase DF-9’s unexpected v1.0 release. http://www.pcgamer.com/tim-schafer-explains-spacebase-df-9s-v1-0-release/, (last visited: Apr 18, 2017)
Romano J, Kromrey JD, Coraggio J, Skowronek J, Devine L (2006) Exploring methods for evaluating group differences on the NSSE and other surveys: are the t-test and Cohen’s d indices the most appropriate choices. In: Annual meeting of the Southern association for institutional research
Stern C (2012) What makes a game indie: a universal definition. http://sinisterdesign.net/what-makes-a-game-indie-a-universal-definition/, (last visited: Apr 18, 2017)
Takkunen P (2015) Moving out of early access. http://store.steampowered.com/news/?appids=316080&headlines=1, (last visited: Apr 18, 2017)
Taylor TL (2006) Beyond management: considering participatory design and governance in player culture. First monday
ThunderPeel2001 (2015) The facts about spacebase DF-9. https://steamcommunity.com/app/246090/discussions/0/616198900637753626/, (last visited: Apr 18, 2017)
TimOfLegend (2014) Tim Schafer answers common questions about v1.0. https://steamcommunity.com/app/246090/discussions/0/613936673464943075/, (last visited: Apr 18, 2017)
Ullrich C, Borau K, Luo H, Tan X, Shen L, Shen R (2008) Why web 2.0 is good for learning and for research: principles and prototypes. In: Proceedings of the 17th international conference on world wide web. ACM, pp 705–714
Valve (2016a) Introducing early access. http://store.steampowered.com/earlyaccessfaq/, (last visited: Apr 18, 2017)
Valve (2016b) Spacebase DF-9 on steam. http://store.steampowered.com/app/246090/, (last visited: Apr 18, 2017)
Waclawek T (2015) Ronin, the turn-based action-platformer, is now on steam. http://store.steampowered.com/news/?appids=274230&headlines=1, (last visited: Apr 18, 2017)
Walker P (2014) Early Access popularity growing, but only 25% have released as a full game. http://www.gamesindustry.biz/articles/2014-11-13-early-access-popularity-growing-but-only-25-percent-have-released-as-a-full-game, (last visited: Apr 18, 2017)
Washburn Jr M., Sathiyanarayanan P, Nagappan M, Zimmermann T, Bird C (2016) What went right and what went wrong: an analysis of 155 postmortems from game development. In: Proceedings of the 38th international conference on software engineering companion. ACM, New York, ICSE ’16, pp 280–289
Welch C (2013) Steam Early Access lets gamers buy and play titles still in development. http://www.theverge.com/2013/3/20/4128644/steam-early-access-buy-and-play-games-still-in-development, (last visited: Apr 18, 2017)
Yin-Poole W (2014) Valve tightens Steam Early Access rules for developers. http://www.eurogamer.net/articles/2014-11-21-valve-tightens-steam-early-access-rules-for-developers, (last visited: Apr 18, 2017)
Acknowledgements
We are grateful to Sergey Galyonkin, the owner of Steam Spy, who generously gave us access to all the historical data of Steam collected by Steam Spy for this research. We also thank professor Jennifer R. Whitson from the University of Waterloo and Mr. Patrick Walker from EEDAR for the inspiring discussions about the gaming industry.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Massimiliano Di Penta
Rights and permissions
About this article
Cite this article
Lin, D., Bezemer, CP. & Hassan, A.E. An empirical study of early access games on the Steam platform. Empir Software Eng 23, 771–799 (2018). https://doi.org/10.1007/s10664-017-9531-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-017-9531-3