Augmented Urban Experiences
Augmented Urban Experiences
ABSTR ACT
The built environment is a complex juxtaposition of static matter and dynamic flows, tangible 1 Urban proxemics and sensory expe-
rience: analysis and representation
objects and human experiences, physical realities and digital spaces. This paper offers an alterna-
of proximity data and EEG readings
tive understanding of those dichotomies by applying experimental design research strategies that in relation to typologies of urban
combine objective quantification and subjective perception of urban contexts. The assumption environments.
is that layers of measurable datasets can be afforded with personal feedback to reveal "hidden"
characteristics of cities. Drawing on studies from data and cognitive sciences, the proposed method
allows us to analyze, quantify and visualize the individual experience of the built environment in
relation to different urban qualities.
By operating in between the scientific domain and the design realm, four design research experi-
ments are presented. Leveraging augmenting and sensing technologies, these studies investigate:
(1) urban attractors and user attention, employing eye-tracking technologies during walking; (2) urban
proxemics and sensory experience, applying proximity sensors and EEG scanners in varying contexts;
(3) urban mood and spatial perception, using mobile applications to merge tangible qualities and
subjective feelings; and (4) urban vibe and paced dynamics, combining vibration sensing and observa-
tional data for studying city beats.
82
INTRODUCTION in between the scientific domain and the design realm, these
The built environment is one of the most fascinating yet studies allow us to merge quantitative and qualitative anal-
enigmatic artifacts of the human being. It fuses the physical ysis into a hybrid form of design research that can eventually
characteristics of a space with the immaterial properties associ- support creative intuition for the development of more engaging,
ated with that space—what Gilles Deleuze would call “intensive” pleasant and responsive spaces and places.
and “extensive” qualities (Deleuze 1994). We perceive our
surrounding world as a complex entity resulting from the juxta- BACKGROUND RESEARCH
position of static matter and dynamic flows, tangible objects and Objective Quantification of Urban Qualities
human experiences, physical realities and digital environments. Over the last few years, research interest has grown on methods
and tools to capture key criteria, parameters, and indicators for
Each urban context possesses certain qualities, and even though depicting and quantifying the specificities of cities in order to
shared characteristics do exist, those qualities vary from place better explain the functioning mechanisms of urban environ-
to place. To capture what makes built environments unique is a ments (Bettencourt 2013). The assumption is that, by drawing on
matter of both quantitative knowledge of their tangible aspects insights of urban dynamics, flows, and activities, the use of "big
and individual perception of their immaterial properties. In the data—a cumulative aggregation of information provided by urban
field of architecture and urban design the tendency has always systems and users—can improve city management and enable
been to emphasize this dichotomy. In fact, scientific methods the development of fine-tuned regulations and the prediction of
are mostly employed in relation to the technical, physical and future needs (Glaeser, Luca, and Naik 2015).
material aspects of the built environment, whereas the individual
perceptions, emotions and responses conveyed and provoked by The first hints of this tendency to measure and quantify urban
architecture are a matter of design intuition (Pallasmaa 2013). parameters can be traced back to the cybernetic movement
of the 1950s and ‘60s. The emergence of computer systems
Peter Zumthor famously stated: “I enter a building, see a room, opened up new possibilities for urban designers and architects
and—in the fraction of a second—have this feeling about it” to explore correlations between design plans and actual urban
(2006, 13). Do urban contexts embed the same evocative activities. In these cybernetic experiments, “the computer could
power? What is the impact of the built environment in shaping further enhance connectivity while allowing a highly customiz-
individual and collective experiences? The complexity of today’s able management of ambience parameters” (Picon 2010, 32). In a
urban dynamics calls for alternative methods of understanding way, the cybernetic ideal has been replaced today by the "smart
the human experience for better informing the design of spaces city" concept, employing technology as a strategy to optimize
and places. In particular, this paper argues that novel digital processes and operations, increase the efficiency of systems, and
technologies and augmenting tools offer unprecedented oppor- monitor urban dynamics (Picon 2015).
tunities for merging quantitative measurements and qualitative
analysis of urban contexts to enrich design research processes. As cities become more automated and embedded with sensors,
data is being generated in real time as a by-product of commu-
Cognitive science claims that the built environment changes our nicating the results of such automation processes (Batty 2016).
brain’s structure by modulating the function of genes, and there- This understanding of the use of technology in urban contexts
fore affects our behavior (Gage, quoted in Farling 2015, 183). is supported by urban theories that employ complex numerical
Without discussing the details of the psychological and biological models to describe networks of systems and flows—the “science
aspects of human perception, we attempt to offer an alternative of cities” (Batty 2013). By applying the principle of “scaling,”
look at the experiential nature of cities through experimental consistent correlations emerge between morphological charac-
strategies. We argue that, by articulating, mapping and exploiting teristics of cities—their size—and socioeconomic markers—such
the specificities of places through a multisensory and multidisci- as creativity and wages (Bettencourt 2007).
plinary approach, it is possible to measure and elaborate on the
subjective experience—eventually revealing "hidden" characteris- Technological innovations in data science have also enabled the
tics of cities. automatic collection and visualization of multiple datasets on
urban dynamics at various levels of aggregation (Andrienko et al.
To validate this assumption a few design research experiments 2013; Sun, Wu and Liu 2013). These tools can then be used to
performed by the authors’ lab are presented, exploring the role make comparisons and extrapolate information about collective
of augmenting and sensing technologies in mediating the interac- urban activities that address, for instance, the vitality (De Nadai
tion between people and the built environment. By operating 2016), rhythm (Miranda 2017), or safety (Naik 2014; Salesses
The experiment correlates gaze duration/intensity and different 4 Comparison between the visibility heat map as spatial analytic metrics and the
attention intensity maps from three test subjects.
qualities of the viewed objects (color, movement, distance,
geometry, etc.). By juxtaposing the eye-tracking results with a
map-drawing task, the visibility and the memorability of different selected street in Cambridge, MA, the 15 test subjects wore
elements are also compared. The employed eye-tracking device the eye-tracker, and at the end were asked to draw a map of
is a Pupil Dev, a product that comes with a world camera and the street. The participants were aware of how the eye-tracker
an eye camera (Figure 2). The video images taken from the two functions, and advised to act naturally. However, they were not
cameras are processed by its proprietary software that translates aware of the map drawing task beforehand, thus distinguishing
the position of the pupil from the eye image to the corre- between the long-term memory of the space and the short-term
sponding gaze position as related to the world image (Figure 3). memory from the immediate experience.
Although the tool is typically used to track eye movement in a
controlled setting, this study takes the device out of the lab and The test results offer an alternative understanding of the relation-
into the real world. ship between the characteristics of the built environment and
the visual attention patterns of users. Through a comparative
The experiment consisted of two main elements, the eye-tracking study of the eye gaze data during the street walk and post-walk
test and the map drawing task. While walking down the mapping exercise, several layers of inconsistency emerge: (1)
Urban Mood and Spatial Perception interests, desires, patterns, and habits. Active measurement is
The way in which we receive information and data about our achieved through a mobile application specifically designed and
interaction and relation to the built environment is largely fed to prototyped by the research team to get a deeper layer of subjec-
us through systematized, sensor-based statistics. The common tive inputs.
static methods of data collection leave little room for interpreta-
tion, and fail to consider the complexities and variables that may The app is based on the premise of engaging individual users
influence the ways in which people perceive their surroundings. to input basic descriptions of sensorial perceptions through
In fact, each individual experience, perception, and view of places valuation of the varying intensity at which they experience
is colored by a personal, subjective interpretation. stimuli. The interface is designed to negotiate through a gradient
of two extremes, allowing for the translation of intuitive experi-
This study addresses such additional layers describing the ences into measured information (Figure 8). When users enter a
subjective views and attitudes that could potentially have an predominant mood or feeling, a correlation between the experi-
impact on and influence how urban data is collected, interpreted, enced phenomena and the emotional response is also created by
and used. In this experiment, "objective" data is quantified by the app, therefore facilitating the personalization of an individual
measuring parameters that can offer an overall representation cognitive landscape (Figure 9).
of the analyzed place. In particular, measurements of tempera-
ture, humidity, spatial qualities, brightness, and sound levels are To test the validity of the research method and tools, a
performed via smartphone apps that employ embedded sensors few experiments were performed in different places within
or add-ons. These parameters and their combination result in a Cambridge (Figure 10). The implementation of the app’s subjec-
quantitative description of tangible features of the environment. tive data in conjunction with the more passive, objective dataset
extracted from complementary sensors, resulted in a series of
The more complex "subjective" experience is instead quanti- urban conditions. This alternative understanding of everyday
fied through two modes of data collection—passive and active. spatial perception allows for novel opportunities and poten-
Passive measurement involves processing existing information tials to enhance the interactions between the user and the city
that is already being transmitted and collected from personal through the recommendation and curation of experiences that
digital devices. These means of data collection are commonly are personally catered to the individual. By revealing the "mood"
utilized by social media, search engines, and apps for extrap- of urban environments, this real-time data could also be used by
olating information about the users and learning more of their city planners to better inform urban strategies and interventions.
This work demonstrated that, by building upon objective layers of Bettencourt, Luís M. A., José Lobo, Dirk Helbing, Christian Kühnert, and
data and affording them the complexity and variation of subjec- Geoffrey B. West. 2007. “Growth, Innovation, Scaling, and the pace of
tive, personal feedback, it is possible to gain a more holistic, Life in Cities.” Proceedings of the National Academy of Sciences of the United
and perhaps novel understanding of the built environment and States of America 104 (17): 301‒306.
of our presence within it. A lexicon of experimented situations
Bettencourt, Luís M. A. 2013. “The Origins of Scaling in Cities.” Science
becomes a potential framework for studying the different typol-
340 (6139): 1438‒1441.
ogies of urban spaces through the creative use of sensing and
augmenting technologies. Cernea, Daniel, Andreas Kerren, and Achim Ebert. 2011. “Detecting
Insight and Emotion in Visualization Applications with a Commercial
Future developments of this work will expand the repertoire EEG Headset.” In Proceedings Of SIGRAD 2011: Evaluations Of Graphics
of "hidden" urban qualities through more refined experimental And Visualization — Efficiency, Usefulness, Accessibility, Usability, 53–60.
investigations, and will ultimately translate the resulting body of Stockholm, Sweden: SIGRAD.
knowledge into recommendations for architects, urban planners,
Christiansen, Lars B., et al. 2016. “International Comparisons of the
and city managers for potential integration into their design
Associations Between Objective Measures of the Built Environment and
and decision-making processes. By setting up metrics that are
Transport-Related Walking and Cycling: IPEN Adult Study.” Journal of
not so commonly considered in design practices, the findings of
Transport & Health 3 (4): 467‒478.
the presented and future experiments aim to offer alternative
insights on how the spatial morphology, the dynamic activi- Collins, Mark, Toru Hasegawa, Robert Johnston, Cecil Barnes, and George
ties, and the subtle varying conditions of places affect people's Peter Louras. 2014. “DUMBO Neural Cartography." Columbia University,
Lynch, Kevin. 1960. The Image of the City. Cambridge, MA: MIT Press.
Ma, Liang and Jennifer Dill. 2015. “Associations Between the Objective
and Perceived Built Environment and Bicycling for Transportation.” Journal
of Transport & Health 2 (2): 248–255.
Farling, Melissa. “From Intuition to Immersion.” In Mind in Architecture: Naik, Nikhil, Jade Philipoom, Ramesh Raskar, and César Hidalgo.
Neuroscience, Embodiment, and the Future of Design, edited by Sarah 2014. “Streetscore ‒ Predicting the Perceived Safety of One Million
Robinson and Juhani Pallasmaa, 181–196. Cambridge, MA: MIT Press. Streetscapes.” In 2014 IEEE Conference On Computer Vision and Pattern
Recognition Workshops, 793–799. Columbus, OH: CVPRW.
Gallagher, Winifred. 1993. The Power of Place: How Our Surroundings
Shape Our Thoughts, Emotions, and Actions. New York: Poseidon Press. Nold, Christian, ed. 2009. Emotional Cartography: Technologies of the Self.
Self-Published. Accessed May 12, 2017. http://www.emotionalcartog-
Gehl, Jan. 2006. Life between Buildings: Using Public Space. Copenhagen:
raphy.net/EmotionalCartographyLow.pdf
The Danish Architectural Press.
Pallasmaa, Juhani. 2013. “Towards a Neuroscience of Architecture:
———. 2010. Cities for People. Washington, DC: Island Press.
Embodied Mind and Imagination.” In Architecture and Neuroscience: A Tapio
Gehl, Jan and Birgitte Svarre. 2013. How to Study Public Life. Washington, Wirkkala-Rut Bryk Design Reader, edited by Philip Tidwell, 4–21. Espoo,
DC: Island Press. Finland: Tapio Wirkkala-Rut Bryk Foundation.
Glaeser, Edward, Michael Luca, and Nikhil Naik. 2015. “Big Data and Big ———. 2014 “Space, Place and Atmosphere. Emotion and Peripherical
Cities: The Promises and Limitations of Improved Measures of Urban Perception in Architectural Experience.” Lebenswelt: Aesthetics and
Life.” NBER Working Paper 21778. Cambridge, MA: National Bureau of Philosophy of Experience 4: 230–245.
Economic Research.
14
Picon, Antoine. 2010. Digital Culture in Architecture: An Introduction for the “The Shortest Path to Happiness: Recommending Beautiful, Quiet, and
Design Professions. Basel: Birkhäuser. Happy Routes in the City.” In Proceedings of the 25th ACM Conference on
Hypertext and Social Media, 116–125. Santiago, Chile: ACM Hypertext.
———. 2015. Smart Cities: A Spatialised Intelligence. Chichester, UK: Wiley.
Zumthor, Peter. 2006. Atmospheres: Architectural Environments,
Picon, Antoine, and Alessandra Ponte. 2003. "Introduction." In
Surrounding Objects. Basel: Birkhäuser Press.
Architecture and the Sciences: Exchanging Metaphors, edited by Antoine
Picon and Alessandra Ponte. New York: Princeton Architectural Press.
IMAGE CREDITS
Rasmussen, Steen Eiler. 1962. Experiencing Architecture. Cambridge, MA: All drawings and images by the authors and the research team of the
MIT Press. Responsive Environments and Artifacts Lab at Harvard GSD.
Sadler, Simon. 1998. The Situationist City. Cambridge: MIT Press. Stefano Andreani is a licensed architectural engineer and educator
Salesses, Philip, Katja Schechtner, and César A. Hidalgo. 2013. “The interested in technologically-driven architectural design, exploring
Collaborative Image of The City: Mapping the Inequality of Urban human-centered design strategies for the built environment and
Perception.” PLoS ONE 8 (7): e68400. augmented experiences. He is a Research and Teaching Associate at
the Graduate School of Design at Harvard University and the Project
Sassen, Saskia. 2011. “Open Source Urbanism.” Domus. Accessed May 12, Manager of the Responsive Environments and Artifacts Lab (REAL) at
2017. http://www.domusweb.it/en/op-ed/2011/06/29/open-source- Harvard. Merging research and practice, Andreani designs and consults
urbanism.html at INVIVIA – a global architectural design, user experience and design
Schwartzman, Madeline. 2011. See Yourself Sensing: Redefining Human research studio based in Boston.
Perception. London: Black Dog Pub. Allen Sayegh is an architect, designer, an educator and the principal
Schreuder, Elaine, Jan van Erp, Alexander Toet, and Victor L. Kallen, 2016. of INVIVIA – an award winning global design firm. He is an Associate
“Emotional Responses to Multisensory Environmental Stimuli.” SAGE Open Professor at Harvard Graduate School of Design and the director of REAL
6 (1). doi:10.1177/1754073915572690. the Responsive Environment and Artifacts Lab at Harvard. His courses
and practice explores the potentials of media and technologically-inte-
Sun, Guo-Dao, Ying-Cai Wu, Rong-Hua Liang, and Shi-Xia Liu. 2013. “A
grated built environment, interaction design and the study of architectural
Survey of Visual Analytics Techniques and Applications: State-of-the-Art
and urban space thought through the impact of changing technology.
Research and Future Challenges.” Journal of Computer Science and
His work is characterized as the synthesis of architecture, digital art and
Technology 28 (5): 852–867.
design in coming up with innovative architectural human interfaces and
Quercia, Daniele, Rossano Schifanella, and Luca Maria Aiello. 2014. responsive environments.