Cognition and perception of body and time: This chapter provides an overview of the referenced cognitive psychology research on spatial and temporal perception. (to be completed)
To ground my research in how qualitative information leads to knowledge production from an aesthetics of temporal perception, I eschew conflicting theorems and deep philosophical arguments in favor of empirically tested cognitive psychology results. After a literature review, I will intersect it with research on bodily self-perception, specifically two significantly referenced research on body perception as they relate to technology and art—the rubber hand illusion and the Alice in Wonderland syndrome. My research is on aesthetically derived knowledge from the undiscussed intersection of these fields where temporal and spatial perception and bodily-self perception overlap:
What new perceptions of time does the scaled body create?
By what aesthetic methods can the new perceptions be studied?
What do we learn from the aesthetics of these new perceptions?
The body exists in three different relationships to the image: it is either a passive observer, or it occupies the image, or it creates the image. Each body has a different ontology where the body passively observing exists in the space outside of the image and detached from the capacity to produce knowledge, while the act of occupying the space of the image with the body creates engagement between the two that is the first necessary function for producing knowledge, and the third relationship of the body creating the image falls in the realm of experiential media and technologies such as AR and VR where in addition, our bodily self-perception constantly shifts.
Aesthetics of Temporal Perception in Virtual Reality
Why do we experience time dilation in virtual reality and what can we do with it?
(Reproduced from Proceedings of XTech Conference (Experiencial Technologies and Neurogaming Conference), San Francisco, April 2016.)
The current hot topic in virtual reality (VR) is time dilation. Though not a new psychological phenomenon, it is a renewed topic for a new medium. The abstract concept of time is a constant topic of study in arts, humanities, and sciences and from architecture and music to cognitive psychology and astrophysics, there is an attempt to understand what time is, how we perceive time, and whether time even exists. I eschew conflicting theorems and deep philosophical arguments in favor of empirically tested cognitive psychology results synthesized with examples from cinema and experimental films to understand why we experience time dilation in VR and to use is as an aesthetic means of developing narrative.
In passive cinema, where the camera is static, edits are common in lieu of tracking, and scenes shift frequently and sequentially for narrative, the engagement is also passive—passively intellectual, passively emotional, passively engaging and by no means physically engaging. In passive cinema we commonly hear ‘Time flies!’ as we step out from a movie theater. Active cinema, here defined as one where the camera is not a mere recorder of a scene and events but an extension of the viewer by being a character in the scene, or part of the dialogue, forces the viewer to interact with the scene. Interaction, or lack thereof, is what affects our time perception, and where we have become used to compressed time in cinema, real time scenes feel longer.
In the opening scene of Andrei Tarkovsky’s The Sacrifice, the camera is tracking in one shot a man and his grandson accompanied by a friend on bicycle as they walk across a field in conversation. The time effect of this long take is amplified by the conjecture of the camera with the viewer, which forces the viewer into becoming an active participant in the scene.
[Screen shots of Sacrifice 5:38 to 15:05]
Béla Tarr’s films are also notoriously composed of lengthy single shot tracking scenes, many of which are of the mundane. What is daily life than a collection of the mundane, from getting dressed to eating to commuting…? In Tarr’s films, the mundane become the sublime through long contemplation of their nature, and the sublime is what extends time (Rudd & Vohs, 2012). And so the five ‘potato’ scenes from Béla Tarr’s The Turin Horse are good grounding scenes to compare and analyze the time dilation of the Tarkovsky example, as well as time dilation in film in general. In these scenes, the camera is static. The action in each scene is in real time and no steps are skipped. You can peel a potato in sync with Ohlsdorfer and savagely eat it with him, or eat measured and in pace with his daughter, then clean up and wash the plates all in the same amount of time. But it feels like forever because we end up mentally peeling the potato with him, feel burnt from eating the hot potato too fast, feel lonely with his daughter, and generally feel the boredom that prolongs time. The lack of edit cuts allows us to physically engage in a scene as close to reality as possible.
[Film strip of potato scene]
19:50-23:37(25:00)
(57:03)57:30-1:00:03(1:01:01)
(1:25:39)1:26:46-1:28:18
(2:18:09) 2:18:48-2:20:00 (2:21:12)
2:28:00-2:33:00 more or less
But there is no speculation in these claims. There may be disagreements by cinema theorists, but the claim stands: scenes that require the viewer to engage leads to the time of these scenes feeling longer than their actual time. Room-scale virtual reality adds an additional dimension to engagement: you are actually physically engaging with the narrative unfolding in an immersive scene.
In virtual reality with six degrees of freedom, one is engaging with just as many schemas (emotional, physical, psychological, spatial, temporal, intellectual, etc.) The combination of freedom and engagement requires self-regulation. Self-regulation is decision-making and behaving based on these schema. In an immersive environment with no other distractions, this self-regulation leads to time dilation. This phenomenon is not new. Any medium that has engaged those experiencing has offered this temporal utopia, from reading a good book or hearing a good story to watching films as given by the examples above. Though not tested in the context of virtual reality, it is proven that regulating the self can elongate the felt duration of time (Vohs & Schmeicel, 2003). This was tested by asking some subjects to suppress their emotions while watching an emotional film. Those who self-regulated consistently guessed longer durations of time of the film than those who did not self-regulate.
Popular agreement is that in VR we experience time dilation, and from empirical research in cognitive psychology we have an understanding as to why. What can we do with this knowledge? I’ll address this aesthetically and open a dialogue on the aesthetics of temporal perception to be tested in virtual reality.
“In our profession everything depends on the extent of how interesting you make your narration.” —Andrei Tarkosky, 1966 interview
From earlier cognitive psychology research on language and time (Boroditsky, 1999) we learn that when speaking of time, aspects of time that are specified by spatial metaphors are shaped by those metaphors. We commonly use spatial metaphors to speak of time, and in spatial terms, we speak of time in one-directional, asymmetric terms as opposed to multi-dimensional and symmetric: up/down, ahead/behind and not shallow/deep and left/right (geological deep time is an exception because it is physical; it can be seen and experienced).
The word ‘dilation’ is symmetric and bi-directional. It presupposes an iris form—a shape with a center. If we speak of time dilation in spatial terms, this places the viewer within the dilating form. We become bodies immersed in the medium of time. A body now sits in a space with many concentrically nested possible times.
In the duration of The Turin Horse, Tarr presents us with five different scenes of Ohlsdorfer and his daughter each eating a potato. The first scene is focused on Ohlsdorfer and establishes his time scale (savagely fast). The second scene is focused on his daughter and establishes her time scale (measured and slow). In the third, fourth and fifth scene we see both centered on the frame, while in the third and fourth scene both are eating at their various speeds, and in the fith scefne, Ohlsdorfer is eating while his daughter is not. These scenes, in addition to expanding the overall time of the scene as presented earlier, introduce additional time scales nested within the overall time.
Three experimental video works that also utilize nesting multiple times within a scene are Camille Utterback’s Liquid Time series (2000-2001), and two of my work, Salton Sea Revisited (2009-2012), and Driving at the speed of the Nordic sun (2015). In Camille’s videos, the scene begins as a static snapshot of a moment in time. Movement of the viewer activates the scene as video, and the tracked movement of the viewer’s body across the screen manipulates the nested times in the video according to where the observer is. This introduces layers of multiple times in the scene. In my videos, all of time is present and running simultaneously. This work has led to a new form of data visualization in photography using temporal aesthetics. In Driving at the speed of the Nordic sun, the spatial metaphor of up and down is manipulated to match the spatial metaphor of sunup and sundown. The result is that a vertical line going straight up from the horizon eliminates the arc of the sun’s east-west trajectory.
Like Tarr’s sublime from the mundane, what can (sometimes) be more mundane than time just passing by? In VR, “the mundane becomes interesting”. As an extension, engagement with the mundane becomes interesting. Unlike reality, virtual reality allows us to actively manipulate the time of our environment and to physically engage within and with it. Taking cue from previous experimental examples, my latest work in VR, The Clocksmith’s Labyrinth, is an interactive scenography of continuous and simultaneous time where the narrative unfolds as nested layers of real time are spatially traversed. We understand time change to be marked by the appearance and disappearance of objects and events. In this scenario, time change is marked by the engagement and movement within the scene—that is itself hiding and revealing objects and events—thus allowing scenes and meta-narratives to emerge from within one continuous scene. This is only a sketch of how time can be used as a narrative element in VR. It is open to testing and discussion.