Theory & Model
The Integrative Theory (Bogart 2014b) is an empirical argument that supports the computational model (Fig 1) at the heart of these artistic works. The theory holds that a biological or computational agent’s subjective visual “experience” depends on a single unified system of internal simulation that constructs and organizes perceptual information. In the computational model, visual “experience” is composed of percepts constructed from the segmentation and clustering of visual stimulus. Dreaming, mind wandering, external perception, and mental imagery—all simulations resulting in subjective visual “experience”—are enabled by differing degrees of feedback in a predictor. The predictor learns the occurrence of percepts corresponding to the current stimulus that could facilitate the agent’s next actions or prime perception. As this model focuses on dreaming, the agent has no actions and perceptual priming is not meaningful in terms of its perceptual mechanism. In mind wandering and dreaming, feedback within the predictor (where it predicts itself, rather than stimulus) determines the sequencing of percepts. These sequences are the flow of the subjective “experiences” of mind wandering (due to a lack of arousal) and dreaming (due to the darkness of stimulus). “Arousal” is the system’s response to significant visual changes in the film such as large movements or cuts.
This work differs from computational modeling precedents by (a) implementing a system that learns from complex real-world sensory data, (b) extending the conception of dreaming as imagination, and (c) integrating systems associated with external perception, mind wandering, and mental imagery. My deep engagement with the cognitive neuroscience literature contrasts with artistic precedents that employ superficial conceptions of cognitive processes.
In Watching and Dreaming (2001: A Space Odyssey), frames from Stanley Kubrick’s 2001: A Space Odyssey are used as stimulus. The work is composed of two channels; the left channel shows the frame presented to the system and the right channel shows the system’s corresponding “mental” state as a collage of activated percepts. These “mental” states (perceptual constructions) ebb and flow between waking, mind wandering, and dreaming. “Waking” occurs when there is sufficient arousal and when the circadian clock indicates “day.” During waking, both channels become synchronized as the system’s “mental” state matches the stimulus. These waking moments are fleeting, as darkness in the film, and Kubrick’s slow style of editing, often lull the system into dreaming and mind wandering, respectively. When the circadian clock indicates “day,” but there is insufficient arousal, the system mind wanders; when the circadian clock indicates “night” the system dreams. The combination of brightness of stimulus and arousal can wake the system from dreaming.
Over the duration of the film, the viewer observes the system’s process as it learns an initial set of percepts and attempts to predict their occurrence. While constructing initial percepts, the machine’s mental state passively mimics the input image. Once an initial set of percepts is constructed, the system can begin to learn predictions; this manifests in an initial flurry of chaotic mental states as the system is unable to make meaningful predictions without more “experience.” Over the course of the film, both percepts and predictions change in response to stimulus. Highly constrained perceptual material at the start of the film gives way to a pool of percepts representing increasingly diverse visual material. Simultaneously, increasing complexity in the occurrence of percepts make predictions difficult. Towards the end of the film, dreaming and mind wandering lead to complex sequences seemingly unrelated to current stimulus.
The series of images, below, illustrates this process.
A strictly neuroscience understanding of dreaming, a highly contested and ill-defined phenomenon, is insufficient for computational implementation. In order to create the artwork, I had to develop a theory of dreaming suitable for computational modeling and resolve conflicts and ambiguities in the literature—the Integrative Theory. The cognitive model is a broad sketch of an agent, not a narrow model of a single well-defined mechanism. To my knowledge, no previous work in art, cognitive science, or computer science has constructed such a complete model of dreaming.
When the viewer sits down to watch Watching and Dreaming (2001: A Space Odyssey), I hope they consider the inherently constructive and creative nature of our perceptions; as the work lulls them into a doze, or a day-dream, they may appreciate the fluidity of mental states and how intermittent their attention to the world outside really is.
- Google’s DeepDream has nothing to do with dreaming; the name refers to the recursion of dreams in the film Inception due to DeepDream’s “network within a network” structure.
Bogart, B.D.R. 2012. An Artist in Process: A Computational Sketch of Dreaming Machine #3. New Forms Festival Exhibition. Vancouver, BC.
——. 2014a. Dreaming Machine #3. Exhibition for the 20th International Symposium on Electronic Art (ISEA) 2014, Zayed University, Dubai, UAE.
——. 2014b. “A Machine that Dreams: An Artistic Enquiry Leading to an Integrative Theory and Computational Artwork.” PhD thesis, Simon Fraser University.
——. 2017a. “Imagination, Art and Reality.” In Intersecting Art and Technology in Practice: Techne/Technique/Technology, edited by C. Baker and K. Sicchio, 73–85. Routledge.
——. 2017b. “A Machine that Dreams: An Artistic Enquiry Leading to an Integrative Theory and Computational Artwork.” In Top-Rated LABS Abstracts 2016 in Leonardo, edited by Sheila Pinkel, vol. 50(5), 530. MIT Press.
Bogart, B.D.R., and P. Pasquier. 2011. “Context Machines: A series of autonomous self-organizing site-specific artworks.” In Proceedings of the 17th International Symposium on Electronic Art (ISEA) 2011, Sabanci University, Istanbul, Turkey.
——. 2013a. “Context Machines: A series of situated and self-organizing artworks.” Leonardo 46 (2): 114–122.
——. 2013b. “Dreaming Machine #3 (prototype 2)”. In Proceedings of the 9th ACM Conference on Creativity & Cognition, 408–409. C&C ’13. Sydney, Australia: ACM. DOI: 10.1145/2466627.2481229.
Bogart, B.D.R., P. Pasquier, and S. J. Barnes. 2013. “An Integrative Theory of Visual Mentation and Spontaneous Creativity.” In Proceedings of the 9th ACM Conference on Creativity & Cognition, 264–273. New York: ACM.
Busch, K. 2009. “Artistic research and the poetics of knowledge.” ART & RESEARCH, A Journal of Ideas, Contexts and Methods 2 (2): 1–7.
Davies, J. 2019. Imagination. Pegasus Books.
Dörfelt, Matthias. 2011. “Selective Memory Theatre, Eine Installation über das selektive Wahrnehmen, Erinnern und Vergessen anhand von Bildern aus dem Internet.” Bachelor’s thesis, Muthesius Kunsthochschule. http://mokafolio.de/works/Selective-Memory-Theatre.
Franco, A. 2007. “Controlled Dream Machine.” MA thesis, University of Plymouth. http://www.anaisafranco.com/controlled.html.
Nir, Y. and G. Tononi. 2010. “Dreaming and the brain: from phenomenology to neurophysiology.” Trends in Cognitive Sciences 14 (2): 88–100.
Treur, J. 2011. “A Computational Agent Model Using Internal Simulation to Generate Emotional Dream Episodes”. In Biologically Inspired Cognitive Architectures 2011: Proceedings of the Second Annual Meeting of the BICA Society, 389–400. IOS Press.
Zhang, Q. 2009. “A computational account of dreaming: Learning and memory consolidation.” Cognitive Systems Research 10 (2): 91–101. DOI: 10.1016/j.cogsys.2008.06.002.