Theory & Model
The Integrative Theory (Bogart 2014b) is an empirical argument that supports the computational model (Fig 1) at the heart of these artistic works. The theory holds that a biological or computational agent’s subjective visual “experience” depends on a single unified system of internal simulation that constructs and organizes perceptual information. In the computational model, visual “experience” is composed of percepts constructed from the segmentation and clustering of visual stimulus. Dreaming, mind wandering, external perception, and mental imagery—all simulations resulting in subjective visual “experience”—are enabled by differing degrees of feedback in a predictor. The predictor learns the occurrence of percepts corresponding to the current stimulus that could facilitate the agent’s next actions or prime perception. As this model focuses on dreaming, the agent has no actions and perceptual priming is not meaningful in terms of its perceptual mechanism. In mind wandering and dreaming, feedback within the predictor (where it predicts itself, rather than stimulus) determines the sequencing of percepts. These sequences are the flow of the subjective “experiences” of mind wandering (due to a lack of arousal) and dreaming (due to the darkness of stimulus). “Arousal” is the system’s response to significant visual changes in the film such as large movements or cuts.
This work differs from computational modeling precedents by (a) implementing a system that learns from complex real-world sensory data, (b) extending the conception of dreaming as imagination, and (c) integrating systems associated with external perception, mind wandering, and mental imagery. My deep engagement with the cognitive neuroscience literature contrasts with artistic precedents that employ superficial conceptions of cognitive processes.
In Watching and Dreaming (2001: A Space Odyssey), frames from Stanley Kubrick’s 2001: A Space Odyssey are used as stimulus. The work is composed of two channels; the left channel shows the frame presented to the system and the right channel shows the system’s corresponding “mental” state as a collage of activated percepts. These “mental” states (perceptual constructions) ebb and flow between waking, mind wandering, and dreaming. “Waking” occurs when there is sufficient arousal and when the circadian clock indicates “day.” During waking, both channels become synchronized as the system’s “mental” state matches the stimulus. These waking moments are fleeting, as darkness in the film, and Kubrick’s slow style of editing, often lull the system into dreaming and mind wandering, respectively. When the circadian clock indicates “day,” but there is insufficient arousal, the system mind wanders; when the circadian clock indicates “night” the system dreams. The combination of brightness of stimulus and arousal can wake the system from dreaming.
Over the duration of the film, the viewer observes the system’s process as it learns an initial set of percepts and attempts to predict their occurrence. While constructing initial percepts, the machine’s mental state passively mimics the input image. Once an initial set of percepts is constructed, the system can begin to learn predictions; this manifests in an initial flurry of chaotic mental states as the system is unable to make meaningful predictions without more “experience.” Over the course of the film, both percepts and predictions change in response to stimulus. Highly constrained perceptual material at the start of the film gives way to a pool of percepts representing increasingly diverse visual material. Simultaneously, increasing complexity in the occurrence of percepts make predictions difficult. Towards the end of the film, dreaming and mind wandering lead to complex sequences seemingly unrelated to current stimulus.
The series of images, below, illustrates this process.
A strictly neuroscience understanding of dreaming, a highly contested and ill-defined phenomenon, is insufficient for computational implementation. In order to create the artwork, I had to develop a theory of dreaming suitable for computational modeling and resolve conflicts and ambiguities in the literature—the Integrative Theory. The cognitive model is a broad sketch of an agent, not a narrow model of a single well-defined mechanism. To my knowledge, no previous work in art, cognitive science, or computer science has constructed such a complete model of dreaming.
When the viewer sits down to watch Watching and Dreaming (2001: A Space Odyssey), I hope they consider the inherently constructive and creative nature of our perceptions; as the work lulls them into a doze, or a day-dream, they may appreciate the fluidity of mental states and how intermittent their attention to the world outside really is.
- Google’s DeepDream has nothing to do with dreaming; the name refers to the recursion of dreams in the film Inception due to DeepDream’s “network within a network” structure.
Bogart, B.D.R. 2012. An Artist in Process: A Computational Sketch of Dreaming Machine #3. New Forms Festival Exhibition. Vancouver, BC.
——. 2014a. Dreaming Machine #3. Exhibition for the 20th International Symposium on Electronic Art (ISEA) 2014, Zayed University, Dubai, UAE.
——. 2014b. “A Machine that Dreams: An Artistic Enquiry Leading to an Integrative Theory and Computational Artwork.” PhD thesis, Simon Fraser University.
——. 2017a. “Imagination, Art and Reality.” In Intersecting Art and Technology in Practice: Techne/Technique/Technology, edited by C. Baker and K. Sicchio, 73–85. Routledge.
——. 2017b. “A Machine that Dreams: An Artistic Enquiry Leading to an Integrative Theory and Computational Artwork.” In Top-Rated LABS Abstracts 2016 in Leonardo, edited by Sheila Pinkel, vol. 50(5), 530. MIT Press.
Bogart, B.D.R., and P. Pasquier. 2011. “Context Machines: A series of autonomous self-organizing site-specific artworks.” In Proceedings of the 17th International Symposium on Electronic Art (ISEA) 2011, Sabanci University, Istanbul, Turkey.
——. 2013a. “Context Machines: A series of situated and self-organizing artworks.” Leonardo 46 (2): 114–122.
——. 2013b. “Dreaming Machine #3 (prototype 2)”. In Proceedings of the 9th ACM Conference on Creativity & Cognition, 408–409. C&C ’13. Sydney, Australia: ACM. DOI: 10.1145/2466627.2481229.
Bogart, B.D.R., P. Pasquier, and S. J. Barnes. 2013. “An Integrative Theory of Visual Mentation and Spontaneous Creativity.” In Proceedings of the 9th ACM Conference on Creativity & Cognition, 264–273. New York: ACM.
Busch, K. 2009. “Artistic research and the poetics of knowledge.” ART & RESEARCH, A Journal of Ideas, Contexts and Methods 2 (2): 1–7.
Davies, J. 2019. Imagination. Pegasus Books.
Dörfelt, Matthias. 2011. “Selective Memory Theatre, Eine Installation über das selektive Wahrnehmen, Erinnern und Vergessen anhand von Bildern aus dem Internet.” Bachelor’s thesis, Muthesius Kunsthochschule. http://mokafolio.de/works/Selective-Memory-Theatre.
Franco, A. 2007. “Controlled Dream Machine.” MA thesis, University of Plymouth. http://www.anaisafranco.com/controlled.html.
Nir, Y. and G. Tononi. 2010. “Dreaming and the brain: from phenomenology to neurophysiology.” Trends in Cognitive Sciences 14 (2): 88–100.
Treur, J. 2011. “A Computational Agent Model Using Internal Simulation to Generate Emotional Dream Episodes”. In Biologically Inspired Cognitive Architectures 2011: Proceedings of the Second Annual Meeting of the BICA Society, 389–400. IOS Press.
Zhang, Q. 2009. “A computational account of dreaming: Learning and memory consolidation.” Cognitive Systems Research 10 (2): 91–101. DOI: 10.1016/j.cogsys.2008.06.002.
A Reflection on 'Machines that Dream'
Author commentary by Benjamin David Robert Bogart
The Watching and Dreaming body of work links AI with the apparent irrationality of dreams. The work is located at a paradoxical intersection of knowledge creation; the emphasis on formalism in computer science, the reductionism and empiricism in cognitive neuroscience, and the open-endedness and fluidity of artistic practice cannot be resolved as merely multidisciplinary.
I frame my doctoral work (Bogart 2014b), in which dreaming machines were initially developed, as art-as-research (Busch 2009) where I center the knowledge-generating capacity of artistic practices, make use of specialized knowledge, and consider argument the central pillar of all research. My aim is not merely to make use of knowledge in cognitive neuroscience, but to contribute through the generative capacity of artistic practice. This goal has not been satisfied, as validating a theory developed through art-as-research is challenging; for the Integrative Theory to be validated in cognitive neuroscience, empirical validation and quantitative analysis are required. Adhering to these disciplinary norms would elevate their primacy; I would be conducting science, not art-as-research. The richness and breadth of the computational model make it unamenable to reductionism and empirical validation. It is unclear what aspects of dreaming should be validated; waking/dreaming dynamics? Integration of waking "experience" in dreaming? Realism or causality in dreams? While this work has been worthy of citation by specialists, e.g. Davies (2019), it is not publishable in the disciplinary literature because of its lack of adherence to disciplinary methods. When publishing in interdisciplinary contexts, such as this one, the depth of my engagement with the cognitive neuroscience literature and its support of my argument are not easily communicated for a non-specialist audience. The work remains in limbo, unpublished in disciplinary contexts and merely summarized in interdisciplinary journals.
Artistic practice—with its interdisciplinarity, emphasis on richness and complexity, and lack of prescription of methods and outcomes—is the backbone for art-as-research. My doctoral work has led to peer-reviewed publications (Bogart and Pasquier 2011; Bogart and Pasquier 2013a; Bogart, Pasquier, and Barnes 2013; Bogart 2017a; Bogart 2017b) and peer-reviewed exhibitions (Bogart 2012; Bogart and Pasquier 2013b; Bogart 2014a), as well as talks and an ongoing body of work. Each of these outcomes have different audiences and different degrees of viewer engagement. No single outcome is a primary site for audience engagement with the knowledge produced through this art-as-research.
The very notion of validation in research requires the deployment of well-entrenched methods situated in stable disciplines. Projects such as this push boundaries without subscribing to the supremacy of methods of one domain over another. Validation according to the methods of all constituent disciplines may be unattainable. This project begs deep questions for interdisciplinary and arts-integrative research, such as: How can a radical inter- or a- disciplinary art-as-research practice exist if contributions can only be validated within disciplinary traditions? What becomes of specialist knowledge developed within a practice when it is not appreciated nor accessible outside of that discipline? Is there a place for a creative knowledge-generating art-as-research practice rooted in argument but independent of disciplinary methods of validation?
The most concise and complete description of the theory requires cognitive neuroscience literacy and remains unpublished:
1359.418 KB · Download / View
November 10, 2020