CONTEXT
ASD is a life-long condition characterized by impairments in social interaction and communication. Individuals with ASD also have restrictive, repetitive behaviors (CDC, 2014), and anxiety affects approximately 80% of people on the autism spectrum (Smith & White, 2020). Yet, while understanding social patterns produces challenges, many on the spectrum possess a high aptitude for perceiving architectural, spatial, mathematical, musical, and motion patterns (Crespi, 2016). Autism advocate Temple Grandin labeled those who create patterns to establish order, meaning, familiarity, and calm -- “pattern thinkers” (Grandin, 2013). Choreography often focuses on manipulating patterns to music, and the penchant for patterning has been shown to be soothing for many people with ASD (Scharoun et al, 2014). Could an immersive environment that produces opportunities for creating choreographic patterns have a palliative effect on anxiety?
Despite the strengths and abilities of people with ASD, most ASD research focuses on early intervention with the goal of diminishing impairments, rather than acknowledging that it is a syndrome with differential strengths and abilities (Hendricks, 2010). Instead, we took a strength-based approach, using game design practices and ideas that have been refined over several decades to improve quality of life, educate, encourage positive behavior, and facilitate healthcare needs (Ratan and Ritterfeld, 2009).
Initially proposed as a tablet-based app, the project that would become Choreografish pivoted to VR early in the process, following the literature on virtual reality and autism (Hong & Mi-Sung Kim, 2020). This allowed us to match the research interests of our collaborators with ASD and to investigate the arts-and-health potential of VR’s full perceptual immersion and capacity for producing wonder.
Integrating Choreographic Thinking and Virtual Reality
Choreography is a diverse artform. However, we found that constraining the definition of choreography helped translate general aspects of its Western concert practice to our game designers, computer engineers, and young adult collaborators with ASD who had never choreographed. We began thinking of choreography as a form of pattern thinking that emphasizes the interplay of moving objects and their coordination to music. The concept of “choreographic thinking” first came to our attention through the Synchronous Objects dance-and-data-visualization collaboration between former Frankfurt Ballett choreographer William Forsythe, Ohio State University (OSU) Professor of Dance Norah Zuniga Shaw, and computer visualization specialists at OSU. Their work suggests that choreographic thinking is a variant on the human impulse for organization, where intentionally designed relationships between moving objects are crafted to effect perceptions, sensations, actions, ideas, and emotions. Importantly, Forsythe and Zuniga Shaw’s work seems to frame choreographic thinking as a cognitive mode that can be generalized beyond dance—the social aspects of which can be a barrier to entry for people with autism spectrum disorder—prompting us to adopt their expansive definition of “choreography” from among many other possible conceptions of that term.
Our first iteration on a user interface involved a virtual pedestal-based control system that translated select choreographic variables into mechanically analogous digital controls like sliders and buttons. For simplicity, we delimited the choreographic variables for a school of fish to speed and scale (expansion and contraction of the school). These two variables, tempo and scale, enable the user to effect changes within a limited field, and potentially create a sense of music-based change. We simplified play in order to 1) reduce the risk of overstimulation and 2) translate the unfamiliar and mutable concept of “choreographic thinking” into clearly defined development goals for the game designers and engineers.
Choreografish players use the HTC Vive, a commercially available virtual reality headset to explore and interact with an expansive underwater world. The game allows users to upload any music of their choice and begin synchronizing the swim patterns of schools of fish. Users can record the swim patterns they create and play them back for review. The user can “teleport” to various spaces in the environment (including onto the back of a whale) to witness their creation from multiple vantage points. We drew design and engineering cues from a number of different games, especially the VR painting game Google Tiltbrush. Inspiration for movement and teleportation came from The Lab, and BOID behaviors were included to animate the schools of fish.
RESEARCH And Ongoing Questions
University of Utah Institutional Review Board-approved focus groups and interviews have been essential to understanding the user experience of Choreografish. The autism researchers on our team led the process, acquired consent, coordinated demos and feedback sessions, and coded data. Game designers and engineers led the demos and focus groups, and transformed the raw data from interviews about the user experience (UX) and user interface (UI) into action items to develop the software. Dance provided the theoretical framework of choreographic thinking that helped team members describe, interpret, contextualize, and evaluate the effectiveness of the UI and UX.
This iterative, heuristic feedback loop produced insights into the intuitive and emotional states that emerged through the VR experience and helped us make game-development decisions that were appropriate to the feedback gained from our research population partners. It has also led to new questions and challenges.
While focus group data confirmed the calming nature of the underwater environment, the same data also suggested the ineffectiveness of the design for the choreographic controls. The pedestal reflects an attempt to translate basic choreographic variables into a logical form, but the design constrains the role of the body, limiting its improvisational, idiosyncratic, and intuitive potential for producing change in the environment. Although working the sliders together in a rhythmic fashion can produce the illusion of a dancing school of fish, we came to see that the restrictive movements of the sliders interfered with freer, more fully embodied rhythmic responses. The restrictiveness of our design unexpectedly led to a longer learning curve and necessitated the development of a virtual tutorial to explain the range of available creative options. Furthermore, the constraining effect that we observed our controls had on the body was compounded by the fact that virtual reality is somewhat disembodying, rendering a person’s body invisible to themselves and making them reliant on proprioception and imagined tactility. We have not yet cracked the code on creating intuitive controls to facilitate “flow” in the body.
ConCLUSION AND Future Plans
Since high degrees of complexity cause anxiety in the lives of young adults with ASD, we are curious to see if Choreografish affects anxiety levels by enabling people to exercise order over complexity. Choreografish provides a model for co-design with youth with autism. It also illustrates the potential for interdisciplinary collaborations, in this case artists, social scientists, and game designers, to develop novel and innovative software solutions to address complex health concerns. The research team has begun to conduct trials to measure the impact of the game on anxiety for youth with ASD. Initial results are promising, showing reduced anxiety in a yet-to-be published feasibility study, but more research must be done before a claim of efficacy can be made. The team strongly believes that embodied choreographic thinking in virtual reality may have potential for leveraging pattern thinking in a non-deterministic, arts-based way to help young adults with ASD self-soothe through an immersive, creative engagement with technology.
Supplemental Material
POSTER.pptx
600.151 KB · Download / View
Bibliography
- Brandt, E. (2006). Designing Exploratory Design Games: A Framework for Participation in Participatory Design, Proceedings of the Ninth Conference on Participatory Design: Expanding Boundaries in Design (Vol. 1, pp. 57-66). ACM.
- Caron, M., Mottron, L., Rainville, C., & Chouinard, S. (2004). Do high functioning persons with autism present superior spatial abilities? Neuropsychologia, 42(4), 467-481.
- Center for Disease Control and Prevention. (2014) Prevalence of Autism Spectrum Disorder Among
- Children Aged 8 Years — Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2010. https://www.cdc.gov/mmwr/pdf/ss/ss6302.pdf
- Crespi, B. (2016). Autism as a disorder of high intelligence. Frontiers in Neuroscience, 10 (300). https://doi.org/10.3389/fnins.2016.00300
- Forsythe, W., Palazzi, M. & Zuniga Shaw, N. (2009) Synchronous Objects. https://synchronousobjects.osu.edu/
- Grandin, T. (2013). The autistic brain: Thinking across the spectrum. Boston, MA: Houghton Mifflin Harcourt.
- Hendricks, D. (2010). Employment and adults with autism spectrum disorders: Challenges and strategies for success. Journal of Vocational Rehabilitation, 2, 125-134.
- Hong, S., & Mi-Sung Kim, H. (2020). Virtual Reality-Based Intervention for Developmental Functioning in ASD: A Meta-Analysis. The Korean Journal of Developmental Psychology, 33(1), 85-100.
- Nicely, M. (2014, March 1). On Choreographic Thinking. Retrieved July 11, 2018, from https://dancersgroup.org/2014/03/on-choreographic-thinking/
- Ratan, R. & Ritterfeld, U. (2009). Classifying serious games. Serious games: Mechanisms and effects (pp. 10-24). New York, NY: Routledge.
- Scharoun, S. M., Reinders, N. J., Bryden, P. J., & Fletcher, P. C. (2014). Dance/Movement Therapy as an Intervention for Children with Autism Spectrum Disorders. American Journal of Dance Therapy, 36(2), 209-228.
- Smith, I., & White, S. (2020). Socio-emotional determinants of depressive symptoms in
- adolescents and adults with autism spectrum disorder: A systematic review. Autism, 1-16. https://doi.org/10.1177/1362361320908101.
- Spier, S. (2011). William Forsythe and the Practice of Choreography: It Starts From Any Point. Routledge.
- Valerie, S., Ventura, M., & Fengfeng, K. (2015). The Power of Play: The Effects of Portal 2 and Lumosity on Cognitive and Noncognitive Skills. Computers & Education, 80, 58-67.
Reviewing “Choreografish” for Ground Works
Reviewer commentary by Sydney Skybetter, Lise Worthen-Chaudhari, Eric Handman, and Veronica Stanich
Publication Date: July 27, 2021
DOI: https://doi.org/10.48807/2022.1.0004
Veronica Stanich
When Ground Works Advisor Cheryl Ball suggested that peer review doesn’t need to be a solitary, purely evaluative process, we got excited about the possibility of a conversation among reviewers. Literally, a conversation--in real time, in which reviewers armed with a few preliminary opinions about the work get together and talk about it, preferably with cocktails. We imagined that they might even extend or expand the conversation to include the work’s creators, fostering a back-and-forth that would fortify the work and counteract scholarly isolation.
I can confidently report that the review-as-real-time-conversation works. By “works,” I mean that project creators get good feedback and reviewers come away happy with the people they’ve met and the lively discussion they’ve had. And sometimes it really works; the reviewers get so engaged in the project and its questions that they do ask to be in touch with the project creators going forward. That’s what happened when Sydney Skybetter and Lise Worthen-Chaudhari got together to review “Choreografish,” an arts-based, virtual reality anxiety intervention for autism by Eric Handman and a team from the University of Utah.
I asked them to do a round-robin of writing, to describe how they each experienced a social and collaborative review process. I wanted to pull back the curtain a bit on peer review because it so often seems shrouded and distant, so here you can read about the submitter’s and the reviewers’ professional context, goals, and concerns, as well as their take-aways from the Ground Works review process.
Sydney Skybetter
Under normal circumstances, if I told you that one of my favorite memories from last year was an academic peer review process, you might be reasonably confident of my sarcasm. Too frequently, peer review is a slow, vestigial process designed to preserve critics’ anonymity and mitigate accountability for folks in positions of power. It also has the effect of maintaining notions of expertise tied to “disciplines” defined by gatekeepers, as well as other generally noxious academic traditions. Personally, as an artist without much trust or love lost for institutional systems, I never thought a peer review process could be equitable, galvanizing, and productively interdisciplinarily. a2ru Ground Works and “Choreografish” showed me just how shortsighted my cranky lack of imagination really was.
The “Choreografish” submission to Ground Works presented the project as a choreographic intervention in the tradition of artists like William Forsythe: a means to commingle contemporary virtual reality technology with choreographic pedagogic technique. In conversation with Lise and Veronica, what became gradually clear was that the project description belied the depth of its roots; it was in effect the culmination of a much longer genealogy of dance, notation, encodement and citational practice. The “Choreografish” proposal was convincing, if conservative. Our role as reviewers became less about articulating any deficits of the initiative, and more about how to resource the project so it could reach its full exponential potential. This spirit was contagious, and—without imposition—inspired my fellow reviewers and I to function less as detached, neutral observers and more like collegial, collaborative partners.
“Choreografish” thoughtfully applies choreographic practice to virtual reality, work that will no doubt shape dancerly engagement with the digital for years to come. What was most inspiring about the project was how it bravely braided strains of expertise that too rarely come into contact. It’s a first that I believe will originate new classes of embodied engagement with computer science generally and virtual reality specifically. As such, it is worth recalling that “Choreografish” is coming up and gaining traction not through a traditional, orthodox peer review process, but through a2ru, and by way of a growing community that thrives in slippery interstices. “Choreografish” is thus a doubly hopeful venture, in that it represents both powerful project work and a review process that aims to embolden inspired oddballs with novel ideas for a better embodied future.
Lise Worthen-Chaudhari
I was intrigued-but-wary when I first received Dr. Stanich’s (Veronica’s) invitation to review for a2ru Ground Works. On the one hand, I felt immediately excited about this novel opportunity. Engage with other makers in a structured academic improvisation at the interstices of dance, science, and medicine? Yes please! My heart and work lie at this intersection. I would LOVE to engage in meaningful ways with brilliant colleagues around emerging work in the space. Additionally, this invitation wasn’t a typical peer-review of work (for reference, I am asked to peer-review about one scientific or dance scholarly manuscript per month and have time to commit to approximately half of these invitations). No, this was an opportunity to challenge the dominant paradigm of how peer-review is operationalized—an opportunity to speak new transdisciplinary space into being with multidisciplinary co-creators. Yes yes, please!
On the other hand, I felt wary. What would the time commitment look like exactly? And would I be able to justify that time to my employer? As a Research Scientist in an academic medical center, I work every day to change culture and create respect for artscience innovations. This incredibly satisfying work involves a blunt reality: I must meet certain milestones, like securing external funding for my salary. Could I claim that this exciting, paradigm-changing opportunity with Ground Works would help me to get that next NIH grant? If the answer to this question was not a clear, quick “yes” then I’d need to do the work off-hours. Given these considerations, could I commit to this particular rabbit hole? I knew I wanted to engage in this experiment...but should I?
My decision came down to comfort level with the editor: Veronica and I had shared experiences from graduate school. Specifically, we had worked together to bridge dance research to human subjects research, so I knew and trusted her commitment to transdisciplinary action. Based on my positive shared experience with V, I decided to take the leap of faith with her and engage in this experiment to reimagine the peer-review process for transdisciplinary work.
I had a wonderful experience! In the end, the hardest part of the process turned out to be finding overlap in our schedules for live discussions. Not so different from scheduling a dance rehearsal or scientific collaborator meeting, really. On a practical note, once we got on the call together, our review of “Choreografish” took less time than a typical review would have taken me, but was more satisfying. Ground Works was well prepared to facilitate the experiment in collaborative peer-review. Firstly, we used their web-based, viewing platform that supported browsing between source material that took the form of scientific results (e.g., methods, data), artistic experience (e.g., videos of user interaction with the installation), and community impact (e.g., news coverage, user testimonials). Secondly, they provided clear guidance for our review discussion; this served as an effective structure for our conversational improv.
The platform took a minute to get oriented within, but then really worked for me, allowing my appreciation of the work to unfold organically, as a personal exploration rather than as a linear narrative that the authors fed to me, thus supporting audience autonomy on a fundamental level. Experienced this way, it quickly became clear that the project grew from compelling roots in dance technology, physical medicine, and health science and could have fit within a traditional journal in any of those defined disciplines. But as a transdisciplinary creative installation, the work manifested beautifully on the Ground Works publishing platform.
Our discussions about the work were invigorating and enabled our solo thoughts to grow larger than the sum of their parts when shared between us. What emerged from the process was as much curation as review; yes we clearly agreed that the work should be published but we were also able to develop concrete suggestions to maximize impact for each scholarly front that the work touched on (e.g., dance, interactive tech, science, medicine) supporting the work in an integrated manner. Sydney, Veronica, and I were able to work together to position the work in terms of dance and interactive tech. To ensure validity of the work in the clinical research domain, I stepped in with specific recommendations such as dialing back claims of efficacy, presenting supporting data clearly within the publication, and celebrating the work in terms of the clinical research stages completed therein: Phase I design and feasibility. In retrospect, I am glad that I engaged this experiment in transdisciplinary review and feel that the conversational review format served to increase my agency as a maker at the intersection of art, science, and medicine.
Eric Michael Handman
I think I collaborate because I need help doing things. My private joke is that I have lots of projects in various stages of stagnation—so I need to work with others to get things done. When I received the initial information about Ground Works from my College of Fine Arts office, it struck me as a new kind of publishing enterprise that might suit the “slippery” (to use Sydney’s descriptor), hard-to-define, hard-to-fund (sigh…) kind of research I was doing. The word “heartful” comes to mind immediately when I think about the process of working with Veronica as she helped me incorporate feedback from Sydney and Lise. What reviewers! And what an education. I realized that my goal was not just to publish, but to learn something more about conducting research, and thereby improve at teaching research methods for artists. When I teach graduate students, I find myself working to help them normalize the chaos of the researcher’s mind that is often hidden by the pristine organization and polished style of the prose. Veronica allowed for me to work through the “shitty first drafts,” incorporating the generous feedback to help my presentation of the research rise to a standard of which we could all be proud.
Ground Works provided an environment in which taken-for-granted notions of “research” and “standards” could be ongoing subjects of discussion. Working with Veronica, Sydney, and Lise showed me the dark, unfinished hallways that needed better illumination, the dead ends that forced me to pivot and define yet again what I really wanted to say—and what I needn’t, gently challenging me to excise any iffy-smelling claims for which the data was too inconclusive. I’m reminded of choreographer Sean Curran once telling me in a rehearsal as he cut and reorganized sections of choreography, “It’s not what you do with the pencil that counts, it’s what you do with the eraser!”
A word about encouragement. Encouragement is a necessary thing. Artists are vulnerable and for some of us, impostor syndrome often prowls nearby. What are we doing in Research Universities? Can I produce Knowledge? Will I know it when I’ve done it? I love wrestling with these questions because of my own (at times) ambivalence to their answers. “It’s the question that drives us,” Trinity says to Neo in The Matrix. Artists name concepts, make provisional statements, suggest possibilities, test the elasticity of boundaries and the permeability of disciplinary borders. We court the unknown and wind up producing art and theory. And maybe our investigations provide models for those who come after us. Regardless, it’s good to have a cadre of collegial and committed artist-editors in your corner who see the potential of your project, and are willing to share their time to help you hit that most common of academic targets: publication. A2RU saw a need, and Ground Works provides the space for those hybrid artist-researchers who are focused on targets no one else sees.
Eric Michael Handman is the lead author on “Choreografish,” crossing disciplines of dance, game design, and autism research.
Veronica Stanich
I’m the beginning and the end of the circle; I get the last word.
I asked Lise, Sydney, and Eric to write about their experience with Ground Works review because I sensed that it had been so positive and gratifying; now, reading their abounding positive commentary, I squirm with both delight and chagrin. Delight, because Ground Works is so young, and so many risks have been taken but not all of them have proven fruitful (yet), and it’s very satisfying to see confirmed our hunch that reviewing together could be a generative thing. Chagrin, because I think I must balance their rosy report on Ground Works collaborative review by saying something about how the process has sometimes faltered and failed. I was horribly unclear in the early reviews, leaving reviewers confused and submitters rightfully pissed off. Even in the “Choreografish” process, there were leaks; for example, the supporting data that Lise pegged as critical for the article didn’t make it into the final publication. For sure, the collaborative review process is itself still in process.
Eric, Lise, and Sydney, thank you for being part of the review, and for taking the time to reflect and write after the fact. Everyone else, jump in. Submit to Ground Works or check the box on the a2ru Community Profile and Engagement inventory that says you want to be a reviewer for Ground Works.