USING CREATIVE DIGITAL TECHNOLOGIES IN APHASIA REHABILITATION

Jane Marshall, Abi Roper, Timothy Neate, Madeline Cruice, Stephanie Wilson

City, University of London

J.Marshall@city.ac.uk

Abstract

Aphasia (language impairment) is common after stroke with negative impacts on all modes of communication. This article describes novel technologies developed at City, University of London, aiming to facilitate creative self-expression in people with aphasia. Outcomes from community workshops are summarised, in which people with aphasia trialled the technologies and gave feedback on them. Possible reasons for the success of the technologies are discussed.

Introduction

Every 40 seconds someone in the United States has a stroke (CDC Gov Facts)  and about a third of the survivors acquire aphasia, or loss of language (Ali et al, 2015).  Aphasia varies across individuals, but commonly involves difficulties in thinking of words and linking words into sentences. When aphasia is severe, speech may be absent or incomprehensible. All modalities of language are typically affected in aphasia, including reading, writing and the understanding of speech.  

Aphasia has a hugely negative impact on quality of life (Lam & Wodchis, 2010).  Those affected report shrinking social networks (Northcott et al, 2016) and feelings of isolation (Parr, 2007). Loss of language for conversation, telling stories, exchanging jokes and endearments can have a profound impact of the person’s sense of self, leading some researchers to describe aphasia as ‘identity theft’ (Shadden, 2010). 

For people living with aphasia, digital technology presents both opportunities and threats. On the plus side, there are many apps that enable people with aphasia to practise language exercises, and so augment their rehabilitation (Palmer et al, 2019).  Video conferencing technologies allow remote delivery of aphasia therapy and can bring people together for both 1:1 and group conversations (Woolf et al, 2016; Cruice et al, 2021; Pitt et al, 2018). Other mainstream devices, such as text-to-speech and voice recognition tools, can also help to compensate for aphasic difficulties (Marshall et al, 2018).  Less positively, most digital technologies require some degree of language for access, meaning that people with aphasia face digital exclusion (Menger et al, 2016).  Consequences include reduced opportunity to share ideas and participate in online creative expression.

The INCA Project

Based at City, University of London, the INCA Project (Inclusive Digital Content for People with Aphasia) aimed to address these concerns. We set out to develop new digital tools that support creative expression in aphasia, not just through language but also through visual and tactile media. We were also concerned with curation; for example, making it possible for people with aphasia to post their creations online and receive feedback.  As the INCA project draws to a close, we reflect here on what we achieved.

INCA has produced four novel technologies, in all cases using a process of co-design that involved people with aphasia (Neate et al, 2019a).  The first was MakeWrite, an iPad app that enables creative writing (Neate et al, 2019b).  The MakeWrite user selects a pre-existing text, either from the options already available in the app or from a personally chosen source. An ‘erase’ function then extracts a limited number of words from this text, which the user can further select and order to create a new poem or other literary output.  The second app, ComicSpin, facilitates the creation of a comic strip, through the selection of images and words (Tamburro et al, 2020).  The Inker app manipulates ‘conventional’ paper-based visual art to generate novel digital images (Neate et al, 2020a). The final technology is CreaTable, which was developed specifically for people with severe aphasia. Via the manipulation of physical cards, pucks or tiles, the user of CreaTable can select and combine words, images and sounds into a novel multi-media output (Neate et al, 2020b).  With all INCA apps the user has the option to post their creations online. The project is also developing a curation tool that will host online galleries and enable people with aphasia to discuss what they have created.

We ran nine community workshops involving people with aphasia in order to explore the accessibility of the apps and whether they promoted creative expression.  During these workshops, members of an aphasia community group were given access to the INCA technologies and invited to create novel outputs with them, working either individually or in groups.  Using feedback forms and interviews, we elicited members’ views about the usability of the apps and their satisfaction with what was created.  With one group we also discussed curation, for example selecting which outputs to share publicly and discussing how they should be presented.  Owing to the onset of COVID 19 our final four workshops were run entirely online. As a result, we also gained insights into the remote deployment of the INCA apps.

The 10 workshop participants reported that the INCA apps were easy to access and imposed few barriers.  This was true even for the participants who took part in the online workshops, where face to face support was not available. Usability was also reflected in the productivity of the workshops. For example, one group created an impressive 129 outputs across 5 meetings.  Reflecting on creativity, some group members voiced anxiety about their capacity to produce visual or literary art, particularly initially.  However, there were also numerous expressions of pride in what was produced.  For example, one individual asserted that the workshops involved genuine artistic creation in which the group had been ‘true to ourselves’. During the curation discussion members were very willing to share their creations publicly and discussed the depth of meaning that they evoked.  Some of these striking creations are illustrated in Figure 1.

Other benefits emerged during the research.  The workshops highlighted the value of the apps as stimulants of conversation. Participants shared what they had produced with each other, frequently making connections to personal events or associated ideas.  For example, a comic strip might be related to a daughter’s marriage proposal, or to an off-colour joke about murder; while an Inker image might be related to a war time dog fight or likened to the work of Paul Nash.  Speech and language therapists often seek to stimulate authentic conversation during aphasia rehabilitation and to re-kindle opportunities for self-expression.  Our experiences suggest that the INCA apps offer a medium for achieving this.   Another benefit relates to the online presence of people with aphasia.  In addition to the online posting of work by individuals, we curated a number of project level initiatives. An example was sharing MakeWrite poems on Twitter, including readings by Juliet Stevenson, Lost Voice Guy and people with aphasia (see: #AphasiaPoems | (city.ac.uk)

Discussion

How were the positive INCA outcomes achieved? The accessibility findings may well reflect the co-design process. This ironed out possible barriers to use during the apps’ development; and ensured that access involved minimal language and a small number of highly transparent steps.  In terms of creativity, we observed users responding warmly to the playful opportunities of the INCA technologies.  One commented: ‘you brought out the inner child in me’.  Many apps previously developed for people with aphasia present language tasks, such as thinking of the name of a picture, or spelling a word.  The INCA apps are not task based.   Rather, they invite the user to experiment and guarantee an output, even if that output is unexpected.  Thus, importantly, there is no possibility of failure. A further facilitation may have been constraint. All the INCA apps impose limitations on what can be done. For example, only three frames can be generated by Comic Spin, and with CreaTable the user can only work with the images, sounds and words that have been pre-uploaded. Some participants were frustrated by this and said that they wanted more words or pictures, or greater freedom to insert their own content. But for others the constraint was essential.   It meant that the language and cognitive demands were manageable.  As expressed by one participant: ‘Three words enough’.  There was another, egalitarian benefit from the constraints. The limitations imposed by the INCA apps mean that outputs from people with aphasia cannot be differentiated from those generated by unimpaired language users.   MakeWrite poems are strange and often wonderful, regardless of who produces them.

The INCA project is one of many to explore the potential contribution of technology for the rehabilitation of aphasia.  Its novelty lies in the nature of that contribution.  We did not set out to remediate the language impairment in aphasia or to address specific communication difficulties.  Rather we aimed to give new opportunities for creative self-expression.  Feedback from our participants suggests that, at least to some extent, we met this aim. 

Acknowledgements

The INCA project was funded by the EPSRC (Grant Number EP/P025587/1) and City, University of London. We thank the many people with aphasia who supported the work of INCA through our co-design and community workshops. Special thanks go to the members and leaders of Dyscover and Aphasia Re-Connect.

References

1. Ali, M., Lyden, P., & Brady, M. (2015). Aphasia and dysarthria in acute stroke: Recovery and functional outcome. International Journal of Stroke, 10(3), 400–406. https://doi.org/10.1111/ijs.12067 
2. Cruice, M., Woolf, C., Caute, A., Monnelly, K., Wilson, S., Marshall, J. (2021). Preliminary outcomes from a pilot study of personalised online supported conversation for participation intervention for people with Aphasia. Aphasiology, 35(10), pp. 1293–1317. doi:10.1080/02687038.2020.1795076.
3. Lam, J. & Wodchis, W. P. (2010) The relationship of 60 disease diagnoses and 15 conditions to preference-based health-related quality of life in Ontario hospital-based long-term care residents. Medical care, 48, 380–387. pmid:20220536
4. Marshall, J., Caute, A., Chadd, K., Cruice, M., Monnelly, K., Wilson, S., Woolf, C. (2018). Technology Enhanced Writing Therapy for People with Aphasia: Results of a Quasi-Randomised Waitlist Controlled Study. International Journal of Language and Communication Disorders. doi:10.1111/1460-6984.123
5. Menger, F., Morris, J., Salis, C. (2016) Aphasia in an Internet age: wider perspectives on digital inclusion, Aphasiology, 30, 2-3, 112-132, DOI: 10.1080/02687038.2015.1109050
6. Neate, T., Bourazeri, A., Roper, A. Stumpf, S., Wilson, S. (2019a) Co-Created Personas: Engaging and Empowering Users with Diverse Needs Within the Design Process. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2019), DOI: https://doi.org/10.1145/3290605.3300880 [video].
7. Neate, T., Roper, A., Wilson, S., Marshall, J. (2019 b) Empowering Expression for Users with Aphasia through Constrained Creativity. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2019). DOI: https://doi.org/10.1145/3290605.3300615 [video].
8. Neate, T., Roper, A., Wilson, S. (2020a) Painting a Picture of Accessible Digital Art. In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’20), Athens, Greece. DOI: https://doi.org/10.1145/3373625.3418019
9. Neate, T., Roper, A., Wilson, S., Marshall, J., Cruice, M. (2020b) CreaTable Content and Tangible Interaction in Aphasia. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2020). DOI: https://dx.doi.org/10.1145/3313831.3376490 
10. Northcott, S., Marshall, J., Hilari, K. (2016) What Factors Predict Who Will Have a Strong Social Network Following a Stroke?, Journal of Speech, Language, and Hearing Research, 59, 4, 772-783, 10.1044/2016_JSLHR-L-15-0201
11. Palmer, R., Dimairo, M., Cooper, C., Enderby, P., Brady, M., Bowen, A., Latimer, N., Julious, S., Cross, E., Alshreef, A., Harrison, M., Bradley, E., Witts, H., Chater, T. (2019) Self-managed, computerised speech and language therapy for patients with chronic aphasia post-stroke compared with usual care or attention control (Big CACTUS): a multicentre, single-blinded, randomised controlled trial. Lancet Neurology, 18, 821-33. https://doi.org/10.1016/S1474-4422(19)30192-9
12. Parr, S. (2007). Living with severe aphasia: Tracking social exclusion. Aphasiology, 21, 123 - 98. https://www.tandfonline.com/doi/full/10.1080/02687030600798337
13. Pitt, R., Theodoros, D., Hill, A., Russell, T. (2018) The impact of the telerehabilitation group aphasia intervention and networking programme on communication, participation, and quality of life in people with aphasia, International Journal of Speech-Language Pathology, pmid:30200788
14. Shadden, B. (2010) Aphasia as identity theft: Theory and practice.  Aphasiology, 19, 3-5, 211-223. https://doi.org/10.1080/02687930444000697
15. Tamburro, C., Neate, T., Roper, A., Wilson, S. (2020) Accessible Creativity with a Comic Spin. In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’20). DOI: https://doi.org/10.1145/3373625.3417012
16. Woolf, C., Caute, A., Haigh, Z., Galliers, J., Wilson, S., Kessie, A., Hirani, S. P., Hegarty, B., Marshall, J. (2016). A comparison of remote therapy, face to face therapy and an attention control intervention for people with aphasia: A quasi-randomised controlled feasibility study. Clinical Rehabilitation, 30(4), pp. 359-373. doi: 10.1177/0269215515582074

About the Authors