Conversational narratives consisting of anecdotes, experiences, and jokes are useful to facilitate interactional communication for augmentative and alternative communication (AAC) users. We developed a new AAC application that uses speech recognition and the Generative Pre-trained Transformer 4 (GPT-4) to generate four types of Japanese response phrases: boke, tsukkomi, neutral, and backchannel phrases. The purpose of this research is to examine (1) whether GPT-4 can be used to generate boke, tsukkomi, neutral, and backchannel phrases, (2) whether the generated phrases are used in conversations, and (3) what type of interaction occurs when the generated phrases are used.

A 22-year-old Japanese man with cerebral palsy participated in the development as an AAC user. We conducted four sessions with him, each lasting 20 to 30 min. In these sessions, we used our application in real conversations and exchanged opinions for further improvements. These sessions were videotaped for analysis.

The results were summarized as followed: (1) Our application could generate four types of Japanese phrases in a mean duration of 5.37 s (SD 2.47) after the partner's utterance. (2) The participant selected one tsukkomi, two boke, and four neutral phrases. The mean duration to produce the selected utterance from the end of the partner's utterance was 17.52 s (SD 14.71). (3) When the participant presented the generated Japanese jokes, the dyad laughed together.

In many developing countries, access to Alternative and Augmentative Communication (AAC) is limited to pictographic symbols designed to reflect languages and cultures other than that of the locality. Simple, functional communication is available, but the breadth and depth of local vocabulary is often restricted. This is usually due to the cost of customization and a lack of support necessary to create symbols in a similar style to those already offered or yet to be created for an individual with speech and language difficulties. Generative AI tools have the potential to affect the use of AAC in diverse situations and settings, by accelerating automated symbol development, supported by participatory evaluation. Symbol Builder uses AI models for image to text processes where individual symbol style descriptions are created. These captions are automatically paired with actual symbols from open licensed symbol sets. The next stage requires the provision of text prompts that engage the AI model, trained on the schema for a specific symbol set, to generate a symbol representing a new concept. The resulting image can be edited or accepted as a new pictograph which then goes through a voting process of acceptance. This is where AAC users and communication partners from the relevant linguistic and cultural setting decide if the symbols are ready to be published or require further adaptations. Finally, symbols are uploaded to a repository of open licensed AAC symbols for public use.

Effective communication can pose significant challenges for non-verbal children with Cerebral Palsy (CP). Augmentative and Alternative Communication (AAC) systems have helped many but can fail to meet the needs of some users. This research proposes a hybrid adaptive approach, utilizing sensors and machine learning (ML) algorithms to create a personalized mobile communication system for those whose abilities are ill-suited to existing approaches. The system aims to tailor to individual abilities, reducing the need for users to adapt to system requirements. Online surveys gathered data on gestures, actions, and sounds used by non-verbal CP children, informing a classification system and functional requirements. The participants reported 28 communication messages with diverse means of expression. Representative examples and their classification highlight the intricacies of non-verbal communication. The proposed architecture emphasizes real-time classification, multiple sensors, and a feedback loop for continuous improvement, enhancing communication for non-verbal children with CP.

ASD (Autism Spectrum Disorder) is a neurodevelopmental disorder that affects social interaction and communication skills. The purpose of this study was to develop a digital therapeutic application (AVATA-AAC: Amusing Verbal and Alternative Communication Tools for Children with ASD - Augmentative and Alternative Communication) to improve the communication skills of children with ASD and to evaluate the effectiveness of the application in improving vocabulary acquisition and AAC-based communication skills. The social validity of this digital therapeutic was also evaluated by assessing its acceptability to caregivers.

This digital therapeutic consists of two main parts: Joint Attention and AAC-based communication learning. The Joint Attention part focuses on developing skills such as pointing at objects, sharing attention with caregivers, and focusing attention. AAC-based communication learning part aims to understand vocabulary, lean AAC graphic symbols, and acquire communication expressions with AAC symbols in various real-life scenarios.

We conducted a user study with four children with ASD between the ages of 3 and 5. The results of the user study indicate that AVATA-AAC is effective in improving the language learning and communication of children with ASD. In addition, parents reported high levels of satisfaction. Future research should aim to diversify the contents and conduct user studies with a larger sample of children with ASD.