The STEMMA (Science, Technology, Engineering, Mathematics, Motivation and Accessibility) project is aimed at reducing the gap in STEM access and curricula related to gender and disabilities by empowering every person toward a scientific career. For this purpose, the project investigates the perceived difficulties and obstacles that may limit both women and visually impaired people from accessing educational and career paths in STEM areas.

The literature reports that there are numerous difficulties in accessing STEM content and careers; consequently, many people may be deterred from pursuing STEM studies for these reasons. However, the reasons could also be related to a lack of awareness and knowledge that with the right tools and educational approaches one can also tackle STEM careers. Motivational and persuasive tools and methodologies could be more proposed and disseminated in the literature, and among students who have to decide for their school career. We believe that this part is still underdeveloped, and the STEMMA project wants to make a contribution in this direction. Starting with a cognitive survey on STEM subjects and possible testimonials in the field, in a first phase of the project we intend to investigate among citizens about their awareness of this. For this purpose, we designed an interactive eBook and set up a survey questionnaire on STEM topics to be disseminated among the population. To this end, exploiting social networks and communities an accessible on line questionnaire has been distributed in Italy and will be now distributed in EU, in order to detect any different point of view also deriving from cultural aspects.

The study investigated the effectiveness of using an iPad App “Listen and Talk” and intergrating augmentative and alternative communication (AAC) intervention to improve social communication participation of a young adult with severe autism spectrum disorder (ASD), the intervention approach combined “preparation, functional vocabulary, sentence structure and using in milieu model” (PVSM Model). The study used case study method. Data were collected in 14 weeks sessions, including social communication performance of the user during intervention and interviewed his communication partner. The results showed that the intervention enhanced user’s participation in taking-a-taxi activity and generalized social communication performance with his communication partner. In conclusion, the study showed that a young adult with severe ASD could benefit from using iPad technology and AAC intervention to increase social communication participation.

In Germany alone, there are around 209,000 people with at least one arm missing or limited in its function and a further 295,000 people with musculoskeletal impairments as a result of an organic brain psychosyndrome (Federal Statistical Office, statistics on severely disabled people, 2022). For those affected, operating digital devices using touch, which is widely used on smartphones and tablets, can become a challenge or even impossible. To overcome these and other barriers, operating systems - such as iOS and Android - have accessibility services. For people with motor impairments in their arms / hands, Android offers the Voice Access and Switch Access accessibility service. With Voice Access, a smartphone / tablet can be operated using voice commands. Clickable elements are labelled or a grid is displayed. With switch access, the clickable elements can be controlled using various scanning methods. Both physical buttons and eye and facial gestures, which are detected by the Android device's front camera, can be used as switches. Both accessibility services enable people with motor impairments to access mobile Android devices. However, they are not suitable per se for all application contexts and affected persons. Voice Access, for example, requires good speech skills to ensure smooth and reliable operation. Particularly in people whose motor impairments are due to a neurological disease (e.g. ALS), their ability to speak is often impaired during the course of the disease. Furthermore, noises (e.g. video playback) can impair speech recognition in Voice Access, whereas switch access works more reliably. However, compared to conventional input methods (touch and mouse), it can be slow and require more interaction steps. An alternative to the aforementioned accessibility services can be a mouse, which is controlled by the affected person as a mouth stick using the tongue or camera-based head tracking. However, both alternatives have in common that additional hardware is required, the acquisition costs of which quickly exceed those of the mobile device and stigmatisation of the user cannot be ruled out.

As part of the "FourWays" research project, funded by the German Federal Ministry of Education and Research, we have developed a pilot Android app for environment control for people with motor impairments that only requires the built-in front camera. It can be operated via manual/automatic scanning or a pointer. When scanning, there is also a choice between linear scanning and scanning in 4 directions. The user controls the scanning or the pointer with facial gestures. We use the ML Kit from Google to detect these gestures. We optimise the sensor values obtained from this with our own algorithms in order to keep the rate of false detections and incorrect entries as low as possible in line with the individual abilities of the user. Specifically, the environment control utilises the following facial gestures:

- Head tilt (up, down, left, right)

- Head rotation (left, right)

- Blinking (distinction between left and right eye)

- Smile

- Open mouth

When developing the environment control, we placed great emphasis on personalisation. For the input methods, this means that each user can set which facial gestures they use (example configuration for manual linear scanning: previous element = turn head to the left, next element = turn head to the right, select element = blink). In addition, the sensitivity at which an input event is triggered can be set for each facial gesture.

In addition to the input methods, the app's tile-based user interface can also be personalised (e.g. number of tiles per screen, font size). For example, very large tiles enable touch operation for people with limited motor skills. The tiles are arranged in a hierarchical menu and each linked to an action (e.g. switch on the socket). The menu is defined via an XML file and can therefore be customised to the application context. Our environment control currently supports thermostats, sockets and lamps from various manufacturers as well as the playback of multimedia content on an external device with a large screen. Thanks to a plug-in concept, third parties can integrate additional devices at any time without having access to the source code of the environment control.

In the future, we plan to expand the environment control with a function for assisted communication and to supplement the configuration with an import and export function for profiles in order to simplify switching between different mobile devices and / or application contexts.

In the demo session at ICCHP 2024, we would like to start with a short presentation (15min) to explain the concept of our gaze-based app for environment control. Afterwards, attendees can either try out the app themselves or have a demo (45min). We hope that the demo session will provide us with feedback on the current state of development and contacts with potential future users.