Piano for Stroke

(To display subtitles in videos: activate the subtitles button and then, from the settings icon, select "automatic translation," after which, choose the desired language). From a technological development standpoint, the main challenges encountered in the realization of C4C research projects (and Brainart before that) have been numerous. Firstly, in these situations, we face the work of a multidisciplinary team: neuroscientists, psychologists, physiatrists, computer scientists, musicians, and healthcare professionals. What does this entail? It involves the existence of an intertwining of different languages. It's as if each profession involved in the team is the expression of a separate "tribe," with its own slang, needs, methodologies, timelines, and workspaces, etc. This was the initial hurdle: finding common ground for communication to understand each other's requests. Therefore, the role of a "translator" becomes essential, in our case embodied by the "scientist musician," someone with cross-disciplinary skills in the realms of music, science, and informatics. Danilo Spada plays a pivotal role in this translation process. Thanks to this "translation" role, we were able to facilitate dialogue among distant working environments, create a list of rehabilitation tasks, and subsequently outline a path for technological development.

In particular, returning to the "Piano for Stroke" research, once the rehabilitation objectives for upper limb movement and the context in which we would operate were established, we could begin to design our development roadmap. This involved identifying the most suitable device, studying its modes of interaction, and hypothesizing which types of musical functions and multimedia content would best suit our purpose.

Art in Rehabilitation: A Multidisciplinary Harmony

The harmonious integration of different elements is vital for the success of an artistic-technological-healing approach. These elements include:

1. Rehabilitation Tasks: Activate fine and gross motor movement of the upper limb in individuals affected by stroke/heart attack through exercises with musical and multimedia content.

2. Interaction Design: (a) Hardware: Create the most suitable set of physical instruments. (b) Software: Develop effective and user-friendly software with a simple user interface and as automated as possible.

3. Artistic Content: Integration of artistic content to emotionally stimulate the user, generating a motivational and fulfilling flow.

These elements must be designed holistically and coexist harmoniously within the proposed user experience. The absence of any one of these would mean the failure of the artistic-technological-healing approach. Other essential design levels include:

  1. Interpersonal Relationships: The involvement of caregivers and other patients in the rehabilitation process should also be related to the technological setup and channeled in a functional way within the user experience.

  2. Data recording To measure the quality of the intervention and monitor progress, a constant collection of data is necessary (while meeting privacy protection criteria).

Now, let's see how, in the case of the development of the "Piano for Stroke" research, we tackled these points.

The rehabilitation task has always been clear: activate fine and gross motor movement of the upper limb in individuals affected by stroke or heart attack through exercises (serious games) with increasing difficulty, incorporating musical or multimedia content, using the interaction of a MIDI keyboard (a small electronic piano that connects to a PC). Obviously, given my lack of experience in the medical field, I initially had to observe and evaluate the movement capabilities of individuals in these conditions. From there, we began to hypothesize what could be the most natural interaction methods with a keyboard and designed the rehabilitation path.

In the choice of the keyboard (point 2A in our list), the following factors played a key role:

1. Key resistance: A key that is too resistant would not have provided haptic feedback suitable for the perception of interaction.

2. Presence of alternative interactive modes: Rotary potentiometers, sliders, pads come in different numbers and forms on various keyboard models on the market. Our choice fell on a keyboard with rotary potentiometers and a set of square pads sensitive to finger pressure.

3. Dimensions The dimensions of the keyboard itself and those of the keys played a key role. A keyboard that is too bulky would have been impractical both in the ward and in the second phase of experimentation, where the keyboard had to be transported between users' homes and the CRAMS facility. The key size could not be too small for obvious reasons.

4. Costs The cost of the keyboard could not exceed the project budget.

In addition to choosing the keyboard, it was necessary to identify and select a laptop to run the serious games. Here too, analyzing devices on the market in terms of budget, performance, and dimensions (including the screen) was essential.

A remaining issue to resolve was the first part of the experimentation, the acute phase, which had to be conducted directly in the ward, thus securing the keyboard to the bed or wheelchair. We found a solution by using a clamp for instruments: a support typically used by drummers to attach electronic pads to their instrument. This way, we could also secure the keyboard at different angles depending on the user's range of motion impairment.

From a software perspective (2B), the choice fell on programming in Max/MSP, a music programming language that allows the development of interactive musical environments (both audio and MIDI), managing video streams, recording data, and customizing the user interface. This way, we could have control over every aspect of the data flow.

We then began to design possible games, incorporating different difficulty levels for both the required musical functions and motor coordination abilities. Indeed, we had to consider not only the impairment of limb use but also the likelihood that users might not have had previous instrumental musical skills. Thus, the initial game levels are simply about the "discovery" of the keyboard-object: for example, triggering videos with the simple constant rubbing of the hand on the keys or pads. Subsequent game levels introduce a more specific element of temporal synchronization. More complex levels also require melodic and pitch recognition skills. Simultaneously, the required movement in the games becomes increasingly intricate and specific on one hand, but broader on the other, utilizing even distant areas of the keyboard simultaneously.

Finally, the pulsating heart of this technological symphony is the emotional element. Art, carefully woven into the musical details, has proven to be the catalyst that transforms a rehabilitative exercise into a gratifying experience. From my perspective as a musician, what I seek is the right balance of variation in musical elements. This is played out on different planes of rhythm, timbre, and repertoire. It is crucial to incorporate the right number of musical contents that the user is familiar with. Prior knowledge of a piece aids in the self-explanation of the exercise itself and guides spontaneous interaction in the initial phases. Once the game's functioning is understood, users can progress to unfamiliar music. From this standpoint, it would be interesting to explore the possibility of composing original music for specific therapeutic or rehabilitative purposes.

From the experience of "Piano for Stroke" and other C4C research, a clear need emerges to design customizable rehabilitation systems, both in terms of physical interaction and content. This customization proves crucial to ensure inclusion and motivation, asserting an intervention direction that embraces the individuality of each patient.

In conclusion, this artistic journey in rehabilitation is not just an innovative approach to care but a promising bet on the power of art to illuminate the darkest moments, turning them into opportunities for growth and life perspectives. In the harmony of personalized care and creative technologies lies the promise of a rebirth beyond the challenges of rehabilitation.

Description

The Interreg Italy-Switzerland project "City 4 Care" has given me the opportunity to work on the research project "Piano for Stroke." With a team of scientists, doctors, musicians, and computer scientists, we attempted to reprogram a MIDI keyboard into an upper limb rehabilitation device for individuals who have experienced a stroke or heart attack. The keyboard then becomes an interface to interact with various serious games and musical/multimedia activities designed to stimulate both gross and fine motor movements of the affected limb.
Here is the link to the project page:
Progetto Interreg City for Care
Regione Lombardia
Asst Lecco, Bellano
ATS Brianza
Università di Pavia
G-RZ65T791V0
en_US