Intelligent cognitive assistant leveraging NLP to support those with dementia

Researchers at the Florida Institute for Human and Machine Cognition are combining technology and healthcare to develop an application that uses natural language processing (NLP) to help dementia patients in real time with vocabulary recall.

Dr. Archna Bhatia, leading the project team, is focused on harnessing NLP to help seniors with Alzheimer’s Disease and related dementias retrieve words in the course of daily life. The effort is being supported by funding from the Massachusetts AI and Technology Center, a member of the a2Collective, which represents the Artificial Intelligence and Technology Collaboratories for Aging Research (AITC). The AITC program is funded by the National Institute on Aging (NIA), part of the National Institutes of Health.

The project’s goal is to develop an assistant that learns from the user and builds a personalized database utilizing natural language processing that can help users in real time by retrieving difficult words, and ultimately could identify the areas of memory where a person is having more difficulty. It could then provide feedback to strengthen the connections in areas where they are weaker.

The IHMC team working on the app development includes Dr. Bhatia, Dr. Peter Pirolli, Roger Carff and William de Beaumont, and collaborates with esteemed colleagues, Dr. George Sperling  from University of California Irvine (UCI), and Dr. Misha Pavel from Northeastern University, alongside a research assistant, Lingyu Gan, from UCI. The app is now in development.

It will identify users’ associative networks, the intricate web of connections woven between the words in each person’s memory, based on the way that person uses them. It then will tailor the feedback it offers “because each individual connects words differently based on their own knowledge and past experiences,” Bhatia said.

By way of example, Bhatia notes that in the brain, the names of all of a person’s neighbors are built in one associative network. The list of all of your medications, instructions from your healthcare providers are built in another. There could be, and there would be connections in between these different sub regions as well, but all of those connections are highly individualized.

Even in the typical pattern of aging, “these connections become weaker and that’s when we start to forget,” Bhatia said. Fatigue, distraction, among other things also play a role in weakening these connections and negatively impact the processing capabilities of these systems.

However, when neurodegenerative disorders and dementias are present, these disorders add yet another layer of disruption to the capacity of these systems to recall the words and phrases we may need to communicate.

This application, it is hoped, will keep collecting data from users as they input it. For example, whenever a user remembers something, they could note it with the assistant at a time when that memory and connection are strong and at the forefront.

Each time the user asks the assistant about something, behind the scenes, the assistant is learning about the associative networks that user relies upon. Ultimately, it could spot patterns in the queries from the user. For example, if a person is increasingly asking questions about medications, or about the names of people, the assistant could send that user games, puzzles or other stimulations to strengthen the connections that are weakening.

A longer-term goal Bhatia said, is that the database built across all of the app’s users could provide insights into what regions of the associative networks are impacted more often as Alzheimer’s disease progresses. Such an insight could help support targeted therapies and further research into these areas.

The Centers for Disease Control and Prevention notes that in 2020, nearly 5.8 million people are living with Alzheimer’s and related dementia. That figure is estimated to reach 14 million people by 2060. The prevalence of dementia-related disorders is a growing challenge for those diagnosed with it and their friends and family member who care for them.

IHMC is a not-for-profit research institute of the Florida University System where researchers pioneer science and technology aimed at leveraging and extending human capabilities. IHMC researchers and staff collaborate extensively with the government, industry, and academia to help develop breakthrough technologies. IHMC research partners have included: DARPA, the National Science Foundation, NASA, Army, Navy, Air Force, National Institutes of Health, IBM, Microsoft, Honda, Boeing, Lockheed, and many others.