A2PEX project leads latest edition of IHMC newsletter

How long can you sustain attention in a cognitively or physically demanding situation where a lapse in focus due to boredom or fatigue could have life-or-death consequences? 

That’s the question at the heart of a cooperative research project between the Florida Institute for Human and Machine Cognition (IHMC) and the Air Force Research Laboratory (AFRL) now under way. 

The project is the cover story of the latest issue of our newsletter, available here.

Assessing and Augmenting Performance in Extreme Environments (A2PEX) aims for real-time assessment and augmentation of Airmen cognitive performance in long-duration missions in extreme environments. 

The goal is to develop robust wearable devices and an integrated system that overcomes fatigue and stress by continually sensing, assessing, and augmenting human performance. The five-year project is sponsored by AFRL’s Materials and Manufacturing Directorate and supported by the 711th Human Performance Wing.  

Principal Investigators are IHMC’s Dr. Morley Stone and Dr. Tim Broderick. It draws on team members from IHMC’s human performance, robotics, and human-centered computing teams, and on Stone’s earlier work within AFRL on the Sense, Assess, Augment paradigm. 

Broderick says A2PEX “…is developing and manufacturing materials and technologies that will allow us to sense key biomarkers of fatigue in extreme environments, and to augment performance by mitigating fatigue. Aviation mishaps often are related to unrecognized fatigue and stress.” 

“If we can recognize and mitigate stress and fatigue, we can potentially save lives,” he says. “It is very hard to accurately measure the physiology and performance of a person during everyday activities – and even more so when a person works in an extreme environment.” 

Academic partners in the project are The Ohio State University and Georgia Institute of Technology. Industry partners are Triple Ring Tech, Abbott Biowearables, electroCore, and Polar. 

“We are excited about the possibilities of this program,” says Regina Shia, program manager at AFRL. “HMC has built a strong team with expertise that ranges from physiological and cognitive monitoring to materials and manufacturing advances. With a closed-loop human sustainment for a long-duration mission’s objective, this team could immensely impact the combat readiness and lethality for the Air Force.”  

Research agreement is key 

Closing the loop for human performance has been a longtime interest for IHMC researchers. More than a decade ago Stone published on the concept of the Sense, Assess, Augment — a human-centric control loop to improve performance.  

“Without the cooperative research agreement, we would not have access to AFRL research infrastructure that allows us to move sense and augmentation concepts out of the lab and into relevant environments. It’s a key component for us as we move into verification and validation before field testing,” Stone says. 

“We have a long-standing and productive relationship with the Air Force,” Broderick says. “We’re working with them in a collaborative way that is not typical. The collaboration benefits both parties and accelerates results.” 

The first year of the project will focus on refining and testing partner devices at IHMC and in the STRONG Lab (at Wright-Patterson Air Force Base). Triple Ring and Ohio State researchers are prototyping devices that will be tested in a joint exercise later in 2023. That exercise could include:  

— Air Force special operators and medics using personal protective equipment during a five-day surge; 

— Aircrew during long duration flights in the Pacific theatre; 

— Remotely Piloted Aircraft (RPA) teams during high tempo, protracted missions; and  

— Intelligence, Surveillance, and Reconnaissance (ISR) analysts during long shifts in classified facilities.  

Year two will focus on analyzing the data generated and revising prototype devices and the integrated system. 

The goal is to supply a sensor system that is comfortably wearable and doesn’t pose a safety risk, yet still supplies accurate electrical and mechanical data in a range of hostile environments. 

“If it’s not able to be worn comfortably, or if it gets in the way of doing a mission, it won’t be used,” Broderick says.  

Partners Polar, Abbott Biowearables, and Triple Ring are key to the effort to maximize adaptation of existing technologies to the needs and demands of the populations A2PEX will serve.  

The Air Force is interested in what this might mean for aviators on long-duration missions, but it’s not only pilots who could benefit. Personnel who engage in long-duration, high-cognitive load activities – such as pilots flying unmanned vehicles or analysts reviewing reconnaissance images – also could benefit.  

A promising approach for A2PEX centers on noninvasive vagal nerve stimulation. The vagal nerves are gatekeepers of the autonomic nervous system. These nerves send messages that adjust cognitive function and alertness, which are especially important to those working for extended stretches of time in high-stress environments. 

How A2PEX came to be 

The thread of A2PEX can be traced back more than a decade, with ties to work Stone did with AFRL and work Broderick was part of with the Defense Advanced Research Projects Agency (DARPA), particularly with the Peerless Operator Biological Aptitude, (PEERLESS).  

That project was designed to increase the biological aptitude of warfighters so that they can increase their adaptability and resilience in extreme conditions.  

In those early years, researchers were gaining the ability to measure quantitative data on human performance. With that knowledge, researchers gained the opportunity to fill a gap in the optimization of a system that until then had focused on how the technology performed — but hadn’t fully accounted for the impact on the human operator. 

The confluence of neurology, biomarkers, and wearable technology made this a “sweet spot” for optimizing the performance of elite warfighters and the technologies they control. PEERLESS, and now A2PEx, originate at least in part from this space. 

For A2PEX, researchers are looking for the intersection of what researchers would like to measure, what materials and sensors need to be developed, and how it can enhance capability and safety in extreme environments. 

While focus has been on pilots on long-duration missions, Stone and Broderick both note that the same concerns about how fatigue and biology could impact cognitive focus and accuity, other personnel could benefit from A2PEX’s findings.  

Analysts working extended shifts in Sensitive Information Compartmented Facilities (SCIF) and those working on Intelligence Surveillance Reconnaissance also face demands on their attention and focus. A2PEX could benefit them as well. 

The implications for military operators are clear, but the findings could ultimately produce benefits for long-haul truckers, commercial pilots on transoceanic routes, and medical personnel working long shifts, Stone notes. 

IHMC is a not-for-profit research institute of the Florida University System where researchers pioneer science and technology aimed at leveraging and extending human capabilities. IHMC researchers and staff collaborate extensively with the government, industry and academia to help develop breakthrough technologies. IHMC research partners have included: DARPA, the National Science Foundation, NASA, Army, Navy, Air Force, National Institutes of Health, IBM, Microsoft, Honda, Boeing, Lockheed, and many others. 

STEM-Talk: Dr. Mark Shelhamer on human spaceflight and missions to Mars

Mark Shelhamer was set upon a path that would take him from “nerdy band kid” to the chief scientist for NASA’s Human Research Program by his Uncle Stanley. 

“One of the smartest people I knew, even though he was not highly educated,” Shelhamer says. “He had been a surveyor in a coal mine. He was ‘the cool uncle,’ so he made it cool to be interested in science and math.”  

Uncle Stanley also gifted 10-year-old Mark with a Radio Shack 150-in-1 Electronics Kit. The idea that he could wire up a radio in a few minutes and listen to broadcasts from hundreds of miles away on something he built with his own hands, “was magic to me.” 

He’s been using that magic ever since in a career spent learning about spaceflight and its impact on human beings.  In this STEM-Talk, Shelhamer talks about his work with NASA and as an advisor to the commercial and consumer spaceflight industry.

Shelhamer is an otolaryngology professor at the Johns Hopkins School of Medicine and is the director of the school’s Human Spaceflight Lab. He also the director and founder of the Bioastronautics at Hopkins initiative

This fascinating discussion covers his role in NASA’s planned human mission to Mars and how he is investigating ways to maintain the health and performance of astronauts on long-duration spaceflights. He also discusses how this knowledge could be applied to improving healthcare on Earth. 

Episode 152 is now available on IHMC’s STEM-Talk webpage, as well as on your favorite podcast platform. Tune in for the conversation hosted by Dr. Ken Ford, IHMC’s founder and CEO, and co-host Dawn Kernagis, which covers:  

• How Shelhamer’s family ties to nursing led him to biomedical engineering, and ultimately, Massachusetts Institute of Technology and Johns Hopkins University. 

• Shelhamer’s research into motion sickness and vestibular issues and his more recent work on Space Motion Sickness. 

• Hiss belief in the need for a multidisciplinary approach to human space flight research — and how that approach could improve healthcare systems on Earth. 

• How close he believes we are to another exciting time with NASA’s Artemis program and plans to return to the moon. 

• His work on the effects of space radiation on people as well as the behavioral risks of being confined with a small group of people in tight quarters on a long-duration spaceflight.