2019-104 – Deep Learning Systems for Scene Understanding, Path Planning and Navigation of Fire Fighter Teams

Background As a notorious hazard to modern human society, fire incidents take more lives than all other natural disasters combined in the United States. According to the report from the National Fire Protection Association (NFPA), there were 3,275 civilian deaths, 15,775 injuries and $11.6 billion lost in the year of 2014, due to fire incidents. As the most reliable force against fire, there are over one million fire fighters and three thousands fire departments nationwide. Firefighting is clearly a dangerous mission as it is a dynamic activity with many operations occurring, simultaneously. Firefighters must make prompt decisions in high-stress environments by constantly assessing the situation, planning their next set of actions, and coordinating with other colleagues, often with an incomplete picture of the situation. Maintaining situational awareness, knowledge of current conditions and activities at the scene, is critical for accurate decision making to result in the safe and successful navigation of the environment. Situational awareness can be heavily impacted by both external hazards related to fire, and the corresponding internal stresses experienced by first responders. Firefighters often carry various sensors in their personal equipment, namely infrared cameras, gas sensors, and microphones. However, there is still a gap between the state-of-the-art communication and information technology and existing firefighting protocols. Since the early 1970s, there has been limited change in the approaches for fireground communication among firefighters and the decision-making by the field commander. Improved data processing techniques can mine this data more effectively and be used to improve situational awareness at all times; thereby, improving real-time decision-making and minimizing errors in judgment induced by hazardous environmental conditions and anxiety levels. Technology Description Researchers from the University of New Mexico have created an automated system to mitigate loss of life by assisting firefighters in path planning during fire incidents through the implementation of situational awareness in real-time. This system not only provides real-time object detection and recognition, but also utilizes the data gathered on-site by employing state-of-the-art machine learning (ML) techniques. Crucial information necessary for safe navigation into and out of actively burning structures is identified and relayed back to firefighters to assist in the decision-making processes. The artificial neural network-based algorithm is sufficient to infer human recognition and posture detection to deduce a victim’s health level, to assist in prioritizing victims by need, and guide firefighters accordingly. This system provides fighters with path recommendations to avoid fire, which results in safe navigation through the burning structure; as well as, identifies critical scenarios and body positions that can be used to prioritize rescue. Andrew Roerick aroerick@innovations.unm.edu 505-277-0608