Program
Electrical Engineeering
College
Engineering
Student Level
Doctoral
Start Date
7-11-2018 3:00 PM
End Date
7-11-2018 4:00 PM
Abstract
Firefighting is a dynamic activity with many operations occurring simultaneously. Maintaining situational awareness, defined as knowledge of current conditions and activities at the scene, are critical to accurate decision making. Firefighters often carry various sensors in their personal equipment, namely thermal cameras, gas sensors, and microphones. Improved data processing techniques can mine this data more effectively and be used to improve situational awareness at all times thereby improving real-time decision making and minimizing errors in judgment induced by environmental conditions and anxiety levels. This objective of this research employs state of the art Machine Learning (ML) techniques to create an automated system that is capable of real-time object detection and recognition utilizing currently gathered data to achieve improved situational awareness of firefighters on the scene. The algorithms authored effectively exploit the information gathered from the infrared camera by using a trained deep Convolutional Neural Network (CNN) system to identify, classify and track objects of interest. Crucial information is identified and relayed back to firefighters to assist their decision making processes and aid in safely navigating the environment. The ANN-based algorithm we are authoring is sufficient to infer human recognition and posture detection to deduce a victim’s health level to assist in prioritizing victims by need and guide firefighters accordingly. We also employ deep-learning based systems path planning and navigation, path reconstruction, scene segmentation, estimation of firefighter condition, Natural Language Processing for informing firefighter about the scene. We will integrate our search and rescue system with the image recognition system to produce a new search and rescue method that adapts to the changing environment by using Deep Q-learning.
Included in
Artificial Intelligence and Robotics Commons, Graphics and Human Computer Interfaces Commons, Signal Processing Commons
End-to-End Deep Learning Systems for Scene Understanding, Path Planning and Navigation in Fire Fighter Teams
Firefighting is a dynamic activity with many operations occurring simultaneously. Maintaining situational awareness, defined as knowledge of current conditions and activities at the scene, are critical to accurate decision making. Firefighters often carry various sensors in their personal equipment, namely thermal cameras, gas sensors, and microphones. Improved data processing techniques can mine this data more effectively and be used to improve situational awareness at all times thereby improving real-time decision making and minimizing errors in judgment induced by environmental conditions and anxiety levels. This objective of this research employs state of the art Machine Learning (ML) techniques to create an automated system that is capable of real-time object detection and recognition utilizing currently gathered data to achieve improved situational awareness of firefighters on the scene. The algorithms authored effectively exploit the information gathered from the infrared camera by using a trained deep Convolutional Neural Network (CNN) system to identify, classify and track objects of interest. Crucial information is identified and relayed back to firefighters to assist their decision making processes and aid in safely navigating the environment. The ANN-based algorithm we are authoring is sufficient to infer human recognition and posture detection to deduce a victim’s health level to assist in prioritizing victims by need and guide firefighters accordingly. We also employ deep-learning based systems path planning and navigation, path reconstruction, scene segmentation, estimation of firefighter condition, Natural Language Processing for informing firefighter about the scene. We will integrate our search and rescue system with the image recognition system to produce a new search and rescue method that adapts to the changing environment by using Deep Q-learning.
