Program

Electrical Engineeering

College

Engineering

Student Level

Doctoral

Start Date

7-11-2018 3:00 PM

End Date

7-11-2018 4:00 PM

Abstract

Firefighting is a dynamic activity with many operations occurring simultaneously. Maintaining situational awareness, defined as knowledge of current conditions and activities at the scene, are critical to accurate decision making. Firefighters often carry various sensors in their personal equipment, namely thermal cameras, gas sensors, and microphones. Improved data processing techniques can mine this data more effectively and be used to improve situational awareness at all times thereby improving real-time decision making and minimizing errors in judgment induced by environmental conditions and anxiety levels. This objective of this research employs state of the art Machine Learning (ML) techniques to create an automated system that is capable of real-time object detection and recognition utilizing currently gathered data to achieve improved situational awareness of firefighters on the scene. The algorithms authored effectively exploit the information gathered from the infrared camera by using a trained deep Convolutional Neural Network (CNN) system to identify, classify and track objects of interest. Crucial information is identified and relayed back to firefighters to assist their decision making processes and aid in safely navigating the environment. The ANN-based algorithm we are authoring is sufficient to infer human recognition and posture detection to deduce a victim’s health level to assist in prioritizing victims by need and guide firefighters accordingly. We also employ deep-learning based systems path planning and navigation, path reconstruction, scene segmentation, estimation of firefighter condition, Natural Language Processing for informing firefighter about the scene. We will integrate our search and rescue system with the image recognition system to produce a new search and rescue method that adapts to the changing environment by using Deep Q-learning.

Share

Import Event to Google Calendar

COinS
 
Nov 7th, 3:00 PM Nov 7th, 4:00 PM

End-to-End Deep Learning Systems for Scene Understanding, Path Planning and Navigation in Fire Fighter Teams

Firefighting is a dynamic activity with many operations occurring simultaneously. Maintaining situational awareness, defined as knowledge of current conditions and activities at the scene, are critical to accurate decision making. Firefighters often carry various sensors in their personal equipment, namely thermal cameras, gas sensors, and microphones. Improved data processing techniques can mine this data more effectively and be used to improve situational awareness at all times thereby improving real-time decision making and minimizing errors in judgment induced by environmental conditions and anxiety levels. This objective of this research employs state of the art Machine Learning (ML) techniques to create an automated system that is capable of real-time object detection and recognition utilizing currently gathered data to achieve improved situational awareness of firefighters on the scene. The algorithms authored effectively exploit the information gathered from the infrared camera by using a trained deep Convolutional Neural Network (CNN) system to identify, classify and track objects of interest. Crucial information is identified and relayed back to firefighters to assist their decision making processes and aid in safely navigating the environment. The ANN-based algorithm we are authoring is sufficient to infer human recognition and posture detection to deduce a victim’s health level to assist in prioritizing victims by need and guide firefighters accordingly. We also employ deep-learning based systems path planning and navigation, path reconstruction, scene segmentation, estimation of firefighter condition, Natural Language Processing for informing firefighter about the scene. We will integrate our search and rescue system with the image recognition system to produce a new search and rescue method that adapts to the changing environment by using Deep Q-learning.

 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.