▼ 

In multitarget tracking scenarios with high false alarm rate and low target detection probability, data association plays a key role in resolving measurement origin uncertainty. The measurement origin uncertainty becomes worse when there are multiple detection per scan from the same target. This thesis proposes efficient data association algorithms for multitarget tracking under these conditions. For a multiple detection scenario, this thesis presents a novel Multiple-Detection Probabilistic Data Association Filter (MD-PDAF) and its multitarget version, Multiple-Detection Joint Probabilistic Data Association Filter (MD-JPDAF). The algorithms are capable of handling multiple detection per scan from target in the presence of clutter and missed detection. The algorithms utilize the multiple-detection pattern, which accounts for many-to-one measurement set-to-track association rather than one-to-one measurement-to-track association, in order to generate multiple detection association events. In addition, a Multiple Detection Posterior Cramer-Rao Lower Bound (MD-PCRLB) is derived in order to evaluate the performance of the proposed filters with theoretical bound. With respect to instantaneous track update, a continuous 2-D assignment for multitarget tracking with rotating radars is proposed. In this approach, the full scan is divided into sectors, which could be as small as a single detection, depending on the scanning rate, sparsity of targets and required track state update speed. The measurement-to-track association followed by filtering and track state update is done dynamically while sweeping from one region to another. As a result, a continuous track update, limited only by the inter-measurement interval, becomes possible. Finally, a new measurement-level fusion algorithm is proposed for a heterogeneous sensors network. In the proposed method, a maritime scenario, where radar measurements and Automatic identification System (AIS) messages are available, is considered. The fusion algorithms improve the estimation accuracy by assigning multiple AIS IDs to a track in order to resolve the AIS ID-to-track association ambiguity. In all cases, the performance of the proposed algorithms is evaluated with a Monte Carlo simulation experiment.

Thesis

Doctor of Philosophy (PhD)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

The main idea of this thesis is to de ne and formulate the role of bias estimation in multitarget{multisensor scenarios as a general framework for various measurement types. After a brief introduction of the work that has been done in this thesis, three main contributions are explained in detail, which exercise the novel ideas. Starting with radar measurements, a new bias estimation method that can estimate o set and scaling biases in large network of radars is proposed. Further, Cram er{Rao Lower Bound is calculated for the bias estimation algorithm to show the theoretical accuracy that can be achieved by the proposed method. In practice, communication loss is also part of the distributed systems, which sometimes can not be avoided. A novel technique is also developed to accompany the proposed bias estimation method in this thesis to compensate for communication loss at di erent rates by the use of tracklets. Next, bearing{only measurements are considered. Biases in this type of measurement can be di cult to tackle because the measurement noise and systematic biases are normally larger than in radar measurements. In addition, target observability is sensitive to sensor{target alignment and can vary over time. In a multitarget{ multisensor bearing{only scenario with biases, a new model is proposed for the biases that is decoupled form the bearing{only measurements. These decoupled bias measurements then are used in a maximum likelihood batch estimator to estimate the biases and then be used for compensation. The thesis is then expanded by applying bias estimation algorithms into video sensor measurements. Video sensor measurements are increasingly implemented in distributed systems because of their economical bene ts. However, geo{location and geo{registration of the targets must be considered in such systems. In last part of the thesis, a new approach proposed for modeling and estimation of biases in a two video sensor platform which can be used as a standalone algorithm. The proposed algorithm can estimate the gimbal elevation and azimuth biases e ectively. It is worth noting that in all parts of the thesis, simulation results of various scenarios with di erent parameter settings are presented to support the ideas, the accuracy, mathematical modelings and proposed algorithms. These results show that the bias estimation methods that have been conducted in this thesis are viable and can handle larger biases and measurement errors than previously proposed methods. Finally, the thesis conclude with suggestions for future research in three main directions.

Thesis

Doctor of Philosophy (PhD)

Advisors/Committee Members: Kirubarajan, Thia, Computational Engineering and Science.

▼ 

This dissertation concerns with the formulation of an improved multi-target multi-Bernoulli (MeMBer) filter and the use of the joint multi-target (JoM) estimator in an effective and efficient manner for a specific implementation of MeMBer filters. After reviewing random finite set (RFS) formalism for multi-target tracking problems and the related Bayes estimators the major contributions of this dissertation are explained in detail. The second chapter of this dissertation is dedicated to the analysis of the relationship between the multi-Bernoulli RFS distribution and the MeMBer corrector. This analysis leads to the formulation of an unbiased MeMBer filter without making any limiting assumption. Hence, as opposed to the popular cardinality balanced multi-target multi-Bernoulli (CBMeMBer) filter, the proposed MeMBer filter can be employed under the cases when sensor detection probability is moderate to low. Furthermore, a statistical refinement process is introduced to improve the stability of the estimated cardinality of targets obtained from the proposed MeMBer filter. The results from simulations demonstrate the effectiveness of the improved MeMBer filter. In Chapters III and IV, the Bayesian optimal estimators proposed for the RFS based multi-target tracking filters are examined in detail. First, an optimal solution to the unknown constant in the definition of the JoM estimator is determined by solving a multi-objective optimization problem. Thus, the JoM estimator can be implemented for tracking of a Bernoulli target using the optimal joint target detection and tracking (JoTT) filter. The results from simulations confirm assertions about its performance obtained by theoretical analysis in the literature. Finally, in the third chapter of this dissertation, the proposed JoM estimator is reformulated for RFS multi-Bernoulli distributions. Hence, an effective and efficient implementation of the JoM estimator is proposed for the Gaussian mixture implementations of the MeMBer filters. Simulation results demonstrate the robustness of the proposed JoM estimator under low-observable conditions.

Thesis

Doctor of Philosophy (PhD)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

In order to aid prediction of future maritime vessel trajectories, it is useful to examine historical vessel information. It is mandatory for large maritime vessels to broadcast, among other fields, spatial, speed, and course information using Automatic Identi- fication System (AIS) transponders. By processing a large historical dataset, it is possible to predict future vessel trajectories. The region of interest is discretized into a grid. Then, using offline computations, the historical data are used to determine second-order transition probabilities and speed information. Predictions will be car- ried out as an online process. If the destination is known, Dijkstra’s Algorithm is used to predict the vessel’s path. If the destination is not known, a path can still be de- termined using transition probabilities, but the prediction will be less accurate. The path is then smoothed using a line of sight algorithm to produce more realistic paths. Finally, the speed information is used to predict travel times. Real data were used to build the graph structure, and predictions were judged against real trajectories.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

In this thesis, a universal framework that is able to extract image spatial relationship among multiple appearance components is proposed, which can be employed to extract additional pattern in wide computer vision tasks. In order to demonstrate its usefulness, three novel algorithms solving different computer vision problems are presented as three main contributions of this thesis, which exercise this framework to improve their performances. Starting with the object tracking task, the framework is utilized to extract object's inner structure. The algorithm makes use of this inner structure to support a discriminative learning process for mitigating the classic error accumulation effect raising in numerous trackers. In this way, the tracking task is formulated as a prior regularized semi-supervised learning problem. To solve this particular problem, a multi-objective optimization approach is developed. The experiment conducted by the author demonstrates that this tracking algorithm advances state-of-the-art performance of object tracking. Next, the background subtraction task is studied. In this algorithm, the background is represented by a probabilistic topic model to deal with the dynamic background challenge. This topic model takes advantage of the framework to control topic proportions, which is shown a good descriptor for recurring pixel movement in dynamic background. In order to make the topic model suitable for this on-line task, an incremental learning approach is designed. In the experiment, this background subtraction algorithm outperforms the alternatives in challenging benchmarks. Finally, the proposed framework is expanded by applying it on a single image processing task, airborne ship detection. The algorithm handles this detection problem by modeling the ocean background and treating the ship pixels as outliers. For simultaneously encoding the dynamic nature and the local similarity of ocean background texture, the framework is used to explore the majority of pixel intensity across the image plane. An extensive experiment shows robustness and accuracy of the ship detection algorithm on a large number of tested images.

Thesis

Doctor of Philosophy (PhD)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ In this thesis, multiple ground target tracking algorithms are studied. From different aspects of the ground target tracking, three different types of tracking algorithms are proposed according to the specialties of the ground target motion and sensors employed. Firstly, the dependent target tracking for ground targets is studied. State dependency is a common assumption in traditional target tracking algorithms, while this may not be the true in ground target tracking as the motion of targets are constraint to certain path. To enhance the tracking algorithm for ground targets, starting with the dependency assumption, Markov Random Field (MRF) based Probabilistic Data Association (PDA) approach is derived to associate motion dependent targets. The driving behavior model is introduced to describe motion relationship among targets. The Posterior Cramer-Rao Lower Bound (PCRLB) is derived for this new motion model. Experiments and simulations show that the proposed algorithm can reduce the false associations and improve the predictions. Eventually, the proposed approach alleviates issues like the track impurity and coalescence problem and achieves better performance comparing to standard trackers assuming state independence. Ground target tracking using cameras is then studied. To build an efficient multi- target visual tracking algorithm, fast single target visual tracking is an important component. A novel visual tracking algorithm that has high speed and better or comparable performance to state-of-the-art trackers is proposed. The proposed approach solves the tracking task by using a mixed-motion proposal based particle filter with Ridge Regression observation likelihood calculation. This approach largely reduces the exhaustive searching in common state-of-art trackers while maintains efficient representation of the target appearance change. Experiments on 100 public benchmark videos, as well as a high frame rate benchmark, are carried out to compare the performance with the state-of-art published algorithms. The results of the experiment show the proposed tracker achieves good performance while beats other algorithms in speed with a large margin. The proposed visual target tracker is integrated into a new multiple ground tar- get tracking algorithm using a single camera. The multi-target tracker addresses the issues in the target detection, data association and track management aside from the single target tracker. A perspective aware detection algorithm utilizing the re- cent advanced Convolutional Neural Networks (CNN) based detector is proposed to detect multiple ground targets and alleviate the weakness of CNN detectors in detecting small objects. A hierarchical class tree based multi-class data association is presented to solve the multi-class association problem with potential misclassified detections. Track management is also improved utilizing the high efficiency detectors and a Support Vector Machine (SVM) based track deletion is proposed to correctly remove the dead tracks. Benchmarking is presented… Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

Commonly experienced bad weather conditions like fog, snow and rain generate pixel intensity changes in images and videos taken in outdoor environment and impair the performance of algorithms in outdoor vision systems. Hence, the impact of bad weather conditions need to be processed to improve the performance of outdoor vision systems. This thesis focuses on three most common weather conditions: fog, snow and rain. Their physical properties are first analyzed. Based on their properties, traditional methods are introduced individually to remove these weather conditions' effect on images or videos. For fog removal, the scattering model is used to describe the fog scene in images and estimate the clear scene radiance from single input images. In this thesis two scenario are discussed, one with videos and the other with single images. The removal of snow and rain in videos is easier than in single images. In videos, temporal and chromatic properties of snow and rain can be used to remove their impact. While in single images, traditional methods with edge preserving filters were discussed. However, there are multiple limitations of traditional methods that are based on physical properties of bad weather conditions. Each of them can only deal with one specific weather condition at a time. In real application scenarios, it is difficult for vision systems to recognize different weather conditions and choose corresponding methods to remove them. Therefore, machine learning methods have advantages compared with traditional methods. In this thesis, Generative Adversarial Network (GAN) is used to remove the effect of these weather conditions. GAN performs the image to image translation instead of analyzing the physical properties of different weather conditions. It gets impressive results to deal with different weather conditions.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

As a result of recent technological advances in modernized sensor sets and sensor platforms, sensor management combined with sensor platform path planning are studied to conduct intelligence, surveillance and reconnaissance (ISR) operations in novel ways. This thesis addresses the path planning and sensor management for aerial vehicles to cover areas of interest (AOIs), scan objects of interest (OOIs) and/or track multiple detected targets in surveillance missions. The problems in this thesis, which include 1) the spatio-temporal coordination of sensor platforms to observe AOIs or OOIs, 2) the optimal sensor geometry and path planning for localization and tracking of targets in a mobile three-dimensional (3D) space, and 3) the scheduling of sensors working in different (i.e., active and passive) modes combined with path planning to track targets in the presence of jammers, emerge from real-world demands and scenarios. The platform path planning combined with sensor management is formulated as optimization problems with problem-dependent performance evaluation metrics and constraints. Firstly, to cover disjoint AOIs over an extended time horizon using multiple aerial vehicles for persistent surveillance, a joint multi-period coverage path planning and temporal scheduling, which allows revisiting in a single-period path, is formulated as a combinatorial optimization with novel objective functions. Secondly, to use a group of unmanned aerial vehicles (UAVs) cooperatively carrying out search-and-track (SAT) in a mobile 3D space with a number of targets, a joint path planning and scanning (JPPS) is formulated based on the predictive information gathered from the search space. The optimal 3D sensor geometry for target localization is also analyzed with the objective to minimize the estimation uncertainty under constraints on sensor altitude, sensor-to-sensor and sensor-to-target distances for active or passive sensors. At last, to accurately track targets in the presence of jammers broadcasting wide-band noise by taking advantage of the platform path planning and the jammer's information captured by passive sensors, a joint path planning and active-passive scheduling (JPPAPS) strategy is developed based on the predicted tracking performance at the future time steps in a 3D contested environment. The constraints on platform kinematic, flyable area and sensing capacity are included in these optimization problems. For these multisensor path planning and decision making, solution techniques based on the genetic algorithm are developed with specific chromosome representations and custom genetic operators using either the non-dominated sorting multiobjective optimization (MOO) architecture or the weighted-sum MOO architecture. Simulation results illustrate the performance and advantage of the proposed strategies and methods in real-world surveillance scenarios.

Thesis

Doctor of Philosophy (PhD)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

A novel paper using Canada's real traffic accident data to propose a cost-prediction-based decision-making framework incorporating different ethical foundations for AVs.

Autonomous Vehicles (AVs) hold out the promise of being safer than manually driven cars. However, it is impossible to guarantee the hundred percent avoidance of collisions in a real-life environment with unpredictable objects and events. When accidents become unavoidable, the different reactions of AVs and their outcome will have different consequences. Thus, AVs should incorporate the so-called ‘ethical decision-making algorithm’ when facing unavoidable collisions. This paper is introducing a novel cost-prediction-based decision-making framework incorporating two common ethical foundations human drivers use when facing unavoidable dilemma inducing collisions: Ethical Egoism and Utilitarianism. The cost-prediction algorithm consists of Collision Injury Severity Level Prediction (CISLP) and Cost Evaluation. The CISLP model was trained using both Multinominal Logistic Regression (MLR) and a Decision Tree Classifier (DTC). Both algorithms consider the combination of relationships among traffic collision explanatory features. Four different Cost Evaluation metrics were purposed and compared to suit different application needs. The data set used for training and testing the cost prediction algorithm is the 1999-2017 National Collision Data Base (NCDB) which ensures the realistic and reliability of the algorithm. This paper is a novel paper using Canada's real traffic accident data to propose a cost-prediction-based decision-making framework incorporating different ethical foundations for AVs.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

Social networking sites such as Twitter, Facebook and Flickr play an important role in disseminating breaking news about natural disasters, terrorist attacks and other events. They serve as sources of first-hand information to deliver instantaneous news to the masses, since millions of users visit these sites to post and read news items regularly. Hence, by exploring e fficient mathematical techniques like Dempster-Shafer theory and Modi ed Dempster's rule of combination, we can process large amounts of data from these sites to extract useful information in a timely manner. In surveillance related applications, the objective of processing voluminous social network data is to predict events like revolutions and terrorist attacks before they unfold. By fusing the soft and often unreliable data from these sites with hard and more reliable data from sensors like radar and the Automatic Identi cation System (AIS), we can improve our event prediction capability. In this paper, we present a class of algorithms to fuse hard sensor data with soft social network data (tweets) in an e ffective manner. Preliminary results using are also presented.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

Big data is a term used to describe quantities of data that are too large to process using traditional means of data processing. The rise of such quantities of data amongst virtually every industry has lead to changes in data processing paradigms that favour distributed architectures over centralized processing. However great care must be taken when designing such processing architectures as slight overhead in computing or communication can nullify gains achieved from distributing computations. In this paper we discuss the theoretical elements involved in designing distributed systems and develop a heuristic for the performance of such a system. Our heuristic helps define when a problem requires distribution and informs designers in choosing the right topology to meet the needs of the problem and hardware involved. Finally we present results from our own distributed computing architecture applied to a prediction problem in radar image processing.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

This thesis focuses on the problem of underwater target tracking with consideration for realistic conditions using active sonar. This thesis addresses the following specific problems: 1) underwater detection in three dimensional (3D) space using multipath detections and an uncertain sound speed profile in heavy clutter, 2) tracking a group of divers whose motion is dependent on each other using sonar detections corrupted by unknown structured background clutter, 3) extended target tracking (ETT) with a high-resolution sonar in the presence of multipath detection and measurement origin uncertainty. Unrealistic assumptions about the environmental conditions may degrade the performance of underwater tracking algorithms. Hence, underwater target tracking with realistic conditions is addressed by integrating the environment-induced uncertainties or constraints into the trackers. First, an iterated Bayesian framework is formulated using the ray-tracing model and an extension of the Maximum Likelihood Probabilistic Data Association (ML-PDA) algorithm to make use of multipath information. With the ray-tracing model, the algorithm can handle more realistic sound speed profile (SSP) instead of using the commonly-assumed constant velocity model or isogradient SSP. Also, by using the iterated framework, we can simultaneously estimate the SSP and target state in uncertain multipath environments. Second, a new diver dynamic motion (DDM) model is integrated into the Probability Hypothesis Density (PHD) to track the dependent motion diver targets. The algorithm is implemented with Gaussian Mixtures (GM) to ensure low computational complexity. The DDM model not only includes inter-target interactions but also the environmental influences (e.g., water flow). Furthermore, a log-Gaussian Cox process (LGCP) model is seamlessly integrated into the proposed filter to distinguish the target-originated measurement and false alarms. The final topic of interest is to address the ETT problem with multipath detections and clutter, which is practically relevant but barely addressed in the literature. An improved filter, namely MP-ET-PDA, with the classical probabilistic data association (PDA) filter and random matrices (RM) is proposed. The optimal estimates can be provided by MP-ET-PDA filter by considering all possible association events. To deal with the high computational load resulting from the data association, a Variational Bayesian (VB) clustering-aided MP-ET-PDA is proposed to provide near real-time processing capability. The traditional Cramer-Rao Lower Bound (CRLB), which is the inverse of the Fisher information matrix (FIM), quantifies the best achievable accuracy of the estimates. For the estimation problems, the corresponding theoretical bounds are derived for performance evaluation under realistic underwater conditions.

Thesis

Doctor of Philosophy (PhD)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ Images and videos captured under adverse weather condition usually suffer from bad visual quality. The reduced visual quality can diminish the the human operator comprehension of the image or video content or even it can make the higher-level computer vision applications such as object segmentation or detection, very challenging. This thesis focuses on recovering the images or videos affected by different adverse weather conditions.Airborne videos are extensively used for object detection and target tracking. However, under bad weather conditions, the presence of clouds and haze or even smoke coming from buildings can make the processing of these videos very challenging. Current cloud detection or classification methods only consider a single image. Moreover, the images they use are often captured by satellites or planes at high altitudes with very long ranges to clouds, which can help distinguish cloudy regions from non-cloudy ones. We propose a new approach for cloud and haze detection by exploiting both spatial and temporal information in airborne videos. In this method, several consecutive frames are divided into patches. Then, consecutive patches are collected as patch sets and fed into a deep convolutional neural network. The network is trained to learn the appearance of clouds as well as their motion characteristics. Therefore, instead of relying on single frame patches, the decision on a patch in the current frame is made based on patches from previous and subsequent consecutive frames. This approach, avoids discarding the temporal information about clouds in videos, which may contain important cues for discriminating between cloudy and non-cloudy regions. Experimental results show that using temporal information besides the spatial characteristics of haze and clouds can greatly increase detection accuracy. The second problem that we address is the removal of haze problem in aerial images or airborne videos. Existing single-image dehazing methods that work on ground-to-ground images, do not perform well on aerial images. Moreover, current dehazing methods are not capable for real-time processing. We propose a new end-to-end aerial image dehazing method using a deep convolutional autoencoder. Using the convolutional autoencoder, the dehazing problem is divided into two parts, namely, encoder, which aims extract important features to dehaze hazy regions and decoder, which aims to reconstruct the dehazed image using the down-sampled image received from the encoder. In this proposed method, we also exploit the superpixels in two different scales to generate synthetic thin cloud data to train our network. Since this network is trained in an end-to-end manner, in the test phase, for each input hazy aerial image, the proposed algorithm outputs a dehazed version without requiring any other information such as transmission map or atmospheric light value. The idea used for generating the synthetic hazy images is further extended for generation of the synthetic hazy frame sequences for airborne video dehazing. The… Advisors/Committee Members: Shirani, Shahram, Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

In this thesis, the data association problem in multisensor-multitarget tracking is explored. Algorithms that improve data association performance by eliminating sensor biases or utilizing available domain knowledge are proposed. Sensor calibration and data association are two essential steps in multisensor-multitarget tracking systems to correct local measurements using estimated sensor biases and to associate measurements from different sensors. The problem of multitarget localization using time difference of arrival (TDOA) measurements at multiple unsynchronized sensors under measurement origin uncertainty is considered. A novel joint multidimensional association algorithm for multisensor synchronization is proposed. This algorithm is extended to a multiframe case to ensure the observability of unknown parameters consisting of target positions and sensor clock offsets. To improve the proposed algorithm's efficiency, a gating method and a multidimensional plus sequential two-dimensional association approach are developed. The Cram\'er-Rao lower bound for this problem is derived as a performance benchmark. Numerical results show that the proposed algorithm outperforms the algorithms that address sensor calibration and data association separately in terms of correct association rate and target position and sensor clock bias estimation accuracies. Exploring and exploiting domain knowledge can improve tracking performance, especially in the context of on-road target tracking. Due to traffic rules and limited lane capacity, on-road targets tend to move in an orderly manner along the centerline of each lane of the roads except for occasional lane changes. A novel sequence-aided 2D assignment (SA-2DA) algorithm, which integrates the target position sequence information into data association by utilizing this information in evaluating the probability of association hypothesis, is proposed. The sequence information is further exploited within the joint probabilistic data association (JPDA) framework, making it suitable for high false alarm rate or high association ambiguity scenarios, and within the tracking framework consisting of the interacting multiple model (IMM) estimator and the JPDA algorithm, making it suitable for tracking maneuvering targets. The uncertainty in target position sequence due to target lane-changing behavior is addressed by two strategies: a) The multiple-hypothesis method combined with the modeling of target lane-changing behavior as a homogeneous Markov chain; b) The track segment association algorithm. The posterior Cram\'er-Rao lower bound is derived for tracking multitarget along a multi-lane road. Numerical results show that the proposed algorithms (i.e., SA-2DA, SA-JPDA and SA-IMMJPDA) achieve better track accuracy and consistency than the existing multitarget tracking algorithms (i.e., standard 2DA, JPDA and IMMJPDA)) that do not make use of target position sequence information.

Thesis

Doctor of Philosophy (PhD)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

This thesis concerns with the localization, tracking, and sensor management in the Multiple-Input Multiple-Output (MIMO) radar systems. The collocated and widely-separated MIMO radars are separately discussed and the signal models are derived for both structures. The first chapter of the thesis is dedicated to the tracking and localization in collocated MIMO radars. A novel signal model is first formulated and the localization algorithm is developed for the derived signal model to estimate the location of multiple targets falling in the same resolution cell. Furthermore, a novel tracking algorithm is proposed in which the maximum bound on the number of uniquely detectable targets in the same cell is relaxed. The performance of the tracking and localization algorithms is finally evaluated using the tracking Posterior Cramer-Rao Lower Bound (PCRLB). After showing the impact of the antennas position on the localization CRLB, a novel sensor management technique is developed for the collocated MIMO radars in Chapter 4. A convex optimization technique is proposed for the antenna allocation in a single-target scenario. When multiple targets fall inside the same cell, a sampling-based technique is formulated to tackle the non-convexity of the optimization problem. The third chapter of this thesis also proposes new approaches for detection, localization, and tracking using a widely-separated MIMO radar. A scenario with multiple-scatterer targets is considered and the detection performance of both MIMO and multistatic radars will be evaluated in the designed scenario. To estimate the location of the multiple-scatterer target, a Multiple-Hypothesis (MH) based approach is proposed where the number and the location of multiple targets are both estimated. A particle filter based approach is also formulated for the dynamic tracking by a widely-separated MIMO radar. Finally, the performance of the MIMO radar and the miultistatic radar in detecting and localizing multiple-scatterer targets is studied.

Doctor of Philosophy (PhD)

Advisors/Committee Members: Kirubarajan, Thia, Reilly, James, Zhang, J. K., Electrical and Computer Engineering.

▼ 

This thesis addresses the problem of tracking multiple ground targets whose motion is dependent on one another. Multiple approaches which integrate the social force based motion model into different filtering algorithms are proposed. The social force concept has previously been used to model pedestrian motion where the interactions among pedestrians are described using social forces. First, the social force based motion model integrated into the Probability Hypothesis Density (PHD) framework is proposed. Two different implementations, namely, the Sequential Monte Carlo (SMC) technique and the Gaussian Mixture (GM) technique, are derived to implement the proposed Social Force PHD (SF-PHD) filter in ground target tracking scenarios. Next, a social-force-based motion model integrated into the stacked Kalman filter (stacked SF-KF) is developed and its multiple model (stacked IMM-SF-KF) variant is derived. Then, the assumption used in the proposed algorithms, that the actual values of the social force parameters are known, is not valid at all times and the assumption is relaxed. Hence, simultaneous parameter estimation techniques for the social force parameters during the tracking are proposed. Three approaches based on the state augmentation method, the Expectation Maximization (EM) method and the maximum likelihood method are derived. The maximum likelihood method can be implemented offline or online, depending on the requirement. The traditional Posterior Cramer Rao Lower Bound (PCRLB), which is the inverse of the Fisher information matrix, gives a bound on the optimal achievable accuracy of the estimated state of a target with independent motion. Subsequently, a modified performance measure based on the PCRLB for targets whose motion is dependent on each other is derived to validate the performance of the proposed algorithms. Finally, the PCRLB that accounts for unknown interactions is derived to validate the proposed simultaneous parameter estimation techniques. Simulated and real data are used to show the performance of the proposed algorithms and simultaneous parameter estimation techniques compared to the algorithms in the literature.

Thesis

Doctor of Philosophy (PhD)

This thesis addresses the problem of tracking multiple ground targets whose motion is dependent on one another. In target tracking literature, it is commonly assumed that a target’s motion follows a nearly constant velocity, constant turn or a constant acceleration model independent of the motion of other targets. But the actual behavior of a ground target may be more intricate than that and it is often affected by the motion of other targets, obstacles in the surrounding and its intended destination. Hence, a more sophisticated motion modeling technique, which integrates the various factors that affect the motion of ground targets, is needed. In this thesis, multiple approaches which integrate the social force based motion model into different filtering algorithms are proposed. The social force concept has previously been used to…

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

This thesis proposes a novel multiple traffic light recognition system based on videos captured by a monocular camera. Advanced driver assistance system (ADAS) and autonomous driving system (ADS) are becoming increasingly important to help drivers maneuvering vehicles and increase the vehicle and road safety in modern life. Traffic light recognition system is a significant part of ADAS and ADS, which can detect traffic light on the road and recognize different types of traffic lights to provide useful signal information for drivers. The proposed method can be applied to real complex environment only based on a monocular camera and is tested in real-world scenarios. This system consists of three parts: multiple traffic light detection, multi-target tracking and state classification. For the first step, a supervised machine learning method, support vector machine (SVM) with two integral features - histogram of oriented gradients (HOG) and histogram of CIELAB color space (HCIELAB), are used to detect traffic lights in the captured image. Then, a new multi-target tracking algorithm is presented to improve the accuracy of detection, reduce the number of false alarm and missing targets, by means of nearest neighbor data association, motion model analysis and Lucas-Kanade optical flow tracking and the region of interest (ROI) prediction. Finally, a SVM-based and a convolution neural network (CNN) based classifiers are introduced to classify the state of traffic lights, that provides the stop, go, warning, straight and turn information. Various experiments have been conducted to demonstrate the practicability of the proposed method. Both GPU-based and CPU-based programming can run real-time on the real street environment.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

Recently data generation has become an important research topic. Simulated data are not expensive and can be used immediately after being generated. Unlike simulated data, real data is expensive and time consuming to collect and in many cases real world data need to be cleaned before using. In this work we have developed a software that can generate soft data from events. This software generates output of NLP without using NLP complex technique, which can be used for testing fusion algorithms or using the generated data for testing data quality, as well as data mining algorithms. All the coding part has been done in C++ using Microsoft Visual Studio.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

Almost every module or project needs to be optimized to get the best results and reduce costs. Multi-sensor, multi-input trackers require a huge number of parameters to run, which have an undefined or unknown to the output of the tracker. It becomes very difficult to manually initialize these parameters to get a good output and there was a need to automate the process of selecting the parameters, validating them and initialing the tracker. The optimizer built to cater for these issues uses heuristic genetic algorithms – Particle Swarm Optimization and Gravitational Search Algorithm to find the best solutions for the problem. The optimizer works with the help of a Parameter Evaluator (developed earlier) to study the output of the tracker and incorporate the multi objective (Pareto) aspect of the problem. The Optimizer can find solutions to any optimization problem if hooked to a corresponding evaluator or fitness function calculator. This feature makes the Optimizer not just another module to the tracker but an independent application that could be used for general purpose optimization solutions.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

Ground Moving Target Indicator (GMTI) uses the concept of airborne surveillance of moving ground objects to observe and take actions accordingly. This concept was established in the late 20th century and was put to test during the Gulf War to observe enemy movement on the other side of the mountain. During the war, due to limitations of technology, information such as enemy movement were usually observed through human readings. With the improvement of surveillance technology, tracking individual target became possible, which allows the extraction of useful features for advance usage. Such features, known as tracks, are the results of GMTI tracking. Although the quality of the tracker plays a crucial role in the system performance of this paper, the development of the tracker is not discussed in this paper. The developed system will use simulated ideal GMTI tracks as input dataset. This paper presents an end-to-end system that includes Anomaly GMTI (AGMTI) track simulation, Pattern of Life (PoL) extraction and Anomaly Detection System (ADS). All the subsystems (AGMTI, PoL and ADS) are independent of each other, so they can either be replaced or disabled to resemble different real-world scenarios. The results from AGMTI will provide inputs for the rest of the subsystems. The results from PoL extraction will be used to improve the performance of ADS. The proposed ADS is a semi-supervised learning detection system in which the system takes prior information to support and improve detection performance, but will still operate without prior information. The AGMTI tracks simulator will be simulated with an open-sourced software called Simulation of Urban Traffic (SUMO). The AGMTI tracks simulator subsystem will make use of SUMO's API to generate normal and anomaly GMTI tracks. The PoL extraction will be accomplished by using various clustering algorithms and statistical functions. The ADS will use combination of various anomaly detection algorithms for different anomaly events including statistical approach using Gaussian Mixture Model Expectation Maximization (GMM-EM), Hidden Markov Model (HMM), graphical approach using Weiler-Atherton Polygon Clipping (WAPC) and various clustering algorithms such as K-means clustering, Spectral clustering and DBSCAN. Finally, as extensions to the proposed system, this paper also presents Contextual Pattern of Life (CPoL) and Grouped Anomaly Detection. The CPoL is an extension to the PoL to enhance the quality and robustness of the extraction. The Grouped Anomaly is extension to both AGMTI track simulator and ADS to diversify the possible scenarios. The results from the ADS will be evaluated. Details of implementation will be provided so the system can be replicated.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

Efficient multi-dimensional assignment algorithms and their application in multi-frame tracking

In this work, we propose a novel convex dual approach to the multidimensional dimensional assignment problem, which is an NP-hard binary programming problem. It is shown that the proposed dual approach is equivalent to the Lagrangian relaxation method in terms of the best value attainable by the two approaches. However, the pure dual representation is not only more elegant, but also makes the theoretical analysis of the algorithm more tractable. In fact, we obtain a su cient and necessary condition for the duality gap to be zero, or equivalently, for the Lagrangian relaxation approach to nd the optimal solution to the assignment problem with a guarantee. Also, we establish a mild and easy-to-check condition, under which the dual problem is equivalent to the original one. In general cases, the optimal value of the dual problem can provide a satisfactory lower bound on the optimal value of the original assignment problem. We then extend the purely dual formulation to handle the more general multidimensional assignment problem. The convex dual representation is derived and its relationship to the Lagrangian relaxation method is investigated once again. Also, we discuss the condition under which the duality gap is zero. It is also pointed out that the process of Lagrangian relaxation is essentially equivalent to one of relaxing the binary constraint condition, thus necessitating the auction search operation to recover the binary constraint. Furthermore, a numerical algorithm based on the dual formulation along with a local search strategy is presented. Finally, the newly proposed algorithm is shown to outperform the Lagrangian relaxation method in a number of multi-target tracking simulations.

Thesis

Doctor of Philosophy (PhD)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

A monocular Visual Odometry system has been developed and tested on different datasets and the outputs have been compared with the available ground truth information to analyze the precision of the system. This system is capable of estimating the 3D position of a ground vehicle robustly and in real time. One of the main challenges of monocular VO is the ambiguity of the scale estimation which is addressed by assuming that the ground is locally planar and the height of the mounted camera from the ground is fi xed and known. In order to improve the VO estimation and to help other stages of VO process an effective fi ltering approach is utilized. It is shown that an IMM fi ltering can address the needs of this speci c application, as the movement of a ground vehicle, is different depending on different scenarios. The results of simulation on the well-known KITTI dataset demonstrates that our system's accuracy improved compared to what is considered to be one of the best state-of-the-art monocular Visual Odometry system.

Thesis

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

In radar signal processing, the vulnerability of the desired signal to homogeneous and heterogeneous interferences increases as the communication traffic increases. The principal challenge in the radar system then is to mitigate the effects of cold (homogeneous) clutter, severe dynamic (heterogeneous) hot clutter and jamming interferences while estimating the states of targets under track. Space-Time Adaptive Processing (STAP) enhances the capability of radar systems to overcome this challenge. However, it is a sample-based system where the adaptive processing is sensitive to the underlying assumptions as well as the diversity of potential interferences. Hence, the performance of STAP deteriorates when basic assumptions are violated due to errors in receiver array elements, non-stationary nature of interferences, inadequate Independent and Identically Distributed (i.i.d.) sample data, and target like-signal in the training data set. This thesis proposes an Adaptive State Estimation (ASE) approach to characterize STAP used simultaneously in spatial and Doppler domains for non-stationary, homogeneous and heterogeneous systems. The contributions presented here are based on the adjustment of the weight vector and the update of associated interference covariance matrix by ASE to minimize the output noise power while maximizing Signal to Interference-plus-Noise Ratio (SINR) in the Mean Squared Error (MSE) sense. The integration of STAP principle with sequential state estimation in order to decode the target signal while rejecting the interferences due to non-stationary heterogeneous clutter and jammer effects without degrading performance is the key contribution of this paper. The Proposed STAP-ASE algorithm is shown to outperform its counterparts in terms of efficiency, IF-improvement factor, Signal to Interference-plus-Noise Ratio (SINR) convergence rate and target detection. Simulation results are presented to illustrate the performance of the proposed technique.

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼ 

Target classification has received significant attention in tracking literature. Algorithms for joint tracking and classification that are capable of improving tracking performance by exploiting the inter-dependency between target class and target kinematic behavior have already been proposed. However, in previous works the possible types of classes were assumed to be known a prior and the problem of class identification itself was not considered. In practice, the prior class information may not be always available. In this thesis, motivated by a people tracking problem, a joint class identification and target classification algorithm that can simultaneously build class types on the basis of target kinematic and feature measurements and classify targets according to the identified classes even when there is switching among classes is proposed. In addition, a new concept called "class quality" is introduced to improve the class identification and target classification accuracy. Accordingly, a modified performance evaluation metric for multiple object estimation, called Quality-based Optimal Subpattern Assignment (Q-OSPA), is proposed to quantify the class identification performance of the proposed algorithm. This metric provides more intuitively appealing results than the original OSPA metric when the quality of estimates is available. This new metric is also applicable in standard tracking problems where classification or class identification is not carried out, but a track quality measure is available as in the case of the Mnltiple Hypothesis Tracking (MHT) or the Joint Integrated Probabilistic Data Association (JIPDA) algorithm. Besides theoretical derivations, extensive simulations are presented to verify the effectiveness of the proposed algorithm.

Master of Applied Science (MASc)

Advisors/Committee Members: Kirubarajan, Thia, Electrical and Computer Engineering.

▼  This thesis focuses on joint model order detection and estimation of the parameters of interest, with applications to narrowband and wideband array signal processing in both off-line and on-line contexts. A novel data model that is capable of handling both narrowband and wideband cases with the use of an interpolation function and signal samples is proposed. In the off-line mode, Markov Chain Monte Carlo methods are applied to obtain a numerical approximation of the joint posterior distribution of the parameters under the condition that they have stationary distribution functions. On the other hand, if the distribution functions are nonstationary, the on-line approach is used. That approach employs a sequential implementation of Monte Carlo methods, applied to probabilistic dynamic systems. Four inter-related problems were addressed in the course of this thesis. 1. A new data structure based on interpolation functions and signal samples to approximate wideband signals was developed. This data model, after appropriate transformation, has similar features found in the conventional narrowband data model. Furthermore, as the novel data model is developed for the wideband scenario, it can also address the narrowband scenario without change of structure or parameters. This novel data model is the basis on which the MCMC and the SMC approaches solve the array signal processing problems developed in the subsequent chapters. 2. The first algorithm presents an advanced approach using sequential MC methods to beamforming for narrowband signals in white noise with unknown variance. Traditionally, beamforming techniques assume that the number of sources is given and the signal of interest (or target) is stationary within an observation period. However, in reality these two assumptions are commonly violated. The former assumption can be dealt with by jointly estimating the number of sources, whereas the latter severely limits the usefulness of conventional beamforming techniques when the target is indeed moving. In the case where the sources are moving, tracking the incident angles of the sources are required, and the accuracy of such tracking significantly affects the performance of signal separation and recovery, which is the objective of beamforming. The proposed method is capable of recursively estimating the time-varying number of sources as well as incident angles of the sources as new data arrive such that the signal amplitudes can be separated and restored in an on-line fashion. 3. The second algorithm presents an application of MCMC methods for the joint detection and estimation problem for the wideband scenario in white noise with unknown variance. In general, compared to the narrowband scenario, it is more difficult and cumbersome to solve this array signal processing problem in the wideband context. Conventional approaches tend to solve this problem in the frequency domain, and as such require a considerable amount of data to sustain accuracy, which imposes a large computational burden for these approaches.… Advisors/Committee Members: Reilly, James P., Kirubarajan, Thia, Electrical and Computer Engineering.