subject:(High dimensional data)

University of Georgia

1. Safo, Sandra Esi. Design and analysis issues in high dimension, low sample size problems.

Degree: PhD, Statistics, 2014, University of Georgia

URL: http://purl.galileo.usg.edu/uga_etd/safo_sandra_e_201408_phd

► Advancement in technology and computing power have led to the generation of data with enormous amount of variables when compared to the number of observations.… (more)

▼ Advancement in technology and computing power have led to the generation of data with enormous amount of variables when compared to the number of observations. These types of data, also known as high dimension, low sample size, are plagued with different challenges that either require modifications of existing traditional methods or development of new statistical methods. One of these challenges is the development of Sparse methods that use only a fraction of the variables. Sparse methods have been shown to perform better at making predictions on real high dimensional problems, hence justifying their studies and use in practice. This dissertation considers three novel methods for designing and analyzing high dimensional studies. We first propose new sample size method to estimate the number of samples required in a training set when allocating new entities into two groups. The methodology exploits the structural similarity between logistic regression prediction and errors-in-variables models. Secondly, we consider the problem of assigning future observations to known classes using linear discriminant analysis. We propose a new classification approach of generalizing existing binary linear discriminant methods to multiclass methods. Our methodology utilizes the equivalence between discriminant subspace using Fisher's linear discriminant analysis and basis vectors of between class scatter. We apply the proposed method to two sparse methods. Thirdly, a general framework that results in sparse vectors for many multivariate statistical methods is developed. The framework uses the relationship between many multivariate statistical problems and generalized eigenvalue problem. We illustrate this framework with two multivariate statistical methods- linear discriminant analysis for classifying new entities into more than two groups, and canonical correlation analysis for studying associations between two different high dimensional data types. The effectiveness of the proposed methods in this dissertation is evaluated by various simulated processes and real data analyses on microarray and RNA sequencing (RNA-seq) data. Advisors/Committee Members: Kevin K Dobbin.

Subjects/Keywords: High dimensional data

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Safo, S. E. (2014). Design and analysis issues in high dimension, low sample size problems. (Doctoral Dissertation). University of Georgia. Retrieved from http://purl.galileo.usg.edu/uga_etd/safo_sandra_e_201408_phd

Chicago Manual of Style (16^th Edition):

Safo, Sandra Esi. “Design and analysis issues in high dimension, low sample size problems.” 2014. Doctoral Dissertation, University of Georgia. Accessed February 24, 2020. http://purl.galileo.usg.edu/uga_etd/safo_sandra_e_201408_phd.

MLA Handbook (7^th Edition):

Safo, Sandra Esi. “Design and analysis issues in high dimension, low sample size problems.” 2014. Web. 24 Feb 2020.

Vancouver:

Safo SE. Design and analysis issues in high dimension, low sample size problems. [Internet] [Doctoral dissertation]. University of Georgia; 2014. [cited 2020 Feb 24]. Available from: http://purl.galileo.usg.edu/uga_etd/safo_sandra_e_201408_phd.

Council of Science Editors:

Safo SE. Design and analysis issues in high dimension, low sample size problems. [Doctoral Dissertation]. University of Georgia; 2014. Available from: http://purl.galileo.usg.edu/uga_etd/safo_sandra_e_201408_phd

2. Freyaldenhoven, Simon. Essays on Factor Models and Latent Variables in Economics.

Degree: Department of Economics, 2018, Brown University

URL: https://repository.library.brown.edu/studio/item/bdr:792643/

► This dissertation examines the modeling of latent variables in economics in a variety of settings. The first two chapters contribute to the growing body of… (more)

▼ This dissertation examines the modeling of latent variables in economics in a variety of settings. The first two chapters contribute to the growing body of work on how best to find meaning in high dimensional datasets. In Chapter 1, I extend the theory on factor models by incorporating ``local'' factors into the model. Local factors affect a decreasing fraction of the observed variables. This implies a continuum of eigenvalues of the covariance matrix, as is commonly observed in applications. I find that the factor strength at which the principal component estimator gives consistent factor estimates coincides with the factor strength at which factors are economically important in many economic models. I further propose a novel class of estimators for the number of those factors. Unlike estimators that have been proposed in the past, my estimators use information in the eigenvectors as well as in the eigenvalues. Monte Carlo evidence suggests significant finite sample gains over existing estimators. In an empirical application, I find evidence of local factors in a large panel of US macroeconomic indicators. In Chapter 2, I establish that Sparse Principal Components can consistently recover local factors. I further develop a unifying framework that encompasses both factor augmented regressions and high-dimensional sparse linear regression models. I argue that factor augmented regressions with local factors partially fill the gap in between those approaches. Chapter 3 considers a linear panel event-study design in which a latent factor may affect both the outcome of interest and the timing of the event. This scenario would invalidate a traditional difference-in-differences approach. However, this chapter presents a novel method that nevertheless allows a practitioner to identify the causal effect of the event on the outcome of interest. A Covariate related to the latent factor, but unaffected by the event, is used to achieve identification via a GMM representation. This approach permits causal inference in the presence of pre-event trends in the outcome. Advisors/Committee Members: McCloskey, Adam (Advisor), Shapiro, Jesse (Reader), Renault, Eric (Reader), Kleibergen, Frank (Reader).

Subjects/Keywords: high dimensional data

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Freyaldenhoven, S. (2018). Essays on Factor Models and Latent Variables in Economics. (Thesis). Brown University. Retrieved from https://repository.library.brown.edu/studio/item/bdr:792643/

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Freyaldenhoven, Simon. “Essays on Factor Models and Latent Variables in Economics.” 2018. Thesis, Brown University. Accessed February 24, 2020. https://repository.library.brown.edu/studio/item/bdr:792643/.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Freyaldenhoven, Simon. “Essays on Factor Models and Latent Variables in Economics.” 2018. Web. 24 Feb 2020.

Vancouver:

Freyaldenhoven S. Essays on Factor Models and Latent Variables in Economics. [Internet] [Thesis]. Brown University; 2018. [cited 2020 Feb 24]. Available from: https://repository.library.brown.edu/studio/item/bdr:792643/.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Freyaldenhoven S. Essays on Factor Models and Latent Variables in Economics. [Thesis]. Brown University; 2018. Available from: https://repository.library.brown.edu/studio/item/bdr:792643/

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

University of Alberta

3. Fedoruk, John P. Dimensionality Reduction via the Johnson and Lindenstrauss Lemma: Mathematical and Computational Improvements.

Degree: MS, Department of Mathematical and Statistical Sciences, 2016, University of Alberta

URL: https://era.library.ualberta.ca/files/cm039k5065

► In an increasingly data-driven society, there is a growing need to simplify high-dimensional data sets. Over the course of the past three decades, the Johnson… (more)

Subjects/Keywords: Dimensionality Reduction; High Dimensional Data; Johnson Lindenstrauss

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Fedoruk, J. P. (2016). Dimensionality Reduction via the Johnson and Lindenstrauss Lemma: Mathematical and Computational Improvements. (Masters Thesis). University of Alberta. Retrieved from https://era.library.ualberta.ca/files/cm039k5065

Chicago Manual of Style (16^th Edition):

Fedoruk, John P. “Dimensionality Reduction via the Johnson and Lindenstrauss Lemma: Mathematical and Computational Improvements.” 2016. Masters Thesis, University of Alberta. Accessed February 24, 2020. https://era.library.ualberta.ca/files/cm039k5065.

MLA Handbook (7^th Edition):

Fedoruk, John P. “Dimensionality Reduction via the Johnson and Lindenstrauss Lemma: Mathematical and Computational Improvements.” 2016. Web. 24 Feb 2020.

Vancouver:

Fedoruk JP. Dimensionality Reduction via the Johnson and Lindenstrauss Lemma: Mathematical and Computational Improvements. [Internet] [Masters thesis]. University of Alberta; 2016. [cited 2020 Feb 24]. Available from: https://era.library.ualberta.ca/files/cm039k5065.

Council of Science Editors:

Fedoruk JP. Dimensionality Reduction via the Johnson and Lindenstrauss Lemma: Mathematical and Computational Improvements. [Masters Thesis]. University of Alberta; 2016. Available from: https://era.library.ualberta.ca/files/cm039k5065

University of Minnesota

4. Ye, Changqing. Network selection, information filtering and scalable computation.

Degree: PhD, Statistics, 2014, University of Minnesota

URL: http://hdl.handle.net/11299/172631

► This dissertation explores two application scenarios of sparsity pursuit method on large scale data sets. The first scenario is classification and regression in analyzing high… (more)

▼ This dissertation explores two application scenarios of sparsity pursuit method on large scale data sets. The first scenario is classification and regression in analyzing high dimensional structured data, where predictors corresponds to nodes of a given directed graph. This arises in, for instance, identification of disease genes for the Parkinson's diseases from a network of candidate genes. In such a situation, directed graph describes dependencies among the genes, where direction of edges represent certain causal effects. Key to high-dimensional structured classification and regression is how to utilize dependencies among predictors as specified by directions of the graph. In this dissertation, we develop a novel method that fully takes into account such dependencies formulated through certain nonlinear constraints. We apply the proposed method to two applications, feature selection in large margin binary classification and in linear regression. We implement the proposed method through difference convex programming for the cost function and constraints. Finally, theoretical and numerical analyses suggest that the proposed method achieves the desired objectives. An application to disease gene identification is presented.The second application scenario is personalized information filtering which extracts the information specifically relevant to a user, predicting his/her preference over a large number of items, based on the opinions of users who think alike or its content. This problem is cast into the framework of regression and classification, where we introduce novel partial latent models to integrate additional user-specific and content-specific predictors, for higher predictive accuracy. In particular, we factorize a user-over-item preference matrix into a product of two matrices, each representing a user's preference and an item preference by users. Then we propose a likelihood method to seek a sparsest latent factorization, from a class of over-complete factorizations, possibly with a high percentage of missing values. This promotes additional sparsity beyond rank reduction. Computationally, we design methods based on a ``decomposition and combination'' strategy, to break large-scale optimization into many small subproblems to solve in a recursive and parallel manner. On this basis, we implement the proposed methods through multi-platform shared-memory parallel programming, and through Mahout, a library for scalable machine learning and data mining, for mapReduce computation. For example, our methods are scalable to a dataset consisting of three billions of observations on a single machine with sufficient memory, having good timings. Both theoretical and numerical investigations show that the proposed methods exhibit significant improvement in accuracy over state-of-the-art scalable methods.

Subjects/Keywords: High dimensional data; Machine learning; Recommendation; Statistics

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Ye, C. (2014). Network selection, information filtering and scalable computation. (Doctoral Dissertation). University of Minnesota. Retrieved from http://hdl.handle.net/11299/172631

Chicago Manual of Style (16^th Edition):

Ye, Changqing. “Network selection, information filtering and scalable computation.” 2014. Doctoral Dissertation, University of Minnesota. Accessed February 24, 2020. http://hdl.handle.net/11299/172631.

MLA Handbook (7^th Edition):

Ye, Changqing. “Network selection, information filtering and scalable computation.” 2014. Web. 24 Feb 2020.

Vancouver:

Ye C. Network selection, information filtering and scalable computation. [Internet] [Doctoral dissertation]. University of Minnesota; 2014. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/11299/172631.

Council of Science Editors:

Ye C. Network selection, information filtering and scalable computation. [Doctoral Dissertation]. University of Minnesota; 2014. Available from: http://hdl.handle.net/11299/172631

University of Rochester

5. Pearson, Alexander T. Subset Selection for High-Dimensional Data, with Applications to Gene Array Data.

Degree: PhD, 2009, University of Rochester

URL: http://hdl.handle.net/1802/8411

► Identifying those genes that are differentially expressed in individuals with cancer could lead to new avenues of treatment or prevention. Gene array information can be… (more)

▼ Identifying those genes that are differentially expressed in individuals with cancer could lead to new avenues of treatment or prevention. Gene array information can be collected for an individual and paired with information about a specific outcome. We consider binary and censored time to event survival outcomes. Improved construction of prognostic gene signatures − risk scores based on groups of genes that can relatively accurately predict an individual’s outcome or risk thereof − would be a step forward in gene array research. We consider the problem of selecting which variables (genes) to include in a prognostic gene signature with high dimensional data in the logistic regression model and the Cox proportional hazards model frameworks. Constructing a prognostic gene signature from gene array data is very difficult because the number of candidate genes greatly outnumbers the sample size available in most studies. In this case, commonly used statistical techniques perform poorly at choosing genes that accurately predict the endpoint of interest. We propose a search algorithm that lies between forward selection and searching over all subsets. This method uses an evolving subgroup paradigm to intelligently select variables over a relatively large model space. We show that our method applied to the Cox proportional hazards model for censored survival time data or the logistic regression model for binary data can yield significant improvements in the number of predictive variables selected and the prediction error compared with the LASSO method, forward selection, and univariate selection, as demonstrated with a simulation study. We also investigate validation methods. Censored survival time data confounds many of the common validation methods because the outcome is not always observed, and splitting a data set can lead to a lack of uncensored observations in some partitions. We pursue validation with the goal of selecting the ideal number of variables to include. We propose a robust version of partial likelihood cross-validation and modified versions of the c-index as metrics to select model complexity. We propose analogous methods in the logistic model setting. We apply our new methods to publicly available lymphoma gene array data (Rosenwald et. al., 2002) with 7399 genes and 240 individuals. After splitting the full data into training and validation portions, running our method on the training data set leads to predictive results in the validation data set.

Subjects/Keywords: Subset Selection; Gene Array; High Dimensional Data

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Pearson, A. T. (2009). Subset Selection for High-Dimensional Data, with Applications to Gene Array Data. (Doctoral Dissertation). University of Rochester. Retrieved from http://hdl.handle.net/1802/8411

Chicago Manual of Style (16^th Edition):

Pearson, Alexander T. “Subset Selection for High-Dimensional Data, with Applications to Gene Array Data.” 2009. Doctoral Dissertation, University of Rochester. Accessed February 24, 2020. http://hdl.handle.net/1802/8411.

MLA Handbook (7^th Edition):

Pearson, Alexander T. “Subset Selection for High-Dimensional Data, with Applications to Gene Array Data.” 2009. Web. 24 Feb 2020.

Vancouver:

Pearson AT. Subset Selection for High-Dimensional Data, with Applications to Gene Array Data. [Internet] [Doctoral dissertation]. University of Rochester; 2009. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/1802/8411.

Council of Science Editors:

Pearson AT. Subset Selection for High-Dimensional Data, with Applications to Gene Array Data. [Doctoral Dissertation]. University of Rochester; 2009. Available from: http://hdl.handle.net/1802/8411

Massey University

6. Ullah, Insha. Contributions to high-dimensional data analysis : some applications of the regularized covariance matrices : a thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Statistics at Massey University, Albany, New Zealand .

Degree: 2015, Massey University

URL: http://hdl.handle.net/10179/6608

► High-dimensional data sets, particularly those where the number of variables exceeds the number of observations, are now common in many subject areas including genetics, ecology,… (more)

▼ High-dimensional data sets, particularly those where the number of variables exceeds the number of observations, are now common in many subject areas including genetics, ecology, and statistical pattern recognition to name but a few. The sample covariance matrix becomes rank deficient and is not invertible when the number of variables are more than the number of observations. This poses a serious problem for many classical multivariate techniques that rely on an inverse of a covariance matrix. Recently, regularized alternatives to the sample covariance have been proposed, which are not only guaranteed to be positive definite but also provide reliable estimates. In this Thesis, we bring together some of the important recent regularized estimators of the covariance matrix and explore their performance in high-dimensional scenarios via numerical simulations. We make use of these regularized estimators and attempt to improve the performance of the three classical multivariate techniques in high-dimensional settings. In a multivariate random effects models, estimating the between-group covariance is a well known problem. Its classical estimator involves the difference of two mean square matrices and often results in negative elements on the main diagonal. We use a lasso-regularized estimate of the between-group mean square and propose a new approach to estimate the between-group covariance based on the EM-algorithm. Using simulation, the procedure is shown to be quite effective and the estimate obtained is always positive definite. Multivariate analysis of variance (MANOVA) face serious challenges due to the undesirable properties of the sample covariance in high-dimensional problems. First, it suffer from low power and does not maintain accurate type-I error when the dimension is large as compared to the sample size. Second, MANOVA relies on the inverse of a covariance matrix and fails to work when the number of variables exceeds the number of observation. We use an approach based on the lasso regularization and present a comparative study of the existing approaches including our proposal. The lasso approach is shown to be an improvement in some cases, in terms of power of the test, over the existing high-dimensional methods. Another problem that is addressed in the Thesis is how to detect unusual future observations when the dimension is large. The Hotelling T2 control chart has traditionally been used for this purpose. The charting statistic in the control chart rely on the inverse of a covariance matrix and is not reliable in high-dimensional problems. To get a reliable estimate of the covariance matrix we use a distribution free shrinkage estimator. We make use of the available baseline set of data and propose a procedure to estimate the control limits for monitoring the individual future observations. The procedure do not assume multivariate normality and seems robust to the violation of multivariate normality. The simulation study shows that the new method performs better than…

Subjects/Keywords: Multivariate analysis; High-dimensional data; Covariance

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Ullah, I. (2015). Contributions to high-dimensional data analysis : some applications of the regularized covariance matrices : a thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Statistics at Massey University, Albany, New Zealand . (Thesis). Massey University. Retrieved from http://hdl.handle.net/10179/6608

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Ullah, Insha. “Contributions to high-dimensional data analysis : some applications of the regularized covariance matrices : a thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Statistics at Massey University, Albany, New Zealand .” 2015. Thesis, Massey University. Accessed February 24, 2020. http://hdl.handle.net/10179/6608.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Ullah, Insha. “Contributions to high-dimensional data analysis : some applications of the regularized covariance matrices : a thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Statistics at Massey University, Albany, New Zealand .” 2015. Web. 24 Feb 2020.

Vancouver:

Ullah I. Contributions to high-dimensional data analysis : some applications of the regularized covariance matrices : a thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Statistics at Massey University, Albany, New Zealand . [Internet] [Thesis]. Massey University; 2015. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/10179/6608.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Ullah I. Contributions to high-dimensional data analysis : some applications of the regularized covariance matrices : a thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Statistics at Massey University, Albany, New Zealand . [Thesis]. Massey University; 2015. Available from: http://hdl.handle.net/10179/6608

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

University of Michigan

7. Qian, Cheng. Some Advances on Modeling High-Dimensional Data with Complex Structures.

Degree: PhD, Statistics, 2017, University of Michigan

URL: http://hdl.handle.net/2027.42/140828

► Recent advances in technology have created an abundance of high-dimensional data and made its analysis possible. These data require new, computationally efficient methodology and new… (more)

▼ Recent advances in technology have created an abundance of high-dimensional data and made its analysis possible. These data require new, computationally efficient methodology and new kind of asymptotic analysis. This thesis consists of four projects that deal with high-dimensional data with complex structures. The first project focuses on the graph estimation problem for Gaussian graphical models. Graphical models are commonly used in representing conditional independence between random variables, and learning the conditional independence structure from data has attracted much attention in recent years. However, almost all commonly used graph learning methods rely on the assumption that the observations share the same mean vector. In the first project, we extend the Gaussian graphical model to the setting where the observations are connected by a network and the mean vector can be different for different observations. We propose an efficient estimation method for the model, and under the assumption of network cohesion, we show that our method can accurately estimate the inverse covariance matrix as well as the corresponding graph structure, both from the theoretical perspective and using numerical studies. To further demonstrate the effectiveness of the proposed method, we also analyze a statisticians' coauthorship network data to learn the term dependency based on statistics publications. The second project addresses the directed acyclic graph (DAG) estimation problem. Estimation of the DAG structure is often a challenging problem as the computational complexity scales exponentially in the graph size when the total ordering of the DAG is unknown. To reduce the computational cost, and also with the aim of improving the estimation accuracy via the bias-variance trade-off, we propose a two-step approach for estimating the DAG, when data are generated from a linear structural equation model. In the first step, we infer the moral graph of the DAG via estimation of the inverse covariance matrix, which reduces the parameter space that one would search for the DAG. In the second step, we apply a penalized likelihood method for estimating the DAG restricted in the reduced space. Numerical studies indicate that the proposed method compares favorably with the one-step method in terms of both computational cost and estimation accuracy. The third and fourth projects investigate supervised learning problems. Specifically, in the third project, we study the cointegration problem for multivariate time series data and propose a method for identifying cointegrating vectors with simultaneously group and elementwise sparse structures. Such a sparsity structure enables the elimination of certain coordinates of the original multivariate series from all cointegrated series, leading to parsimonious and potentially more interpretable cointegrating vectors. Specifically, we formulate an optimization problem based on the profile likelihood and propose an iterative algorithm for solving the optimization problem. The proposed… Advisors/Committee Members: Zhu, Ji (committee member), Jin, Judy (committee member), Levina, Elizaveta (committee member), Shedden, Kerby A (committee member).

Subjects/Keywords: High-Dimensional; Statistics and Numeric Data; Science

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Qian, C. (2017). Some Advances on Modeling High-Dimensional Data with Complex Structures. (Doctoral Dissertation). University of Michigan. Retrieved from http://hdl.handle.net/2027.42/140828

Chicago Manual of Style (16^th Edition):

Qian, Cheng. “Some Advances on Modeling High-Dimensional Data with Complex Structures.” 2017. Doctoral Dissertation, University of Michigan. Accessed February 24, 2020. http://hdl.handle.net/2027.42/140828.

MLA Handbook (7^th Edition):

Qian, Cheng. “Some Advances on Modeling High-Dimensional Data with Complex Structures.” 2017. Web. 24 Feb 2020.

Vancouver:

Qian C. Some Advances on Modeling High-Dimensional Data with Complex Structures. [Internet] [Doctoral dissertation]. University of Michigan; 2017. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/2027.42/140828.

Council of Science Editors:

Qian C. Some Advances on Modeling High-Dimensional Data with Complex Structures. [Doctoral Dissertation]. University of Michigan; 2017. Available from: http://hdl.handle.net/2027.42/140828

Virginia Tech

8. Blake, Patrick Michael. Biclustering and Visualization of High Dimensional Data using VIsual Statistical Data Analyzer.

Degree: MS, Electrical Engineering, 2019, Virginia Tech

URL: http://hdl.handle.net/10919/87392

► Many data sets have too many features for conventional pattern recognition techniques to work properly. This thesis investigates techniques that alleviate these difficulties. One such… (more)

Subjects/Keywords: high-dimensional data; biclustering; VISDA; VISDApy

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Blake, P. M. (2019). Biclustering and Visualization of High Dimensional Data using VIsual Statistical Data Analyzer. (Masters Thesis). Virginia Tech. Retrieved from http://hdl.handle.net/10919/87392

Chicago Manual of Style (16^th Edition):

Blake, Patrick Michael. “Biclustering and Visualization of High Dimensional Data using VIsual Statistical Data Analyzer.” 2019. Masters Thesis, Virginia Tech. Accessed February 24, 2020. http://hdl.handle.net/10919/87392.

MLA Handbook (7^th Edition):

Blake, Patrick Michael. “Biclustering and Visualization of High Dimensional Data using VIsual Statistical Data Analyzer.” 2019. Web. 24 Feb 2020.

Vancouver:

Blake PM. Biclustering and Visualization of High Dimensional Data using VIsual Statistical Data Analyzer. [Internet] [Masters thesis]. Virginia Tech; 2019. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/10919/87392.

Council of Science Editors:

Blake PM. Biclustering and Visualization of High Dimensional Data using VIsual Statistical Data Analyzer. [Masters Thesis]. Virginia Tech; 2019. Available from: http://hdl.handle.net/10919/87392

University of Minnesota

9. Datta, Abhirup. Statistical Methods for Large Complex Datasets.

Degree: PhD, Biostatistics, 2016, University of Minnesota

URL: http://hdl.handle.net/11299/199089

► Modern technological advancements have enabled massive-scale collection, processing and storage of information triggering the onset of the `big data' era where in every two days… (more)

Subjects/Keywords: Big data; High dimensional data; Large spatial data

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Datta, A. (2016). Statistical Methods for Large Complex Datasets. (Doctoral Dissertation). University of Minnesota. Retrieved from http://hdl.handle.net/11299/199089

Chicago Manual of Style (16^th Edition):

Datta, Abhirup. “Statistical Methods for Large Complex Datasets.” 2016. Doctoral Dissertation, University of Minnesota. Accessed February 24, 2020. http://hdl.handle.net/11299/199089.

MLA Handbook (7^th Edition):

Datta, Abhirup. “Statistical Methods for Large Complex Datasets.” 2016. Web. 24 Feb 2020.

Vancouver:

Datta A. Statistical Methods for Large Complex Datasets. [Internet] [Doctoral dissertation]. University of Minnesota; 2016. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/11299/199089.

Council of Science Editors:

Datta A. Statistical Methods for Large Complex Datasets. [Doctoral Dissertation]. University of Minnesota; 2016. Available from: http://hdl.handle.net/11299/199089

University of Arizona

10. Washburn, Ammon. High-Confidence Learning from Uncertain Data with High Dimensionality .

Degree: 2018, University of Arizona

URL: http://hdl.handle.net/10150/631476

► Some of the most challenging issues in big data are size, scalability and reliability. Big data, such as pictures, videos, and text, have innate structure… (more)

▼ Some of the most challenging issues in big data are size, scalability and reliability. Big data, such as pictures, videos, and text, have innate structure that does not fit into the structure of the normal data table. Often the sources come from the internet or other domains where accuracy is not possible. When drawn from these sources, it is likely that important information is missing or cannot be measured. This leads to situations where identifying the important part of the data would lead to good solutions, but with all the data the tasks become ill-posed. Another case is where all the data is useful but there is some important and/or hidden structure of which classical methods are not equipped to take advantage. However, many methods have been developed to either deal with data uncertainty or with ill-posed problems. Data uncertainty can come from missing or distributional data. Data imputation combined with uncertainty quantification can allow regular statistical and machine learning methods to be applied and then verified. Other methods combine the steps in a robust way to directly inform the model. This last type of method is common in chance-constrained, robust or distributionally robust programs from the mathematical optimization community. Well-posed problems have a solution which is unique and changes slowly and continuously with the initial conditions. For standard machine learning models, a data set with many irrelevant features gives rise to ill-posed problems. Regularization and feature selection are two possible ways to deal with these problems. Both the regularization and feature selection techniques have been around for a long time. Regularization approaches can include Lp norms or the matrix trace which will give certain properties. Feature selection has been achieved in many ways including a preprocessing step to rank and select features and the use of stepwise regression to classical modern techniques such as LASSO. For many applications, there are both uncertainties and a high-dimensional component of the data. By combining methods that deal with both of these methods and then deriving quick computational algorithms, we can formulate robust, highly-generalizable machine learning models that achieve very good results. Two of our classification models handle samples of points to be classified as one. Traditional machine learning models in classification expect to classify one point but with an interesting data set from Karyometry, several hundred points must be consolidated into one classification. One of the algorithms also can take advantage of a certain nested structure in this data set to gain further information useful for doctors. The third model deals with data and label uncertainty in classification. We do it in a data-driven, distributionally robust way that gives us some confidence intervals on our classification. A large part of this dissertation also deals with the algorithms used to solve these optimization formulations. We advance the solution path algorithms to… Advisors/Committee Members: Fan, Neng (advisor), Zhang, Helen Hao (advisor), Kennedy, Thomas G. (committeemember), Missoum, Samy (committeemember).

Subjects/Keywords: data classification; data uncertainty; high dimensional data; machine learning; optimization

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Washburn, A. (2018). High-Confidence Learning from Uncertain Data with High Dimensionality . (Doctoral Dissertation). University of Arizona. Retrieved from http://hdl.handle.net/10150/631476

Chicago Manual of Style (16^th Edition):

Washburn, Ammon. “High-Confidence Learning from Uncertain Data with High Dimensionality .” 2018. Doctoral Dissertation, University of Arizona. Accessed February 24, 2020. http://hdl.handle.net/10150/631476.

MLA Handbook (7^th Edition):

Washburn, Ammon. “High-Confidence Learning from Uncertain Data with High Dimensionality .” 2018. Web. 24 Feb 2020.

Vancouver:

Washburn A. High-Confidence Learning from Uncertain Data with High Dimensionality . [Internet] [Doctoral dissertation]. University of Arizona; 2018. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/10150/631476.

Council of Science Editors:

Washburn A. High-Confidence Learning from Uncertain Data with High Dimensionality . [Doctoral Dissertation]. University of Arizona; 2018. Available from: http://hdl.handle.net/10150/631476

University of Minnesota

11. O'Connell, Michael. Integrative Analyses for Multi-source Data with Multiple Shared Dimensions.

Degree: PhD, Biostatistics, 2018, University of Minnesota

URL: http://hdl.handle.net/11299/200286

► High dimensional data consists of matrices with a large number of features and is common across many fields of study, including genetics, imaging, and toxicology.… (more)

Subjects/Keywords: data integration; high-dimensional data; matrix decomposition; multi-source

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

O'Connell, M. (2018). Integrative Analyses for Multi-source Data with Multiple Shared Dimensions. (Doctoral Dissertation). University of Minnesota. Retrieved from http://hdl.handle.net/11299/200286

Chicago Manual of Style (16^th Edition):

O'Connell, Michael. “Integrative Analyses for Multi-source Data with Multiple Shared Dimensions.” 2018. Doctoral Dissertation, University of Minnesota. Accessed February 24, 2020. http://hdl.handle.net/11299/200286.

MLA Handbook (7^th Edition):

O'Connell, Michael. “Integrative Analyses for Multi-source Data with Multiple Shared Dimensions.” 2018. Web. 24 Feb 2020.

Vancouver:

O'Connell M. Integrative Analyses for Multi-source Data with Multiple Shared Dimensions. [Internet] [Doctoral dissertation]. University of Minnesota; 2018. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/11299/200286.

Council of Science Editors:

O'Connell M. Integrative Analyses for Multi-source Data with Multiple Shared Dimensions. [Doctoral Dissertation]. University of Minnesota; 2018. Available from: http://hdl.handle.net/11299/200286

University of Adelaide

12. Conway, Annie. Clustering of proteomics imaging mass spectrometry data.

Degree: 2016, University of Adelaide

URL: http://hdl.handle.net/2440/112036

► This thesis presents a toolbox for the exploratory analysis of multivariate data, in particular proteomics imaging mass spectrometry data. Typically such data consist of 15000… (more)

Subjects/Keywords: clustering; proteomics; multivariate data analysis; high-dimensional data analysis; machine learning

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Conway, A. (2016). Clustering of proteomics imaging mass spectrometry data. (Thesis). University of Adelaide. Retrieved from http://hdl.handle.net/2440/112036

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Conway, Annie. “Clustering of proteomics imaging mass spectrometry data.” 2016. Thesis, University of Adelaide. Accessed February 24, 2020. http://hdl.handle.net/2440/112036.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Conway, Annie. “Clustering of proteomics imaging mass spectrometry data.” 2016. Web. 24 Feb 2020.

Vancouver:

Conway A. Clustering of proteomics imaging mass spectrometry data. [Internet] [Thesis]. University of Adelaide; 2016. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/2440/112036.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Conway A. Clustering of proteomics imaging mass spectrometry data. [Thesis]. University of Adelaide; 2016. Available from: http://hdl.handle.net/2440/112036

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

13. Waddell, Adrian. Interactive Visualization and Exploration of High-Dimensional Data.

Degree: 2016, University of Waterloo

URL: http://hdl.handle.net/10012/10188

► Visualizing data is an essential part of good statistical practice. Plots are useful for revealing structure in the data, checking model assumptions, detecting outliers and… (more)

▼ Visualizing data is an essential part of good statistical practice. Plots are useful for revealing structure in the data, checking model assumptions, detecting outliers and finding unanticipated patterns. Post-analysis visualization is commonly used to communicate the results of statistical analyses. The availability of good statistical visualization software is key in effectively performing data analysis and in exploring and developing new methods for data visualization. Compared to static visualization, interactive visualization adds natural and powerful ways to explore the data. With interactive visualization an analyst can dive into the data and quickly react to visual clues by, for example, re-focusing and creating interactive queries of the data. Further, linking visual attributes of the data points such as color and size allows the analyst to compare different visual representations of the data such as histograms and scatterplots. In this thesis, we explore and develop new interactive data visualization and exploration tools for high-dimensional data. The original focus of our research was a software implementation of navigation graphs. Navigation graphs are navigational infrastructures for controlled exploration of high-dimensional data. As part of this thesis, we developed the first interactive implementation of these navigation graphs called RnavGraph. With RnavGraph we explored various features for enhancing the usability of navigation graphs. We concluded that a powerful interactive scatterplot display and methods to deal with large graphs were two areas that would add great value to the navigation graph framework. RnavGraph's scatterplot display proved to be particularly useful for data analysis and we continued our research with the design and implementation of a general-purpose interactive visualization toolkit called loon. The core contributions of loon are as follows. loon implements a general design for interactive statistical graphic displays that supports layering of visual information such as point objects, lines and polygons. These displays further support zooming, panning and selection, and modification and deactivation of plot elements and layers. Interactions with plots are provided with mouse and keyboard gestures as well as via command line control and with inspectors. These inspectors provide graphical user interfaces for modifying and overseeing the plots. loon also implements a novel dynamic linking mechanism that can be used to assign the plots that are to be linked and the linking rules at run time. Additionally, loon's design provides several different types of event bindings to add and customize functionality of loon's displays. In this thesis, we discuss loon's design and framework by giving concrete examples that show how these design choices can be used to efficiently explore and visualize data interactively. These examples revolve around loon's statistical interactive displays such as histograms, scatterplots and graph displays. We also illustrate how loon's design can be…

Subjects/Keywords: Interactive Data Visualization; High-dimensional Data; Statistical Visualization

11 more images…

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Waddell, A. (2016). Interactive Visualization and Exploration of High-Dimensional Data. (Thesis). University of Waterloo. Retrieved from http://hdl.handle.net/10012/10188

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Waddell, Adrian. “Interactive Visualization and Exploration of High-Dimensional Data.” 2016. Thesis, University of Waterloo. Accessed February 24, 2020. http://hdl.handle.net/10012/10188.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Waddell, Adrian. “Interactive Visualization and Exploration of High-Dimensional Data.” 2016. Web. 24 Feb 2020.

Vancouver:

Waddell A. Interactive Visualization and Exploration of High-Dimensional Data. [Internet] [Thesis]. University of Waterloo; 2016. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/10012/10188.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Waddell A. Interactive Visualization and Exploration of High-Dimensional Data. [Thesis]. University of Waterloo; 2016. Available from: http://hdl.handle.net/10012/10188

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Tulane University

14. Qu, Zhe. High-dimensional statistical data integration.

Degree: 2019, Tulane University

URL: https://digitallibrary.tulane.edu/islandora/object/tulane:106916

►

[email protected]

Modern biomedical studies often collect multiple types of high-dimensional data on a common set of objects. A representative model for the integrative analysis of… (more)

Subjects/Keywords: High-dimensional data analysis; Data integration; Canonical correlation analysis

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Qu, Z. (2019). High-dimensional statistical data integration. (Thesis). Tulane University. Retrieved from https://digitallibrary.tulane.edu/islandora/object/tulane:106916

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Qu, Zhe. “High-dimensional statistical data integration.” 2019. Thesis, Tulane University. Accessed February 24, 2020. https://digitallibrary.tulane.edu/islandora/object/tulane:106916.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Qu, Zhe. “High-dimensional statistical data integration.” 2019. Web. 24 Feb 2020.

Vancouver:

Qu Z. High-dimensional statistical data integration. [Internet] [Thesis]. Tulane University; 2019. [cited 2020 Feb 24]. Available from: https://digitallibrary.tulane.edu/islandora/object/tulane:106916.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Qu Z. High-dimensional statistical data integration. [Thesis]. Tulane University; 2019. Available from: https://digitallibrary.tulane.edu/islandora/object/tulane:106916

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

University of California – Riverside

15. Zakaria, Jesin. Developing Efficient Algorithms for Data Mining Large Scale High Dimensional Data.

Degree: Computer Science, 2013, University of California – Riverside

URL: http://www.escholarship.org/uc/item/660316zp

► Data mining and knowledge discovery has attracted a great deal of attention in information technology in recent years. The rapid progress of computer hardware technology… (more)

▼ Data mining and knowledge discovery has attracted a great deal of attention in information technology in recent years. The rapid progress of computer hardware technology in the past three decades provides a great enhancement to the database and information industry. The size and complexity of real world data is dramatically increasing with the growth of hardware technology. Although new efficient algorithms to deal with such data are constantly being proposed, the mining of large scale high dimensional data still presents a lot of challenges. In this dissertation, several novel algorithms are proposed to handle such challenges. These algorithms are applied to domains as diverse as electrocardiography (ECG), stock market data, geospatial data, power supply data, audio data, image data, etc. This dissertation contributes to the data mining community in the following three ways:Firstly, we propose a novel algorithm for clustering time series data efficiently in the presence of noise or extraneous data. Most existing methods for time series clustering rely on distances calculated from the entire raw data. As a consequence, most work on time series clustering only considers the clustering of individual time series "behaviors," e.g., individual heart beats and contrives the time series in some way to make them all equal in length. However, for any real world problem, formatting the data in such a way is often a harder task than the clustering itself. In order to remove these unrealistic assumptions, we have developed a new primitive called unsupervised shapelet or u-shapelet and shown its utility for clustering time series.Secondly, in order to speed up the discovery of u-shapelet and make it scalable we have proposed two optimization techniques which can speed up the unsupervised shapelet discovery independently of each other. Moreover, if we combine the two optimization procedures, it results in a super linear speedup. In addition to the above, we can also cast our u-shapelet discovery algorithm as an anytime algorithm. In my final contribution, we have developed a novel and robust algorithm for mining mice vocalizations with symbolized representation. Our algorithm processes large scale, high dimensional, noisy mice vocalization by dimensionality reduction and cardinality reduction and make it suitable for knowledge discovery like classification, clustering, similarity search, motif discovery, contrast set mining etc.

Subjects/Keywords: Computer science; Clustering; Data Mining; High Dimensional Data; Scalable; Time Series

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Zakaria, J. (2013). Developing Efficient Algorithms for Data Mining Large Scale High Dimensional Data. (Thesis). University of California – Riverside. Retrieved from http://www.escholarship.org/uc/item/660316zp

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Zakaria, Jesin. “Developing Efficient Algorithms for Data Mining Large Scale High Dimensional Data.” 2013. Thesis, University of California – Riverside. Accessed February 24, 2020. http://www.escholarship.org/uc/item/660316zp.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Zakaria, Jesin. “Developing Efficient Algorithms for Data Mining Large Scale High Dimensional Data.” 2013. Web. 24 Feb 2020.

Vancouver:

Zakaria J. Developing Efficient Algorithms for Data Mining Large Scale High Dimensional Data. [Internet] [Thesis]. University of California – Riverside; 2013. [cited 2020 Feb 24]. Available from: http://www.escholarship.org/uc/item/660316zp.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Zakaria J. Developing Efficient Algorithms for Data Mining Large Scale High Dimensional Data. [Thesis]. University of California – Riverside; 2013. Available from: http://www.escholarship.org/uc/item/660316zp

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

University of Southern California

16. Ren, Jie. Robust feature selection with penalized regression in imbalanced high dimensional data.

Degree: PhD, Statistical Genetics and Genetic Epidemiology, 2014, University of Southern California

URL: http://digitallibrary.usc.edu/cdm/compoundobject/collection/p15799coll3/id/443080/rec/5613

► This work is motivated by an ongoing USC/Illumina study of prostate cancer recurrence after radical prostatectomy. The study generated gene expression data for nearly thirty… (more)

▼ This work is motivated by an ongoing USC/Illumina study of prostate cancer recurrence after radical prostatectomy. The study generated gene expression data for nearly thirty thousand probes from 187 tumor samples, of which 33 came from patients with recurrent prostate cancer and 154 came from patients with non‐recurrent prostate cancer after years of follow‐up. Our goal was to use penalized logistic regression and stability selection to find a “gene signature” of recurrence that can improve upon PSA and Gleason‐score, which are well‐established but poor predictors. For interpretability and future clinical use, the gene signature should ideally involve a small proportion of probes to predict recurrence in new patients. Due to the skewness in the data, the model selected by tuning the LASSO penalty parameter based on the average misclassification rate in cross validation did not have a balanced performance, i.e. it predicted non‐recurrent cancer with high accuracy but predicted recurrent cancer with very low accuracy. In addition, standard penalized regression with cross validation appeared to select many noise features. In my simulation study in Chapter 2, I evaluated the performance of models selected by different metrics in imbalanced data. I concluded that G‐mean‐thr (G‐mean with an alternative cutoff) and area under the ROC curve (AUC‐ROC) were the most robust metrics to class imbalance. In Chapter 3, I examined the performance of stability selection (SS‐thr) in simulation studies and found that its feature selection capability (a) depended on the stability cutoff chosen and (b) is conservative as a result of a stringent error control. To address these problems, I proposed new feature selectors based on stability selection, including SS‐test, an essentially parameter‐free test‐based outlier‐detection approach, and SS‐rank and SS‐ranktest, parameter‐free rank‐based methods. I demonstrated their advantage over SS‐thr, ULR, and LASSO with cross validation in extensive simulation studies and also found that all these stability selection based methods were robust to class imbalance. These newly developed methods and procedures was applied to the prostate cancer recurrence data. I used a variety of metrics to do model selection within the penalized logistic regression framework using imbalanced prostate cancer recurrence data and demonstrated that G‐mean with the case‐proportion cutoff selected the model with the most balanced prediction accuracy in cross validation. In addition, I also showed the importance of using an appropriate cutoff to evaluate models when models were built from skewed data. I also applied stability selection based methods including SS‐thr, SS‐test, SS‐rank and SS‐ranktest to select important genes from the same data. Three genes, ABCC1, NKX2‐1 and ZYG11A were identified by all of the methods and also appeared to stand out from other features in the stability path plot. I fit a logistic regression model using these genes and clinical features, which has significantly higher prediction accuracy… Advisors/Committee Members: Lewinger, Juan PabloWatanabe, Richard M. (Committee Chair), Siegmund, Kimberly D. (Committee Member), Stern, Mariana C. (Committee Member), Lv, Jinchi (Committee Member).

Subjects/Keywords: feature selection; penalized regression; imbalanced data; high dimensional data; stability selection

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Ren, J. (2014). Robust feature selection with penalized regression in imbalanced high dimensional data. (Doctoral Dissertation). University of Southern California. Retrieved from http://digitallibrary.usc.edu/cdm/compoundobject/collection/p15799coll3/id/443080/rec/5613

Chicago Manual of Style (16^th Edition):

Ren, Jie. “Robust feature selection with penalized regression in imbalanced high dimensional data.” 2014. Doctoral Dissertation, University of Southern California. Accessed February 24, 2020. http://digitallibrary.usc.edu/cdm/compoundobject/collection/p15799coll3/id/443080/rec/5613.

MLA Handbook (7^th Edition):

Ren, Jie. “Robust feature selection with penalized regression in imbalanced high dimensional data.” 2014. Web. 24 Feb 2020.

Vancouver:

Ren J. Robust feature selection with penalized regression in imbalanced high dimensional data. [Internet] [Doctoral dissertation]. University of Southern California; 2014. [cited 2020 Feb 24]. Available from: http://digitallibrary.usc.edu/cdm/compoundobject/collection/p15799coll3/id/443080/rec/5613.

Council of Science Editors:

Ren J. Robust feature selection with penalized regression in imbalanced high dimensional data. [Doctoral Dissertation]. University of Southern California; 2014. Available from: http://digitallibrary.usc.edu/cdm/compoundobject/collection/p15799coll3/id/443080/rec/5613

Temple University

17. Lou, Qiang. LEARNING FROM INCOMPLETE HIGH-DIMENSIONAL DATA.

Degree: PhD, 2013, Temple University

URL: http://digital.library.temple.edu/u?/p245801coll10,214785

►

Computer and Information Science

Data sets with irrelevant and redundant features and large fraction of missing values are common in the real life application. Learning… (more)

▼

Computer and Information Science

Data sets with irrelevant and redundant features and large fraction of missing values are common in the real life application. Learning such data usually requires some preprocess such as selecting informative features and imputing missing values based on observed data. These processes can provide more accurate and more efficient prediction as well as better understanding of the data distribution. In my dissertation I will describe my work in both of these aspects and also my following up work on feature selection in incomplete dataset without imputing missing values. In the last part of my dissertation, I will present my current work on more challenging situation where high-dimensional data is time-involving. The first two parts of my dissertation consist of my methods that focus on handling such data in a straightforward way: imputing missing values first, and then applying traditional feature selection method to select informative features. We proposed two novel methods, one for imputing missing values and the other one for selecting informative features. We proposed a new method that imputes the missing attributes by exploiting temporal correlation of attributes, correlations among multiple attributes collected at the same time and space, and spatial correlations among attributes from multiple sources. The proposed feature selection method aims to find a minimum subset of the most informative variables for classification/regression by efficiently approximating the Markov Blanket which is a set of variables that can shield a certain variable from the target. I present, in the third part, how to perform feature selection in incomplete high-dimensional data without imputation, since imputation methods only work well when data is missing completely at random, when fraction of missing values is small, or when there is prior knowledge about the data distribution. We define the objective function of the uncertainty margin-based feature selection method to maximize each instance's uncertainty margin in its own relevant subspace. In optimization, we take into account the uncertainty of each instance due to the missing values. The experimental results on synthetic and 6 benchmark data sets with few missing values (less than 25%) provide evidence that our method can select the same accurate features as the alternative methods which apply an imputation method first. However, when there is a large fraction of missing values (more than 25%) in data, our feature selection method outperforms the alternatives, which impute missing values first. In the fourth part, I introduce my method handling more challenging situation where the high-dimensional data varies in time. Existing way to handle such data is to flatten temporal data into single static data matrix, and then applying traditional feature selection method. In order to keep the dynamics in the time series data, our method avoid flattening the data in advance. We propose a way to measure the distance between multivariate temporal data from…

Advisors/Committee Members: Obradovic, Zoran, Vucetic, Slobodan, Latecki, Longin, Davey, Adam.

Subjects/Keywords: Computer science; data mining; feature selection; high dimensional data; incomplete data; machine learning

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Lou, Q. (2013). LEARNING FROM INCOMPLETE HIGH-DIMENSIONAL DATA. (Doctoral Dissertation). Temple University. Retrieved from http://digital.library.temple.edu/u?/p245801coll10,214785

Chicago Manual of Style (16^th Edition):

Lou, Qiang. “LEARNING FROM INCOMPLETE HIGH-DIMENSIONAL DATA.” 2013. Doctoral Dissertation, Temple University. Accessed February 24, 2020. http://digital.library.temple.edu/u?/p245801coll10,214785.

MLA Handbook (7^th Edition):

Lou, Qiang. “LEARNING FROM INCOMPLETE HIGH-DIMENSIONAL DATA.” 2013. Web. 24 Feb 2020.

Vancouver:

Lou Q. LEARNING FROM INCOMPLETE HIGH-DIMENSIONAL DATA. [Internet] [Doctoral dissertation]. Temple University; 2013. [cited 2020 Feb 24]. Available from: http://digital.library.temple.edu/u?/p245801coll10,214785.

Council of Science Editors:

Lou Q. LEARNING FROM INCOMPLETE HIGH-DIMENSIONAL DATA. [Doctoral Dissertation]. Temple University; 2013. Available from: http://digital.library.temple.edu/u?/p245801coll10,214785

18. Shou, Haochang. Statistical Methods for Structured Multilevel Functional Data: Estimation and Reliability.

Degree: 2014, Johns Hopkins University

URL: http://jhir.library.jhu.edu/handle/1774.2/37867

► The thesis investigates a specific type of functional data with multilevel structures induced by complex experimental designs. Novel statistical methods based on principal component analysis… (more)

Subjects/Keywords: functional data analysis; multilevel and structured data; high-dimensional data; imaging reproducibility; shrinkage estimation

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Shou, H. (2014). Statistical Methods for Structured Multilevel Functional Data: Estimation and Reliability. (Thesis). Johns Hopkins University. Retrieved from http://jhir.library.jhu.edu/handle/1774.2/37867

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Shou, Haochang. “Statistical Methods for Structured Multilevel Functional Data: Estimation and Reliability.” 2014. Thesis, Johns Hopkins University. Accessed February 24, 2020. http://jhir.library.jhu.edu/handle/1774.2/37867.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Shou, Haochang. “Statistical Methods for Structured Multilevel Functional Data: Estimation and Reliability.” 2014. Web. 24 Feb 2020.

Vancouver:

Shou H. Statistical Methods for Structured Multilevel Functional Data: Estimation and Reliability. [Internet] [Thesis]. Johns Hopkins University; 2014. [cited 2020 Feb 24]. Available from: http://jhir.library.jhu.edu/handle/1774.2/37867.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Shou H. Statistical Methods for Structured Multilevel Functional Data: Estimation and Reliability. [Thesis]. Johns Hopkins University; 2014. Available from: http://jhir.library.jhu.edu/handle/1774.2/37867

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Texas A&M University

19. Song, Qifan. Variable Selection for Ultra High Dimensional Data.

Degree: 2014, Texas A&M University

URL: http://hdl.handle.net/1969.1/153224

► Variable selection plays an important role for the high dimensional data analysis. In this work, we first propose a Bayesian variable selection approach for ultra-high… (more)

▼ Variable selection plays an important role for the high dimensional data analysis. In this work, we first propose a Bayesian variable selection approach for ultra-high dimensional linear regression based on the strategy of split-and-merge. The proposed approach consists of two stages: (i) split the ultra-high dimensional data set into a number of lower dimensional subsets and select relevant variables from each of the subsets, and (ii) aggregate the variables selected from each subset and then select relevant variables from the aggregated data set. Since the proposed approach has an embarrassingly parallel structure, it can be easily implemented in a parallel architecture and applied to big data problems with millions or more of explanatory variables. Under mild conditions, we show that the proposed approach is consistent. That is, asymptotically, the true explanatory variables will be correctly identified by the proposed approach as the sample size becomes large. Extensive comparisons of the proposed approach have been made with the penalized likelihood approaches, such as Lasso, elastic net, SIS and ISIS. The numerical results show that the proposed approach generally outperforms the penalized likelihood approaches. The models selected by the proposed approach tend to be more sparse and closer to the true model. In the frequentist realm, penalized likelihood methods have been widely used in variable selection problems, where the penalty functions are typically symmetric about 0, continuous and nondecreasing in (0,?). The second contribution of this work is that, we propose a new penalized likelihood method, reciprocal Lasso (or in short, rLasso), based on a new class of penalty functions which are decreasing in (0,?), discontinuous at 0, and converge to infinity when the coefficients approach zero. The new penalty functions give nearly zero coefficients infinity penalties; in contrast, the conventional penalty functions give nearly zero coefficients nearly zero penalties (e.g., Lasso and SCAD) or constant penalties (e.g., L0 penalty). This distinguishing feature makes rLasso very attractive for variable selection: It can effectively avoid selecting overly dense models. We establish the consistency of the rLasso for variable selection and coefficient estimation under both the low and high dimensional settings. Since the rLasso penalty functions induce an objective function with multiple local minima, we also propose an efficient Monte Carlo optimization algorithm to solve the minimization problem. Our simulation results show that the rLasso outperforms other popular penalized likelihood methods, such as Lasso, SCAD, MCP, SIS, ISIS and EBIC. It can produce sparser and more accurate coefficient estimates, and have a higher probability to catch true models. Advisors/Committee Members: Liang, Faming (advisor), Carroll, Raymond (committee member), Johnson, Valen (committee member), Lahiri, Soumendra (committee member), Zhou, Jianxin (committee member).

Subjects/Keywords: High Dimensional Variable Selection; Big Data; Penalized Likelihood Approach; Posterior Consistency

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Song, Q. (2014). Variable Selection for Ultra High Dimensional Data. (Thesis). Texas A&M University. Retrieved from http://hdl.handle.net/1969.1/153224

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Song, Qifan. “Variable Selection for Ultra High Dimensional Data.” 2014. Thesis, Texas A&M University. Accessed February 24, 2020. http://hdl.handle.net/1969.1/153224.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Song, Qifan. “Variable Selection for Ultra High Dimensional Data.” 2014. Web. 24 Feb 2020.

Vancouver:

Song Q. Variable Selection for Ultra High Dimensional Data. [Internet] [Thesis]. Texas A&M University; 2014. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/1969.1/153224.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Song Q. Variable Selection for Ultra High Dimensional Data. [Thesis]. Texas A&M University; 2014. Available from: http://hdl.handle.net/1969.1/153224

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

University of Minnesota

20. Peng, Bo. Methodologies and Algorithms on Some Non-convex Penalized Models for Ultra High Dimensional Data.

Degree: PhD, Statistics, 2016, University of Minnesota

URL: http://hdl.handle.net/11299/182177

► In recent years, penalized models have gained considerable importance on deal- ing with variable selection and estimation problems under high dimensional settings. Of all the… (more)

Subjects/Keywords: High dimensional data; Non-convex penalty; Oracle property; Quantile regression; SVM

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Peng, B. (2016). Methodologies and Algorithms on Some Non-convex Penalized Models for Ultra High Dimensional Data. (Doctoral Dissertation). University of Minnesota. Retrieved from http://hdl.handle.net/11299/182177

Chicago Manual of Style (16^th Edition):

Peng, Bo. “Methodologies and Algorithms on Some Non-convex Penalized Models for Ultra High Dimensional Data.” 2016. Doctoral Dissertation, University of Minnesota. Accessed February 24, 2020. http://hdl.handle.net/11299/182177.

MLA Handbook (7^th Edition):

Peng, Bo. “Methodologies and Algorithms on Some Non-convex Penalized Models for Ultra High Dimensional Data.” 2016. Web. 24 Feb 2020.

Vancouver:

Peng B. Methodologies and Algorithms on Some Non-convex Penalized Models for Ultra High Dimensional Data. [Internet] [Doctoral dissertation]. University of Minnesota; 2016. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/11299/182177.

Council of Science Editors:

Peng B. Methodologies and Algorithms on Some Non-convex Penalized Models for Ultra High Dimensional Data. [Doctoral Dissertation]. University of Minnesota; 2016. Available from: http://hdl.handle.net/11299/182177

Harvard University

21. Minnier, Jessica. Inference and Prediction for High Dimensional Data via Penalized Regression and Kernel Machine Methods.

Degree: PhD, Biostatistics, 2012, Harvard University

URL: http://nrs.harvard.edu/urn-3:HUL.InstRepos:9367010

► Analysis of high dimensional data often seeks to identify a subset of important features and assess their effects on the outcome. Furthermore, the ultimate goal… (more)

▼ Analysis of high dimensional data often seeks to identify a subset of important features and assess their effects on the outcome. Furthermore, the ultimate goal is often to build a prediction model with these features that accurately assesses risk for future subjects. Such statistical challenges arise in the study of genetic associations with health outcomes. However, accurate inference and prediction with genetic information remains challenging, in part due to the complexity in the genetic architecture of human health and disease. A valuable approach for improving prediction models with a large number of potential predictors is to build a parsimonious model that includes only important variables. Regularized regression methods are useful, though often pose challenges for inference due to nonstandard limiting distributions or finite sample distributions that are difficult to approximate. In Chapter 1 we propose and theoretically justify a perturbation-resampling method to derive confidence regions and covariance estimates for marker effects estimated from regularized procedures with a general class of objective functions and concave penalties. Our methods outperform their asymptotic-based counterparts, even when effects are estimated as zero. In Chapters 2 and 3 we focus on genetic risk prediction. The difficulty in accurate risk assessment with genetic studies can in part be attributed to several potential obstacles: sparsity in marker effects, a large number of weak signals, and non-linear effects. Single marker analyses often lack power to select informative markers and typically do not account for non-linearity. One approach to gain predictive power and efficiency is to group markers based on biological knowledge such genetic pathways or gene structure. In Chapter 2 we propose and theoretically justify a multi-stage method for risk assessment that imposes a naive bayes kernel machine (KM) model to estimate gene-set specific risk models, and then aggregates information across all gene-sets by adaptively estimating gene-set weights via a regularization procedure. In Chapter 3 we extend these methods to meta-analyses by introducing sampling-based weights in the KM model. This permits building risk prediction models with multiple studies that have heterogeneous sampling schemes Advisors/Committee Members: Cai, Tianxi (advisor).

Subjects/Keywords: biostatistics; high dimensional data; kernel machine learning; prediction; statistical genetics; statistics

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Minnier, J. (2012). Inference and Prediction for High Dimensional Data via Penalized Regression and Kernel Machine Methods. (Doctoral Dissertation). Harvard University. Retrieved from http://nrs.harvard.edu/urn-3:HUL.InstRepos:9367010

Chicago Manual of Style (16^th Edition):

Minnier, Jessica. “Inference and Prediction for High Dimensional Data via Penalized Regression and Kernel Machine Methods.” 2012. Doctoral Dissertation, Harvard University. Accessed February 24, 2020. http://nrs.harvard.edu/urn-3:HUL.InstRepos:9367010.

MLA Handbook (7^th Edition):

Minnier, Jessica. “Inference and Prediction for High Dimensional Data via Penalized Regression and Kernel Machine Methods.” 2012. Web. 24 Feb 2020.

Vancouver:

Minnier J. Inference and Prediction for High Dimensional Data via Penalized Regression and Kernel Machine Methods. [Internet] [Doctoral dissertation]. Harvard University; 2012. [cited 2020 Feb 24]. Available from: http://nrs.harvard.edu/urn-3:HUL.InstRepos:9367010.

Council of Science Editors:

Minnier J. Inference and Prediction for High Dimensional Data via Penalized Regression and Kernel Machine Methods. [Doctoral Dissertation]. Harvard University; 2012. Available from: http://nrs.harvard.edu/urn-3:HUL.InstRepos:9367010

Harvard University

22. Sinnott, Jennifer Anne. Kernel Machine Methods for Risk Prediction with High Dimensional Data.

Degree: PhD, Biostatistics, 2012, Harvard University

URL: http://nrs.harvard.edu/urn-3:HUL.InstRepos:9793867

► Understanding the relationship between genomic markers and complex disease could have a profound impact on medicine, but the large number of potential markers can make… (more)

▼ Understanding the relationship between genomic markers and complex disease could have a profound impact on medicine, but the large number of potential markers can make it hard to differentiate true biological signal from noise and false positive associations. A standard approach for relating genetic markers to complex disease is to test each marker for its association with disease outcome by comparing disease cases to healthy controls. It would be cost-effective to use control groups across studies of many different diseases; however, this can be problematic when the controls are genotyped on a platform different from the one used for cases. Since different platforms genotype different SNPs, imputation is needed to provide full genomic coverage, but introduces differential measurement error. In Chapter 1, we consider the effects of this differential error on association tests. We quantify the inﬂation in Type I Error by comparing two healthy control groups drawn from the same cohort study but genotyped on different platforms, and assess several methods for mitigating this error. Analyzing genomic data one marker at a time can effectively identify associations, but the resulting lists of signiﬁcant SNPs or differentially expressed genes can be hard to interpret. Integrating prior biological knowledge into risk prediction with such data by grouping genomic features into pathways reduces the dimensionality of the problem and could improve models by making them more biologically grounded and interpretable. The kernel machine framework has been proposed to model pathway effects because it allows nonlinear associations between the genes in a pathway and disease risk. In Chapter 2, we propose kernel machine regression under the accelerated failure time model. We derive a pseudo-score statistic for testing and a risk score for prediction using genes in a single pathway. We propose omnibus procedures that alleviate the need to prespecify the kernel and allow the data to drive the complexity of the resulting model. In Chapter 3, we extend methods for risk prediction using a single pathway to methods for risk prediction model using multiple pathways using a multiple kernel learning approach to select important pathways and efﬁciently combine information across pathways. Advisors/Committee Members: Cai, Tianxi (advisor), Kraft, Peter (committee member), Mucci, Lorelei (committee member).

Subjects/Keywords: high dimensional data; kernel machines; pathways; risk prediction; biostatistics

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Sinnott, J. A. (2012). Kernel Machine Methods for Risk Prediction with High Dimensional Data. (Doctoral Dissertation). Harvard University. Retrieved from http://nrs.harvard.edu/urn-3:HUL.InstRepos:9793867

Chicago Manual of Style (16^th Edition):

Sinnott, Jennifer Anne. “Kernel Machine Methods for Risk Prediction with High Dimensional Data.” 2012. Doctoral Dissertation, Harvard University. Accessed February 24, 2020. http://nrs.harvard.edu/urn-3:HUL.InstRepos:9793867.

MLA Handbook (7^th Edition):

Sinnott, Jennifer Anne. “Kernel Machine Methods for Risk Prediction with High Dimensional Data.” 2012. Web. 24 Feb 2020.

Vancouver:

Sinnott JA. Kernel Machine Methods for Risk Prediction with High Dimensional Data. [Internet] [Doctoral dissertation]. Harvard University; 2012. [cited 2020 Feb 24]. Available from: http://nrs.harvard.edu/urn-3:HUL.InstRepos:9793867.

Council of Science Editors:

Sinnott JA. Kernel Machine Methods for Risk Prediction with High Dimensional Data. [Doctoral Dissertation]. Harvard University; 2012. Available from: http://nrs.harvard.edu/urn-3:HUL.InstRepos:9793867

Université Catholique de Louvain

23. Ballarini, Robin. Random intersection trees for genomic data analysis.

Degree: 2016, Université Catholique de Louvain

URL: http://hdl.handle.net/2078.1/thesis:4593

►

In Machine Learning classification, searching for informative interactions in large high-dimensional datasets is computationally intensive. Most algorithms that attempt this usually start with an empty… (more)

Subjects/Keywords: machine learning; classification; interactions; random intersection trees; high-dimensional data

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Ballarini, R. (2016). Random intersection trees for genomic data analysis. (Thesis). Université Catholique de Louvain. Retrieved from http://hdl.handle.net/2078.1/thesis:4593

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Ballarini, Robin. “Random intersection trees for genomic data analysis.” 2016. Thesis, Université Catholique de Louvain. Accessed February 24, 2020. http://hdl.handle.net/2078.1/thesis:4593.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Ballarini, Robin. “Random intersection trees for genomic data analysis.” 2016. Web. 24 Feb 2020.

Vancouver:

Ballarini R. Random intersection trees for genomic data analysis. [Internet] [Thesis]. Université Catholique de Louvain; 2016. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/2078.1/thesis:4593.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Ballarini R. Random intersection trees for genomic data analysis. [Thesis]. Université Catholique de Louvain; 2016. Available from: http://hdl.handle.net/2078.1/thesis:4593

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Virginia Tech

24. Sun, Jinhui. Robust Feature Screening Procedures for Mixed Type of Data.

Degree: PhD, Statistics, 2016, Virginia Tech

URL: http://hdl.handle.net/10919/73709

► High dimensional data have been frequently collected in many fields of scientific research and technological development. The traditional idea of best subset selection methods, which… (more)

▼ High dimensional data have been frequently collected in many fields of scientific research and technological development. The traditional idea of best subset selection methods, which use penalized L_0 regularization, is computationally too expensive for many modern statistical applications. A large number of variable selection approaches via various forms of penalized least squares or likelihood have been developed to select significant variables and estimate their effects simultaneously in high dimensional statistical inference. However, in modern applications in areas such as genomics and proteomics, ultra-high dimensional data are often collected, where the dimension of data may grow exponentially with the sample size. In such problems, the regularization methods can become computationally unstable or even infeasible. To deal with the ultra-high dimensionality, Fan and Lv (2008) proposed a variable screening procedure via correlation learning to reduce dimensionality in sparse ultra-high dimensional models. Since then many authors further developed the procedure and applied to various statistical models. However, they all focused on single type of predictors, that is, the predictors are either all continuous or all discrete. In practice, we often collect mixed type of data, which contains both continuous and discrete predictors. For example, in genetic studies, we can collect information on both gene expression profiles and single nucleotide polymorphism (SNP) genotypes. Furthermore, outliers are often present in the observations due to experimental errors and other reasons. And the true trend underlying the data might not follow the parametric models assumed in many existing screening procedures. Hence a robust screening procedure against outliers and model misspecification is desired. In my dissertation, I shall propose a robust feature screening procedure for mixed type of data. To gain insights on screening for individual types of data, I first studied feature screening procedures for single type of data in Chapter 2 based on marginal quantities. For each type of data, new feature screening procedures are proposed and simulation studies are performed to compare their performances with existing procedures. The aim is to identify a best robust screening procedure for each type of data. In Chapter 3, I combine these best screening procedures to form the robust feature screening procedure for mixed type of data. Its performance will be assessed by simulation studies. I shall further illustrate the proposed procedure by the analysis of a real example. Advisors/Committee Members: Du, Pang (committeechair), Deng, Xinwei (committee member), Hong, Yili (committee member), Kim, Inyoung (committee member).

Subjects/Keywords: ultra-high dimensional variable selection; feature screening; mixed type of data

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Sun, J. (2016). Robust Feature Screening Procedures for Mixed Type of Data. (Doctoral Dissertation). Virginia Tech. Retrieved from http://hdl.handle.net/10919/73709

Chicago Manual of Style (16^th Edition):

Sun, Jinhui. “Robust Feature Screening Procedures for Mixed Type of Data.” 2016. Doctoral Dissertation, Virginia Tech. Accessed February 24, 2020. http://hdl.handle.net/10919/73709.

MLA Handbook (7^th Edition):

Sun, Jinhui. “Robust Feature Screening Procedures for Mixed Type of Data.” 2016. Web. 24 Feb 2020.

Vancouver:

Sun J. Robust Feature Screening Procedures for Mixed Type of Data. [Internet] [Doctoral dissertation]. Virginia Tech; 2016. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/10919/73709.

Council of Science Editors:

Sun J. Robust Feature Screening Procedures for Mixed Type of Data. [Doctoral Dissertation]. Virginia Tech; 2016. Available from: http://hdl.handle.net/10919/73709

25. Wang, Xiaofei. Randomization test and correlation effects in high dimensional data.

Degree: MS, Department of Statistics, 2012, Kansas State University

URL: http://hdl.handle.net/2097/14039

► High-dimensional data (HDD) have been encountered in many fields and are characterized by a “large p, small n” paradigm that arises in genomic, lipidomic, and… (more)

Subjects/Keywords: Randomization test; Correlation effect; High dimensional data; Statistics (0463)

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Wang, X. (2012). Randomization test and correlation effects in high dimensional data. (Masters Thesis). Kansas State University. Retrieved from http://hdl.handle.net/2097/14039

Chicago Manual of Style (16^th Edition):

Wang, Xiaofei. “Randomization test and correlation effects in high dimensional data.” 2012. Masters Thesis, Kansas State University. Accessed February 24, 2020. http://hdl.handle.net/2097/14039.

MLA Handbook (7^th Edition):

Wang, Xiaofei. “Randomization test and correlation effects in high dimensional data.” 2012. Web. 24 Feb 2020.

Vancouver:

Wang X. Randomization test and correlation effects in high dimensional data. [Internet] [Masters thesis]. Kansas State University; 2012. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/2097/14039.

Council of Science Editors:

Wang X. Randomization test and correlation effects in high dimensional data. [Masters Thesis]. Kansas State University; 2012. Available from: http://hdl.handle.net/2097/14039

University of Colorado

26. Kaslovsky, Daniel N. Geometric Sparsity in High Dimension.

Degree: PhD, Mathematics, 2012, University of Colorado

URL: https://scholar.colorado.edu/math_gradetds/15

► While typically complex and high-dimensional, modern data sets often have a concise underlying structure. This thesis explores the sparsity inherent in the geometric structure… (more)

▼ While typically complex and high-dimensional, modern data sets often have a concise underlying structure. This thesis explores the sparsity inherent in the geometric structure of many high-dimensional data sets. Constructing an efficient parametrization of a large data set of points lying close to a smooth manifold in high dimension remains a fundamental problem. One approach, guided by geometry, consists in recovering a local parametrization (a chart) using the local tangent plane. In practice, the data are noisy and the estimation of a low-dimensional tangent plane in high dimension becomes ill posed. Principal component analysis (PCA) is often the tool of choice, as it returns an optimal basis in the case of noise-free samples from a linear subspace. To process noisy data, PCA must be applied locally, at a scale small enough such that the manifold is approximately linear, but at a scale large enough such that structure may be discerned from noise. We present an approach that uses the geometry of the data to guide our definition of locality, discovering the optimal balance of this noise-curvature trade-off. Using eigenspace perturbation theory, we study the stability of the subspace estimated by PCA as a function of scale, and bound (with high probability) the angle it forms with the true tangent space. By adaptively selecting the scale that minimizes this bound, our analysis reveals the optimal scale for local tangent plane recovery. Additionally, we are able to accurately and efficiently estimate the curvature of the local neighborhood, and we introduce a geometric uncertainty principle quantifying the limits of noise-curvature perturbation for tangent plane recovery. An algorithm for partitioning a noisy data set is then studied, yielding an appropriate scale for practical tangent plane estimation. Next, we study the interaction of sparsity, scale, and noise from a signal decomposition perspective. Empirical Mode Decomposition is a time-frequency analysis tool for nonstationary data that adaptively defines modes based on the intrinsic frequency scales of a signal. A novel understanding of the scales at which noise corrupts the otherwise sparse frequency decomposition is presented. The thesis concludes with a discussion of future work, including applications to image processing and the continued development of sparse representation from a geometric perspective. Advisors/Committee Members: Francois G. Meyer, James H. Curry, Per-Gunnar Martinsson, Gregory Beylkin, Thomas Manteuffel.

Subjects/Keywords: Geometry; High-dimensional data; Noise; Sparsity; Applied Mathematics

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Kaslovsky, D. N. (2012). Geometric Sparsity in High Dimension. (Doctoral Dissertation). University of Colorado. Retrieved from https://scholar.colorado.edu/math_gradetds/15

Chicago Manual of Style (16^th Edition):

Kaslovsky, Daniel N. “Geometric Sparsity in High Dimension.” 2012. Doctoral Dissertation, University of Colorado. Accessed February 24, 2020. https://scholar.colorado.edu/math_gradetds/15.

MLA Handbook (7^th Edition):

Kaslovsky, Daniel N. “Geometric Sparsity in High Dimension.” 2012. Web. 24 Feb 2020.

Vancouver:

Kaslovsky DN. Geometric Sparsity in High Dimension. [Internet] [Doctoral dissertation]. University of Colorado; 2012. [cited 2020 Feb 24]. Available from: https://scholar.colorado.edu/math_gradetds/15.

Council of Science Editors:

Kaslovsky DN. Geometric Sparsity in High Dimension. [Doctoral Dissertation]. University of Colorado; 2012. Available from: https://scholar.colorado.edu/math_gradetds/15

Louisiana State University

27. Kaur, Gurminder. Effective Visualization Approaches For Ultra-High Dimensional Datasets.

Degree: PhD, Databases and Information Systems, 2018, Louisiana State University

URL: https://digitalcommons.lsu.edu/gradschool_dissertations/4750

► Multivariate informational data, which are abstract as well as complex, are becoming increasingly common in many areas such as scientific, medical, social, business, and… (more)

▼ Multivariate informational data, which are abstract as well as complex, are becoming increasingly common in many areas such as scientific, medical, social, business, and so on. Displaying and analyzing large amounts of multivariate data with more than three variables of different types is quite challenging. Visualization of such multivariate data suffers from a high degree of clutter when the numbers of dimensions/variables and data observations become too large. We propose multiple approaches to effectively visualize large datasets of ultrahigh number of dimensions by generalizing two standard multivariate visualization methods, namely star plot and parallel coordinates plot. We refine three variants of the star plot, which include overlapped star plot, shifted origin plot, and multilevel star plot by embedding distribution plots, displaying dataset in groups, and supporting adjustable positioning of the star axes. We introduce a bifocal parallel coordinates plot (BPCP) based on the focus + context approach. BPCP splits vertically the overall rendering area into the focus and context regions. The focus area maps a few selected dimensions of interest at sufficiently wide spacing. The remaining dimensions are represented in the context area in a compact way to retain useful information and provide the data continuity. The focus display can be further enriched with various options, such as axes overlays, scatterplot, and nested PCPs. In order to accommodate an arbitrarily large number of dimensions, the context display supports the multi-level stacked view. Finally, we present two innovative ways of enhancing parallel coordinates axes to better understand all variables and their interrelationships in high-dimensional datasets. Histogram and circle/ellipse plots based on uniform and non-uniform frequency/density mappings are adopted to visualize distributions of numerical and categorical data values. Color-mapped axis stripes are designed in the parallel coordinates layout so that correlations can be fully realized in the same display plot irrespective of axes locations. These colors are also propagated to histograms as stacked bars and categorical values as pie charts to further facilitate data exploration. By using the datasets consisting of 25 to 130 variables of different data types we have demonstrated effectiveness of the proposed multivariate visualization enhancements.

Subjects/Keywords: Computer Visualization; Exploratory Analysis; Multivariate Data Visualization; Ultra-High Dimensional Datasets

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Kaur, G. (2018). Effective Visualization Approaches For Ultra-High Dimensional Datasets. (Doctoral Dissertation). Louisiana State University. Retrieved from https://digitalcommons.lsu.edu/gradschool_dissertations/4750

Chicago Manual of Style (16^th Edition):

Kaur, Gurminder. “Effective Visualization Approaches For Ultra-High Dimensional Datasets.” 2018. Doctoral Dissertation, Louisiana State University. Accessed February 24, 2020. https://digitalcommons.lsu.edu/gradschool_dissertations/4750.

MLA Handbook (7^th Edition):

Kaur, Gurminder. “Effective Visualization Approaches For Ultra-High Dimensional Datasets.” 2018. Web. 24 Feb 2020.

Vancouver:

Kaur G. Effective Visualization Approaches For Ultra-High Dimensional Datasets. [Internet] [Doctoral dissertation]. Louisiana State University; 2018. [cited 2020 Feb 24]. Available from: https://digitalcommons.lsu.edu/gradschool_dissertations/4750.

Council of Science Editors:

Kaur G. Effective Visualization Approaches For Ultra-High Dimensional Datasets. [Doctoral Dissertation]. Louisiana State University; 2018. Available from: https://digitalcommons.lsu.edu/gradschool_dissertations/4750

28. Suyundikov, Anvar. Statistical Dependence in Imputed High-Dimensional Data for a Colorectal Cancer Study.

Degree: PhD, Mathematics and Statistics, 2015, Utah State University

URL: https://digitalcommons.usu.edu/etd/4371

► The main purpose of this dissertation was to examine the statistical dependence of imputed microRNA (miRNA) data in a colorectal cancer study. The dissertation… (more)

Subjects/Keywords: Statistical Dependence; High-Dimensional Data; Colorectal Cancer Study; Mathematics

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Suyundikov, A. (2015). Statistical Dependence in Imputed High-Dimensional Data for a Colorectal Cancer Study. (Doctoral Dissertation). Utah State University. Retrieved from https://digitalcommons.usu.edu/etd/4371

Chicago Manual of Style (16^th Edition):

Suyundikov, Anvar. “Statistical Dependence in Imputed High-Dimensional Data for a Colorectal Cancer Study.” 2015. Doctoral Dissertation, Utah State University. Accessed February 24, 2020. https://digitalcommons.usu.edu/etd/4371.

MLA Handbook (7^th Edition):

Suyundikov, Anvar. “Statistical Dependence in Imputed High-Dimensional Data for a Colorectal Cancer Study.” 2015. Web. 24 Feb 2020.

Vancouver:

Suyundikov A. Statistical Dependence in Imputed High-Dimensional Data for a Colorectal Cancer Study. [Internet] [Doctoral dissertation]. Utah State University; 2015. [cited 2020 Feb 24]. Available from: https://digitalcommons.usu.edu/etd/4371.

Council of Science Editors:

Suyundikov A. Statistical Dependence in Imputed High-Dimensional Data for a Colorectal Cancer Study. [Doctoral Dissertation]. Utah State University; 2015. Available from: https://digitalcommons.usu.edu/etd/4371

University of Waterloo

29. Wang, Xinghao. Conditional Scenario Generation with a GVAR Model.

Degree: 2016, University of Waterloo

URL: http://hdl.handle.net/10012/11108

► The stress-testing method formed an integral part of the practice of risk management. However, the underlying models for scenarios generation have not been much studied… (more)

Subjects/Keywords: Stress-testing; Conditional Scenario Generation; High-dimensional Data; GVAR

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Wang, X. (2016). Conditional Scenario Generation with a GVAR Model. (Thesis). University of Waterloo. Retrieved from http://hdl.handle.net/10012/11108

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16^th Edition):

Wang, Xinghao. “Conditional Scenario Generation with a GVAR Model.” 2016. Thesis, University of Waterloo. Accessed February 24, 2020. http://hdl.handle.net/10012/11108.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7^th Edition):

Wang, Xinghao. “Conditional Scenario Generation with a GVAR Model.” 2016. Web. 24 Feb 2020.

Vancouver:

Wang X. Conditional Scenario Generation with a GVAR Model. [Internet] [Thesis]. University of Waterloo; 2016. [cited 2020 Feb 24]. Available from: http://hdl.handle.net/10012/11108.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Wang X. Conditional Scenario Generation with a GVAR Model. [Thesis]. University of Waterloo; 2016. Available from: http://hdl.handle.net/10012/11108

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Wayne State University

30. Li, Yan. Novel Regression Models For High-Dimensional Survival Analysis.

Degree: PhD, Computer Science, 2016, Wayne State University

URL: https://digitalcommons.wayne.edu/oa_dissertations/1555

► Survival analysis aims to predict the occurrence of specific events of interest at future time points. The presence of incomplete observations due to censoring… (more)

▼ Survival analysis aims to predict the occurrence of specific events of interest at future time points. The presence of incomplete observations due to censoring brings unique challenges in this domain and differentiates survival analysis techniques from other standard regression methods. In this thesis, we propose four models to deal with the high-dimensional survival analysis. Firstly, we propose a regularized linear regression model with weighted least-squares to handle the survival prediction in the presence of censored instances. We employ the elastic net penalty term for inducing sparsity into the linear model to effectively handle high-dimensional data. As opposed to the existing censored linear models, the parameter estimation of our model does not need any prior estimation of survival times of censored instances. The second model we proposed is a unified model for regularized parametric survival regression for an arbitrary survival distribution. We employ a generalized linear model to approximate the negative log-likelihood and use the elastic net as a sparsity-inducing penalty to effectively deal with high-dimensional data. The proposed model is then formulated as a penalized iteratively reweighted least squares and solved using a cyclical coordinate descent-based method.Considering the fact that the popularly used survival analysis methods such as Cox proportional hazard model and parametric survival regression suffer from some strict assumptions and hypotheses that are not realistic in many real-world applications. we reformulate the survival analysis problem as a multi-task learning problem in the third model which predicts the survival time by estimating the survival status at each time interval during the study duration. We propose an indicator matrix to enable the multi-task learning algorithm to handle censored instances and incorporate some of the important characteristics of survival problems such as non-negative non-increasing list structure into our model through max-heap projection. And the proposed formulation is solved via an Alternating Direction Method of Multipliers (ADMM) based algorithm. Besides above three methods which aim at solving standard survival prediction problem, we also propose a transfer learning model for survival analysis. During our study, we noticed that obtaining sufficient labeled training instances for learning a robust prediction model is a very time consuming process and can be extremely difficult in practice. Thus, we proposed a Cox based model which uses the L2,1-norm penalty to encourage source predictors and target predictors share similar sparsity patterns and hence learns a shared representation across source and target domains to improve the model performance on the target task. We demonstrate the performance of the proposed models using several real-world high-dimensional biomedical benchmark datasets and our experimental results indicate that our model outperforms other state-of-the-art related competing methods and attains very competitive performance on… Advisors/Committee Members: Chandan K. Reddy.

Subjects/Keywords: High-dimensional data; Regularization; sparsity; Survival Analysis; Computer Sciences

Record Details Similar Records Cite « Share

❌

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6^th Edition):

Li, Y. (2016). Novel Regression Models For High-Dimensional Survival Analysis. (Doctoral Dissertation). Wayne State University. Retrieved from https://digitalcommons.wayne.edu/oa_dissertations/1555

Chicago Manual of Style (16^th Edition):

Li, Yan. “Novel Regression Models For High-Dimensional Survival Analysis.” 2016. Doctoral Dissertation, Wayne State University. Accessed February 24, 2020. https://digitalcommons.wayne.edu/oa_dissertations/1555.

MLA Handbook (7^th Edition):

Li, Yan. “Novel Regression Models For High-Dimensional Survival Analysis.” 2016. Web. 24 Feb 2020.

Vancouver:

Li Y. Novel Regression Models For High-Dimensional Survival Analysis. [Internet] [Doctoral dissertation]. Wayne State University; 2016. [cited 2020 Feb 24]. Available from: https://digitalcommons.wayne.edu/oa_dissertations/1555.

Council of Science Editors:

Li Y. Novel Regression Models For High-Dimensional Survival Analysis. [Doctoral Dissertation]. Wayne State University; 2016. Available from: https://digitalcommons.wayne.edu/oa_dissertations/1555

Open Access Theses and Dissertations